Calibration and validation of accelerometry to measure physical activity in adult clinical groups: A systematic review.

A growing body of research calibrating and validating accelerometers to classify physical activity intensities has led to a range of cut-points. However, the applicability of current calibration protocols to clinical populations remains to be addressed. The aim of this review was to evaluate the accuracy of the methods for calibrating and validating of accelerometers to estimate physical activity intensity thresholds for clinical populations. Six databases were searched between March and July to 2017 using text words and subject headings. Studies developing moderate-to-vigorous intensity physical activity cut-points for adult clinical populations were included. The risk of bias was assessed using the health measurement instruments and a specific checklist for calibration studies. A total of 543,741 titles were found and 323 articles were selected for full-text assessment, with 11 meeting the inclusion criteria. Twenty-three different methods for calibration were identified using different models of ActiGraph and Actical accelerometers. Disease-specific cut-points ranged from 591 to 2717 counts·min-1 and were identified for two main groups of clinical conditions: neuromusculoskeletal disorders and metabolic diseases. The heterogeneity in the available clinical protocols hinders the applicability and comparison of the developed cut-points. As such, a mixed protocol containing a controlled laboratory exercise test and activities of daily-life is suggested. It is recommended that this be combined with a statistical approach that allows for adjustments according to disease severity or the use of machine learning models. Finally, this review highlights the generalisation of cut-points developed on healthy populations to clinical populations is inappropriate.

Background

Physical activity (PA) is defined as any bodily movement that requires an energy expenditure above resting (Caspersen et al., 1985). Regular PA has been associated with the prevention and treatment of a range of diseases, such as cardiovascular disease (Li et al., 2013), type II diabetes (Colberg et al., 2010), osteoporosis (McMillan et al., 2017, Senderovich et al., 2017) and breast cancer (Goncalves et al., 2014). However, 31% of adults are inactive, making physical inactivity a major international public health concern (Hallal et al., 2012).

Although accelerometers are capable of measuring raw acceleration at high sampling frequencies, the majority of studies rely on cut-points to classify PA intensities. Consequently, a growing body of calibration studies has led to a range of cut-points to classify PA intensities in adults (Freedson et al., 1998, Troiano et al., 2008), with little consensus as to the optimal cut-points or their applicability to populations other than those in which they were developed. Indeed, inter-study comparisons and cut-point generalisability are limited by a lack of standardisation of methodologies. Specifically, considerable variation in calibration protocols has arisen due, at least in part, to the progression from uni- to tri-axial accelerometry, the growing range of accelerometer models available and the broad range of configuration options (e.g., epoch, frequency). Furthermore, inter-study discrepancies in moderate-to-vigorous physical activity (MVPA) cut-points may also be attributable to variations in the criterion measures adopted and to the specific calibration protocol utilised; calibration protocols may range from a laboratory-based treadmill or walking protocol (Freedson et al., 2011) to a field-based protocol (Payey et al., 2017), or a combination of both (Midorikawa et al., 2017). Finally, the statistical approach used to translate activity counts into thresholds aligned with the criterion varies considerably between studies, with little evidence currently available regarding the comparability of different statistical methods.

A key question that remains to be addressed is the applicability of current calibration protocols to clinical populations. Specifically, physiological and biomechanical differences, common in many chronic conditions such as Chronic Obstructive Pulmonary Disease and Parkinson’s Disease (PD), may result in a higher cost of breathing or daily living activities and altered resting metabolic rate (RMR) demands (Bell et al., 1996, Goldstein et al., 1987, Levi et al., 1990, Psota and Chen, 2013, Sandroff et al., 2014a, Serra et al., 2016). Subsequently, cut-points developed for healthy populations are unlikely to appropriately reflect the activity levels of those with such diseases (McGinley et al., 2015, Serra et al., 2017) and population-specific cut-points, accounting for condition-specific energy expenditure (EE), are warranted. For example, applying cut-points developed on healthy populations was shown to be inappropriate for some clinical conditions, such as chronic stroke (Serra et al., 2017) and type II diabetes (McGinley et al., 2015). However, whilst accelerometry seems to be valid for some clinical conditions (Clarke et al., 2017), the development of population-specific cut-points was shown to improve the accuracy of the PA measurement in multiple sclerosis (MS) and in obese populations (Valenti et al., 2014). Given this lack of consensus, a synthesis of currently available cut-points, and calibration protocols, in clinical populations could afford valuable information for future clinical physical activity research.

Therefore, the aim of this systematic review was to describe current protocols utilised for the calibration of accelerometry to estimate MVPA thresholds for adult clinical populations. Secondly, the purpose was to provide recommendations for future studies seeking to calibrate accelerometers for clinical conditions in adults.

Methods

This review was performed according to the Preferred Reporting Items for Systematic Review and Meta-Analysis statement (Liberati et al., 2009, Moher et al., 2015), and registered on the International Prospective Register of Systematic Review (PROSPERO registration ID: CRD42016053880).

Search methods

The search was performed between March and July 2017 using six databases (PubMed, SPORTDiscus, ScienceDirect, Scopus, ISI Web of Knowledge, Wiley Online Library). Further details regarding the full search can be found on the Web-Appendix. The protocol was revised by an experienced librarian and a pilot was performed to assure feasibility. The search terms were in accordance with the 2017 Medical Subject Headings and were inserted as keywords to all the databases as follows: acceleromet*; acceleromet* AND (validation OR calibration); acceleromet* AND physical activity; wearable monitors AND (calibration OR validation); physical activity AND (calibration OR validation); acceleromet* thresholds; acceleromet* (cut-points OR cut-points); energy expenditure AND acceleromet*; and classification AND physical activity intensities. To check for any further studies meeting the inclusion criteria, the reference list of all the included studies and any systematic reviews on a similar topic were examined.

Eligibility criteria

In order to be included, studies needed to be published in or after the year 2000 in English and generate MVPA cut-points for accelerometry in adults with any chronic clinical condition. Chronic conditions were considered any long-term disease with slow progression (Goodman et al., 2013). Book chapters, theses, monographs, dissertations, abstracts, non-human, unpublished and non-English studies were not included. Studies using accelerometers associated with other technologies (e.g. microcontroller), calibrating for healthy population, sedentary behaviour or conditions that required a dispositive for gait (e.g. wheelchair), were excluded.

Data extraction and management

Following the creation of an EndNote X7 (Clarivate Analytics, US) database of potential studies, the lead author screened all the studies based on their titles and abstracts. Where any discrepancies on paper inclusion arose, a second author was available to consult to reach a consensus. All full texts were subsequently independently screened by two authors (MAM and KAM) according to the pre-established criteria. Studies that generated more than one MVPA cut-point were analysed as separate studies since protocols using multiple accelerometers or calibrations in different populations (e.g., different diseases) might lead to different MVPA thresholds. Supplementary information for each study was consulted when available or necessary for data extraction. No additional data was provided after consulting the authors. Data was extracted by the first author (MSB) and cross-checked by two co-authors (KAM and MAM). Further details of the data extraction are presented in Table 1. The risk of bias in individual studies was assessed by two authors (MSB and MAM), independently, using a checklist that was specifically tailored for calibration of accelerometry protocols (Table 2) based on previous literature (Bassett et al., 2012, Freedson et al., 2005, Lyden et al., 2014, Welk, 2005). This checklist rates studies as good, fair or poor for six elements of the calibration protocol (sample characteristics, accelerometry settings, criterion, statistical approach for calibration, and statistical approach for validation). Studies scoring poor for all the sections were excluded in order to prevent potentially biased and skewed results (Kane et al., 2017). Inter-rater reliability was determined by using Kappa scores, with 0.8 considered the minimum acceptable inter-rater agreement (McHugh, 2012). Following the risk assessment, all three authors discussed any discrepancies until a consensus had been reached.

Table 1

Summary of the extracted from the included studies.

Data data extraction field	Information extracted
Context and participants	The author, year and sample size of the study; participant characteristics, such as age, health status, height, weight, BMI, ethnicity; and covariates measured, such as self-report questionnaire data, health scales related to disease assessments
Study design and methods used	Any information related to the accelerometer, such as accelerometer model (e.g., number of axes); accelerometer placement (e.g., wrist [dominant/non-dominant], hip, chest); accelerometer settings (e.g., epoch, sampling frequency, use of low frequency filter); and data processing decisions (e.g., wear-time criteria) were extracted. Additionally, any information related to the calibration protocol, such as protocol design (e.g., laboratory-based, field-based, daily-life protocol); duration of the protocol; adjustment of specific variables (e.g., age, body mass); performance of individual calibration; criterion anchoring (e.g., energy expenditure, direct observation, heart rate); resting metabolic rate assessment; statistical approach (e.g., ROC-curve analyses, linear regression, machine learning); validation method (e.g., validation, cross-validation leave-one-out, cross-validation k-fold); and assessment for agreement (e.g., Kappa, Bland-Altman)
Findings	The extracted outcomes were protocol design and cut-points. All the extracted protocols were classified in four categories: laboratory-based (walking or running, over-ground or on a treadmill), free-living (assessment of participant routine), daily-life (daily-life activities performed at the research site) and mixed (at least two of laboratory-based, free-living and daily-life) protocols
Quality of the study	Checklist rating for performing calibration for accelerometry in clinical adult population

Table 2

Guideline Rating for Performing Calibration for Accelerometry in Clinical Adult Population (checklist).

Standard	Poor	Fair	Good
1. Sample Characteristics	Calibration study that did not provide any descriptive variables other than age and sex	Calibration study that assessed descriptive variables such as height, weight, body mass index and variables specific to the clinical condition	Calibration study that assessed descriptive variables such as height, weight, body mass index, ethnicity, resting metabolic rate and variables specific to the clinical condition
2. Accelerometry Settings	Study just described the accelerometer model	Study described the accelerometer model, number of axes and placement	Study described the accelerometer model, number of axes, placement, wear-time criteria (in case of free-living protocols), sampling frequency, epoch length and filtering procedures
3. Protocol Design	Study performed the calibration using a laboratory-based protocol composed only by walking or treadmill test	Study used a mixed protocol combining daily-life activities with a laboratory protocol test on a treadmill	Study used a mixed protocol combining daily-life activities with a laboratory protocol test on a treadmill and free-living assessments
4. Criterion	Used speed or direct observation to anchor the accelerometry counts	Used heart rate or metabolic equivalent to anchor the accelerometry counts	Used energy expenditure measures, considering resting metabolic rate* estimation, to anchor the accelerometry counts
5. Statistical Approach for Calibration	Study used group linear regression or individual linear regression to develop the cut-points	Study used ROC curve analyses to develop the cut-points	Study used machine learning techniques, hierarchical models or multilevel modelling, adjusting for factors related to participant characteristics and to the pathophysiology of the clinical condition to develop the cut-point(s)
6. Statistical Approach for Validation	Study did not perform a validation of the cut-points	Study performed a leave-one-out cross-validation of the cut-points and agreement using Bland-Altman or kappa score	Study performed a k-fold cross-validation using different samples and activities, determined agreement using Bland-Altman or Kappa score, and estimated the intraclass correlation coefficient, and / or limits of agreement

ROC: receiver operating characteristic.

*The criteria for a valid resting metabolic rate estimation was a minimum of 15 mins of steady state, preferably adopting the formula of de Weir (1948).

A narrative synthesis was performed covering each area of the protocol design: participant information; inclusion of disease-specific factors; accelerometry model and settings; protocol design; criterion; statistical approach for generating and validating MVPA cut-points.

Results

A total of 543,741 titles were identified from all databases, with 540,630 titles remaining after the removal of duplicates. Subsequently, the main author applied the eligibility criteria to all 540,630 titles and abstracts, which resulted in 619 articles remaining for full-text assessment. In total, 608 studies were excluded, primarily due the inclusion of healthy populations (279 studies), resulting in 11 studies involving a total of 488 participants aged 24 – 73 years being included in this review. Descriptive characteristics of the study samples are provided in Table 3. Twenty-three disease-specific MVPA thresholds for six different clinical conditions were identified. For the final synthesis, the six clinical conditions were stratified into either metabolic (n = 4; obesity, type II diabetes mellitus) or neuromusculoskeletal diseases (n = 7; MS, PD, down syndrome, chronic stroke) (Fig. 1).

Table 3

Summary of the included studies characteristics.

StudiesAuthor, year	ParticipantsSample size (n)Sex (male/female)Health statusControl GroupAge (range or mean ± SD)Height (mean ± SD)Weight (range or mean ± SD)BMI (range or mean ± SD)EthnicityCovariates	AccelerometerDevice ModelNumber of axesPlacementSampling frequencyFilterEpochMonitoring periodWear time	Calibration ProtocolPhysiological/ObservationalEE estimationRMR estimationIndividual calibrationProtocol typeDuration	Statistical ApproachCalibrationValidationAgreement	OutcomeCut-Points
Motl et al., 2009	n = 4840 females and 8 malesMultiple Sclerosis (n = 24)Control (n = 24)43.5 ± 12.2 years167.0 ± 11.6 cm76.7 ± 19.2 kgDemographic scalePatient Determined Disease Steps Scale	ActiGraph UniaxialRight hipEpoch: 30 s	Physiological: VO2RMR: 3.5 ml.kg−1Individual calibration: noProtocol type: laboratoryDuration: 30 min	Calibration: linear regressionValidation: noneAgreement: none	Cut-points (counts·min−1): Multiple Sclerosis:LPA: <591MPA: 591–6,460VPA: >6,460Control:LPA: <1,289MPA: 1,289–7,694VPA: >7,694
Lopes et al., 2009	n = 2615 females and 11 malesOverweight/obese/ Type 2 Diabetes MellitusControl = no62.6 ± 6.5 yearsCalibration group (n: 14):Male: 168.07 ± 5.18 cmFemale: 151.49 ± 8.54 cmMale: 80.32 ± 7.21 kg Female: 77.05 ± 21.03 kg31 ± 5.17 kg·m−2Obese: 57.1%Overweight: 42.9%CaucasiansHBA1c: 7.2 ± 1.8%Insulin: 9.6 ± 4.41 mg·dL−1HOMA-IR: 1.59 ± 0.71Validation group (n = 12):Male: 162.63 ± 3.54 cmFemale: 155.1 ± 7.99 cmMale: 75.9 ± 16.03 kgFemale: 72.19 ± 17.58 kg29.33 ± 4.85 kg·m−2Obese: 41.7%Overweight: 58.3%CaucasiansHBA1c: 7.34 ± 1.81%Insulin: 9.25 ± 4.47 mg·dL−1HOMA-IR: 1.53 ± 0.72	ActiGraphRight hipEpoch: 60 s	Physiological: VO2 and HRRMR: 15 min restIndividual calibration: noneProtocol type: laboratoryDuration: 30 min	Calibration: Hierarchal Model for equation and ROC for cut-pointsValidation: cross-validation for the regressionAgreement: concordance correlation coefficient	Cut-points (counts·min−1):SED / LPA: 200LPA / MPA: 1,240MPA / VPA: 2,400
Weikert et al., 2011	n = 2420 females and 4 malesMultiple SclerosisGroup with gait disability (n = 10)Group without gait disability (n = 14)Control: no42 ± 11.7 years20 Caucasian18 graduated from collegePatient-Determined Disease Steps − 1 (0–4)Multiple Sclerosis Walking Scale	ActiGraph (7164)UniaxialWaist nondominant hip10 HzEpoch: 1 s	Physiological: VO2Individual calibration: noneProtocol type: laboratoryDuration: 16 min	Calibration: linear regressionValidation: noneAgreement: none	Cut-points (counts·min−1):Overall:MPVA 2371 ± 847gait disability: 1,886 ± 739without gait disability: 2,717 ± 763
Aadland and Anderssen, 2012	n = 4231 females and 11 malesObesityControl: no43.2 ± 9.2 years172.2 ± 9.1 cm118.2 ± 18.2 kg39.8 ± 5.7 kg·m−2Waist circumference: 127.6 ± 13.2 cm	ActiGraph (GT1M)UniaxialRight hipNormal FilteringEpoch: 10 s	Physiological: VO2 and HRRMR: 1 h fast – 10 min in restIndividual calibration: yesProtocol type: laboratoryDuration: 40 min	Calibration: linear regression, linear mixed model and 1. ROC curve with high sensitivity and specificity and 2. ROC with high accuracy.Validation: cross-validationAgreement: none	Cut-points (counts·min−1):Linear regression:3 METS: 720Linear mixed model:3 METS: 612ROC 1:3 METS: 1,646ROC2:3 METS: 1,310Linear regression:6 METs: 5,779Linear mixed model:6 METs: 4,980ROC1:6 METs: 3,061ROC2:6 METs: 7,220
Aadland and Steene-Johannessen, 2012	n = 4231 females and 11 malesObesityControl: no43.2 ± 9.2 years172.2 ± 9.1 cm118.2 ± 18.2 kg39.8 ± 5.7 kg·m−2Waist circumference: 127.6 ± 13.2 cm	ActiGraph (GT1M)UniaxialRight hip and left hip (n: 22)Normal FilteringEpoch: 10 s	Physiological: VO2 and HRRMR: 1 h fast – 10 min in restIndividual calibration: yesProtocol type: laboratoryDuration: 40 min	Calibration: Linear regression (individual calibration) and mixed model (group calibration).Validation: noneAgreement: Bland-Altman	Cut-points (counts·min−1): Individual calibration:Right hip:MPVA: 1,078 Left hip:MPVA: 1,095
Agiovlasitis et al., 2012	n = 3821 females and 27 malesControl26.3 ± 5.2 years171.1 ± 8.2 cm73.4 ± 22.6 kg24.9 ± 7.4 kg·m−2n = 17Down Syndrome9 females24.7 ± 6.9 years154 ± 79 cm76.9 ± 16.8 kg32.6 ± 7.7 kg·m−2	ActiGraph (7164)UniaxialRight wristEpoch: 30 s	Physiological: VO2RMR: 3 h fast – 6 min rest in sitting position.Individual calibration: noneProtocol type: laboratoryDuration: 30 min	Calibration: multilevel modelling.Validation: noneAgreement: Bland-Altman	Cut-points (counts·min−1): Control:self-paced walking: 2,758 ± 1,3730.5 m/s: 714 ± 2790.75 m/s: 1036 ± 420;1 m/s: 1,992 ± 6691.25 m/s: 2,743 ± 1,1401.5 m/s: 3,185 ± 15683 METs: 1,526Down Syndrome:self-paced walking: 2888 ± 14680.5 m/s: 862 ± 4430.75 m/s: 1,712 ± 7471 m/s: 2,708 ± 1,0131.25 m/s: 4,052 ± 18621.5 m/s: 5,768 ± 28083 METs: 1,1376 METs: 4,525
Giffuni et al., 2012	n = 2917 females and 12 malesObese / overweight31.9 ± 9 years169.1 ± 8.3 cm100.8 ± 23.3 kg35.2 ± 7.6 kg·m−2ѴO2: 29.1 ± 11.5 ml·kg−1·min−1n = 25Control13 males26.1 ± 9.4 years174.3 ± 8.7 cm70 ± 10 kg23 ± 2.2 kg·m−2VO2: 40.8 ± 10.2 ml·kg−1·min−1	ActicalUniaxialMidline of the right tightEpoch: 60 s	Physiological: VO2RMR: 2 min restIndividual calibration: yesProtocol type: laboratoryDuration: 45 min	Calibration: Linear regression.Validation: noneAgreement: none	Cut-points (counts·min−1):Obese:3 METs: 1,9236 METs: 4,032Control:3MET: 1,7266MET: 4,117
Sandroff et al., 2012	n = 8676 females and 10 malesControl46.5 ± 10 years168.5 ± 8.9 cm75.4 ± 16.2 kgn = 43Multiple Sclerosis47.2 ± 9.1 years168.2 ± 8.3 cm75.7 ± 19.4 kgDemographic and exercise history questionnaires7DPAR26ft GAITRite mat Patient-Determined Steps12-item MS walking scale	ActiGraph (7164, GT3X)Uniaxial and triaxialNon-dominant hip30 HzEpoch: 15 s	Physiological: VO2Individual calibration: yesProtocol type: laboratoryDuration: 20 min	Calibration: linear regressionValidation: noneAgreement: none	Cut-points (counts·min−1):Multiple Sclerosis: MVPA: 1,723 ± 732Control:MVPA: 2,017 ± 801GT3X:Multiple Sclerosis: MVPA: 1,584 ± 697Control: 1,950 ± 852
Sandroff et al., 2014b	n = 5445 females and 9 malesMultiple SclerosisControl: no50.9 ± 9.2 years168.3 ± 7.6 cm82.3 ± 23 kg	ActiGraph (GT3X + )TriaxialFilter: Low frequency extensionEpoch: 60 s	Physiological: VO2RMR: 10–15 min restIndividual calibration: yesProtocol type: laboratoryDuration:	Calibration: individual regressionValidation: noneAgreement: none	Cut-points (counts·min−1):Vertical axis:Overall sample:MVPA: 1,754Mild and moderate disability:MVPA: 1,980Severe disability:MVPA: 1,185
Nero et al., 2015	n = 3013 females and 17 malesParkinson diseaseControl: no73 ± 5.4 years24.6 ± 3.3 kg·m−2Unified Parkinson’s Disease Rating Scale part II (UPDRS-ADL)Freezing of Gait Questionnaire (FOGQ)Borg and Perceived Exertion Scale	ActiGraph (GT3X + )TriaxialWaist30 HzFilter: normalEpoch: 15 s	Physiological: HR and speedsRMRIndividual calibrationProtocol type: laboratoryDuration: 9 min	Calibration: ROC curveValidation: leave-one-out cross-validationAgreement: Cohen’s Kappa	Cut-points (15 s):Vertical Axis:<1 ms: <328>1.3 m/s: >730Vector Magnitude:<1 ms: <470>1.3 m/s: >851
Serra et al., 2017	n = 2810 females and 18 malesChronic Stroke - chronic hemiparetic gaitControl: no60.4 ± 1.6 (47–83) years31.5 ± 1.1 (19–48) kg·m−243% Caucasian56% African-american6MWTLean mass (kg)	ActicalUniaxialnon-paretic hip	Physiological: VO2, HR, karvonen formula (HR reserve).RMR: 10 min restIndividual calibration: noneProtocol type: mixedDuration: 60 min	Calibration: Regression analysisValidation: noneAgreement: non	Cut-points (counts·min−1):SED/LPA: 125LPA/MPA: 667MPA/VPA: 1,546

SED: sedentary activity, LPA: light physical activity, MPA: moderate physical activity, VPA: vigorous physical activity, MVPA: moderate to vigorous physical activity, RMR: resting metabolic rate, VO2: oxygen uptake, HR: heart rate, ROC: receiver operating characteristic, ROC 1: ROC with best sensitivity and specificity, ROC 2: ROC with best accuracy definition, MET: metabolic equivalent.

Fig. 1

PRISMA flow presenting the systematic literature search.

Initially the reviewers achieved an inter-rater kappa score of 0.716 for the risk of bias assessment, with the criteria utilised to define RMR the main reasons for disagreement. Subsequently, MSB and MAM resolved discrepancies by discussing each point which resulted in a kappa score of 1 and the criteria to define RMR being specified in the checklist. The majority of the studies had high scores for sample characteristics and accelerometry settings (Table 4), with 5 studies classified as good, five as fair and two as poor for both criteria. Similar results were not found for protocol design, with 10 studies scoring as poor and one as fair. For physiological criterion, 9 studies were classified as fair, 1 as good and 1 as poor. Only two studies scored as good for statistical approach for calibration, with the majority classified as poor (n = 5) and fair (n = 4). Almost all studies were poor (n = 8) for statistical approach for validation, with only 3 studies classified as fair.

Table 4

Checklist risk of bias assessment results.

Study	Sample Characteristics	Accelerometry Settings	Protocol Design	Criterion	Statistical Approach for Calibrations	Statistical Approach for Validations
Motl et al., 2009	Fair	Fair	Poor	Fair	Poor	Poor
Weikert et al., 2011	Poor	Fair	Poor	Fair	Poor	Poor
Sandroff et al., 2012	Good	Good	Poor	Fair	Poor	Poor
Sandroff et al., 2014b	Fair	Good	Poor	Fair	Poor	Poor
Lopes et al., 2009	Good	Poor	Poor	Fair	Fair	Fair
Giffuni et al., 2012	Fair	Fair	Poor	Fair	Fair	Poor
Aadland and Anderssen, 2012	Good	Good	Poor	Fair	Good	Fair
Aadland and Steene-Johannessen, 2012	Good	Good	Poor	Fair	Fair	Poor
Agiovlasitis et al., 2012	Fair	Fair	Poor	Good	Good	Poor
Nero et al., 2015	Fair	Good	Poor	Poor	Fair	Fair
Serra et al., 2017	Good	Fair	Fair	Fair	Poor	Poor

Indirect calorimetry was the most common method (n = 10) used to estimate the physiological criterion (e.g. EE, METs or O2). Covariates were considered by nine studies, five of which utilised disease-specific assessments (e.g., Multiple Sclerosis Walking Scale). Among the studies including covariates, four either included disease-related factors in the analysis or investigated whether the inclusion of those variables would improve the model adopted for calibration. Four studies also included demographic factors in the analysis. Two studies investigated the relationship of the covariates through correlations with accelerometry derived counts·min−1.

Accelerometers

Thresholds were developed for 6 different accelerometers (Table 5); the majority were different models of ActiGraph (n = 9) (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Agiovlasitis et al., 2012, Motl et al., 2009, Nero et al., 2015, Sandroff et al., 2014a, Sandroff et al., 2014b, Weikert et al., 2011, Lopes et al., 2009) with the others using Actical (Giffuni et al., 2012, Serra et al., 2017). Seventeen of the MVPA cut-points were developed using a uni-axial accelerometer and six using a tri-axial accelerometer. The hip was the most common placement, adopted by nine studies to develop 22 MVPA cut-points. Nine of the MVPA cut-points were developed with the accelerometer placed on the right hip (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Giffuni et al., 2012, Lopes et al., 2009, Motl et al., 2009), seven on non-dominant hip (Sandroff et al., 2014a, Sandroff et al., 2014b), one on non-paretic hip (Serra et al., 2017), one on both hips (Aadland and Steene-Johannessen, 2012), two on the left hip (Aadland and Steene-Johannessen, 2012). One study placed the accelerometer on the right wrist (Agiovlasitis et al., 2012) and one did not specify the side (Nero et al., 2015). Reported sampling frequency varied from 10 Hz (Weikert et al., 2011) to 30 Hz (Nero et al., 2015, Sandroff et al., 2014a, Sandroff et al., 2014b), although eight studies did not report the sampling frequency used (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Agiovlasitis et al., 2012, Giffuni et al., 2012, Lopes et al., 2009, Motl et al., 2009, Sandroff et al., 2014b, Serra et al., 2017). Furthermore, only four studies described how they filtered the accelerometer data, with three (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Nero et al., 2015) using the standard filtering provided by the accelerometer software and one (Sandroff et al., 2014b) applying the low-filtering extension provided by ActiLife software. A wide variety of epoch lengths were used to develop the MVPA cut-points, with five studies using 60 s epochs (Giffuni et al., 2012, Lopes et al., 2009, Sandroff et al., 2014b, Serra et al., 2017, Weikert et al., 2011), followed by one using 10 s (Aadland and Steene-Johannessen, 2012), two studies using 15 s (Nero et al., 2015, Sandroff et al., 2014a, Sandroff et al., 2014b) and two using 30 s epochs (Agiovlasitis et al., 2012, Motl et al., 2009). The epoch length was extracted from MVPA cut-point unit (i.e. counts·min−1, counts per 15 s) when not specified in the methodology (Nero et al., 2015).

Table 5

Summary of Accelerometer Models Calibrated in the Included Studies.

Name/Model	Manufacturer	Dimensions (Weight and Size)	Memory Capacity	Axis	Frequency Sampling
ActiGraph 7164 (CSA)	ActiGraph LLC Pensacola, FL	45,5g5.1 × 4.1 × 1.5 cm	22 days of data with 60 s epoch	Uniaxial	10 Hz
GT1M ActiGraph	ActiGraph LLC Pensacola, FL	27 g3.8 × 3.7 × 1.8 cm	378 days using 60 s epoch	Biaxial	30 Hz
ActiGraph GT3X	ActiGraph LLC Pensacola, FL	27 g3.8 × 3.7 × 1.8 cm	378 days using 60 s epoch	Triaxial	30 Hz
ActiGraph GT3X+	ActiGraph LLC Pensacola, FL	19 g4.6 × 3.3 1.5 cm	38 days using 100 Hz	Triaxial	30–100 Hz
ActiGraph wGT3X+	ActiGraph LLC Pensacola, FL	19 g4.6 × 3.3 1.5 cm	38 days 100 Hz	Triaxial	30–100 Hz
Actical	Mini-Mitter Sunriver, OR	17.5 g2.8 × 2.7 × 1.0 cm	45d using 60 s epoch	Uniaxial	32 Hz

Calibration protocol settings

Laboratory-based protocols were utilised in 10 studies (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Agiovlasitis et al., 2012, Motl et al., 2009, Nero et al., 2015, Sandroff et al., 2014a, Sandroff et al., 2014b, Weikert et al., 2012, Giffuni et al., 2012, Lopes et al., 2009), with only one study (Lopes et al., 2009) applying a mixed protocol. Indirect calorimetry was performed by 10 of the studies (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Agiovlasitis et al., 2012, Giffuni et al., 2012, Lopes et al., 2009, Motl et al., 2009, Sandroff et al., 2014a, Sandroff et al., 2014b, Serra et al., 2017, Weikert et al., 2011), with one study (Nero et al., 2015, Serra et al., 2017) using both indirect calorimetry and HR and another using speed (Nero et al., 2015) and the duration of the protocol varied from 9 to 60 min. Indirect calorimetry was utilised as the physiological criterion by the majority of studies (n = 10). Specifically, six studies derived Metabolic Equivalents of Task (MET) from oxygen uptake (VO2), whereas four studies used the VO2 itself to determine the relationship with accelerometry counts. Four studies performed an individual calibration (Aadland and Steene-Johannessen, 2012, Giffuni et al., 2012, Sandroff et al., 2014a, Sandroff et al., 2014b), five performed a group calibration (Motl et al., 2009, Nero et al., 2015, Serra et al., 2017, Weikert et al., 2011, Lopes et al., 2009) and one study performed both (Aadland and Steene-Johannessen, 2012).

Statistical approach

Linear regression was the most common technique employed to generate eight MVPA cut-points in adult clinical populations (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Motl et al., 2009, Sandroff et al., 2014a, Sandroff et al., 2014b, Serra et al., 2017, Weikert et al., 2011, Weikert et al., 2012), followed by hierarchical modelling, generating four MVPA cut-points (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Agiovlasitis et al., 2012, Lopes et al., 2009), and receiver operating characteristic (ROC) analysis, developing five MVPA cut-points (Aadland and Anderssen, 2012, Nero et al., 2015, Giffuni et al., 2012). Thus, one study (Aadland and Anderssen, 2012) applied two different ROC models; the first model prioritized higher sensitivity (true positives/total positives) and specificity (true negatives/total negatives), whilst the second model used overall accuracy (true positives and true negatives/total positives and negatives). Ten studies (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Agiovlasitis et al., 2012, Giffuni et al., 2012, Lopes et al., 2009, Motl et al., 2009, Sandroff et al., 2014a, Sandroff et al., 2014b, Weikert et al., 2011) did not perform any kind of validation and one performed a leave-one-out cross-validation (Nero et al., 2015). Furthermore, most of the studies did not perform any agreement assessment (n = 8) (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Lopes et al., 2009, Motl et al., 2009, Sandroff et al., 2014a, Sandroff et al., 2014b, Serra et al., 2017, Weikert et al., 2011); one study performed Bland-Altman (Agiovlasitis et al., 2012) and one calculated the Kappa Score (Nero et al., 2015).

Outcome

All the disease-specific MVPA cut-points extracted from the included studies were integrated to a 60 s epoch to allow comparison between thresholds when not available in this format (Table 6). Most studies presented their cut-points in counts·min−1, despite using different epoch lengths for processing the activity counts. Disease-specific cut-points of MVPA varied from a minimum of 612 counts·min−1 to a maximum of 6,460 counts·min−1.

Table 6

Summary of MVPA Disease-specific Cut-points.

Disease (n*)	Study	Reason for split	Cut-points MVPA (original)	Cut-points MVPA converted to counts.min−1a	Criterion Validity
Multiple Sclerosis (7)	Motl et al., 2009	N/A	6460 (counts·min−1)	N/A	N/A
	Weikert et al., 2011	No Gait-disability Group	2717 (counts·min−1)	N/A	N/A
	Weikert et al., 2011	Overall Group (gait and non-gait-disability)	2371 (counts·min−1)	N/A	N/A
	Weikert et al., 2011	Gait-disability Group	1886 (counts·min−1)	N/A	N/A
	Sandroff et al., 2012	ActiGraph 7164	1723 (counts·min−1)	N/A	N/A
	Sandroff et al., 2012	ActiGraphGT3X	1584 (counts·min−1)	N/A	N/A
	Sandroff et al., 2014b	Overall Group (gait and non-gait-disability)	1745 (counts·min−1)	N/A	N/A
	Sandroff et al., 2014b	Gait-disability Group	1185 (counts·min−1)	N/A	N/A
	Sandroff et al., 2014b	No Gait-disability Group	1980 (counts·min−1)	N/A	N/A
Overweight/obesity/Type 2 Diabetes Mellitus (10)	Lopes et al., 2009	N/A	2400 (counts·min−1)	N/A	Concordance Correlation Coefficient: 0.8
	Giffuni et al., 2012	N/A	4032 (counts·min−1)	N/A	N/A
	Aadland and Anderssen, 2012	ROC 1	1646 (counts·min−1)	N/A	N/A
	Aadland and Anderssen, 2012	ROC 2	1310 (counts·min−1)	N/A	N/A
	Aadland and Steene-Johannessen, 2012	Individual Calibration/Linear Regression	1151 (counts·min−1)	N/A	Bland-Altman/LOA
	Aadland and Steene-Johannessen, 2012	Linear Regression/Left Hip	1095 (counts·min−1)	N/A	Bland-Altman/LOA
	Aadland and Steene-Johannessen, 2012	Linear Regression/Right Hip	1078 (counts·min−1)	N/A	Bland-Altman/LOA
	Aadland and Anderssen, 2012	OLR/Right Hip	720 (counts·min−1)	N/A	N/A
	Aadland and Anderssen, 2012	MIX REG/Left Hip	685 (counts·min−1)	N/A	Bland-Altman/LOA
	Aadland and Anderssen, 2012	MIX REG/Right Hip	612 (counts·min−1)	N/A	N/A
Down Syndrome (1)	Agiovlasitis et al., 2012	N/A	1137 (counts·min−1)	N/A	Bland-Altman/LOA
Parkinson disease (2)	Nero et al., 2015	N/A	730 (counts·15 s−1)	2980	Cross-validation: 74%–64% of agreement; Kappa Score: 0.79 for y axis and kappa score: 0.69 for VM.
	Nero et al., 2015		851 (counts·15 s−1)	3404
Chronic Stroke (1)	Serra et al., 2017	N/A	1546 (counts·min−1)	N/A	N/A

ROC: receiver operating characteristic, ROC 1: Roc with best sensitivity and specificity, ROC 2: ROC with better accuracy definition, OLR: Ordinary Linear Regression, MIX REG: Linear Mixed Model Regression. LOA: limits of agreement.

aConverted when not available.

Discussion

In total, 11 studies generating 23 MVPA cut-points in clinical conditions revealed a broad range of MVPA cut-points. Key recommendations for future studies are to include a variety of free-living activities that are applicable to the specific disease-population, of various intensities, and to ensure that a robust measure of EE and precise estimation of RMR are included to account for disease-related alterations.

Calibration protocol for clinical populations

Numerous factors should be considered in the development of a calibration protocol for clinical populations, including the inclusion of participant demographics and disease-related factors. Another key consideration in the development of cut-points for clinical populations is the addition of a physiological criterion to the calibration protocol, particularly related to energetic cost. Specifically, some conditions might be associated with an alteration in the daily total EE. This variation is likely to occur due to many factors, including impaired biomechanics (e.g., neuromusculoskeletal disorders), higher energetic cost of breathing (e.g., respiratory conditions) and disease severity and treatments (e.g., medications; (Bell et al., 1996, Psota and Chen, 2013, Sandroff et al., 2014a, Serra et al., 2016). Thus, numerous factors in addition to PA contribute to total EE, such as the thermal effect of food intake and RMR. Indeed, indirect calorimetry was shown to overestimate EE when the RMR was not properly assessed (Fares et al., 2008). Therefore, RMR estimation is highly recommended to avoid bias, particularly as it was shown to be altered in many clinical conditions (Agiovlasitis et al., 2012, Alawad et al., 2013, Gajewski et al., 2017, Mahler et al., 2012, Montaurier et al., 2007, Nawata et al., 2004, Serra et al., 2015, Wens et al., 2014). Alternatively, METs can be used to estimate EE; Serra et al. (Serra et al., 2017) found METs to be the strongest predictor of activity counts, despite explaining only 65% of the accelerometer activity counts. Thus, most of the included studies derived MET values from a measure of oxygen uptake, which arguably would encompass any possible alteration in energetic cost arising from the disease. However, careful consideration must be given when using METs due to the controversial nature of this method and its failure to represent clinical subgroups (McMurray et al., 2014). Indeed, the standard MET value of 3.5 ml·kg−1·min−1 was developed based on healthy populations and therefore does not reflect pathological, biomechanical, metabolic and respiratory adaptations which are common in many clinical conditions (Byrne et al., 2005).

Accelerometer setting and analysis description

Whilst hip was the most popular choice among the included studies (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Giffuni et al., 2012, Lopes et al., 2009, Motl et al., 2009, Sandroff et al., 2014a, Sandroff et al., 2014b, Serra et al., 2017), the best location for monitor placement in clinical populations is unclear. Indeed, comparisons of hip- and wrist-generated thresholds demonstrated great variability which may be explained by biomechanical differences related to dominance (Aadland and Steene-Johannessen, 2012) or functional adaptations due to clinical conditions (Lerner et al., 2014, Ling et al., 2012). For example, in PD, freezing of gait can lead to a rapid trembling in the legs, which would be more efficiently measured by an accelerometer placed on the lower limb (Suzuki et al., 2017). Similarly, other conditions affecting the gait biomechanics might benefit from hip or lower limb placements, as demonstrated under a free-living protocol for chronic stroke and MS patients (Rand et al., 2009, Sparaco et al., 2018).

The choice of accelerometry settings and signal processing should be described in the calibration protocol to allow comparability between studies and generalisability of the developed cut-points (Brond and Arvidsson, 2016). Nonetheless, five of the included studies did not report the sampling frequency and filtering methods used (Agiovlasitis et al., 2012, Giffuni et al., 2012, Lopes et al., 2009, Motl et al., 2009, Serra et al., 2017). The most popular choice of epoch was 60 s (Giffuni et al., 2012, Lopes et al., 2009, Sandroff et al., 2014a, Sandroff et al., 2014b, Serra et al., 2017, Weikert et al., 2011), with the majority of studies presenting MVPA in counts·min−1. Alternatively, the choice of 1 or 5 s epochs is appropriate to capture short bursts of activities and could be a suitable choice for free-living protocols or for analyses utilising pattern recognition (Gabriel et al., 2010, Staudenmayer et al., 2009a). Whilst counts·min−1 are commonly used, the units are somewhat arbitrary and lack direct practical meaning and transparency due to their proprietary nature (Sievanen and Kujala, 2017). Indeed, the brand-specific units limit inter-study comparisons. In contrast, the use of raw acceleration signals allow more complex analyses and, consequently, higher prediction accuracy (Montoye et al., 2018).

Protocol design

The calibration protocols were classified into four categories: laboratory-based protocols that involved walking or running on a treadmill; free-living protocols that assessed participants during their daily routines; daily-life protocols that involved daily-life activities in the laboratory and mixed protocols that utilised more than one of the previously described protocols. A free-living protocol is widely considered the most appropriate for calibration as it determines the relationship between EE and PA in an ecologically valid manner (Freedson et al., 2005, Mackintosh et al., 2012). Despite that, almost all the studies in the neuromusculoskeletal disease group (Agiovlasitis et al., 2012, Motl et al., 2009, Nero et al., 2015, Sandroff et al., 2014a, Sandroff et al., 2014b, Weikert et al., 2011) utilised over-ground walking protocols. Likewise, almost all of the studies in metabolic disease populations (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Giffuni et al., 2012, Lopes et al., 2009) performed treadmill walking protocols, with only one study encompassing jogging. A limitation of such walking protocols is that they are unlikely to provide a fair classification of activities beyond those of locomotion (Crouter et al., 2006). In addition, studies suggest that individuals with chronic stroke and PD are more prone to adopt a different strategy to increase gait speed when walking on the treadmill (Lamontagne et al., 2016, Warlop et al., 2018). During treadmill ambulation, the lack of visual cues and a moving floor results in a cautious gait, with individuals adopting slower speeds and increased stride length compared to overground walking (Lamontagne et al., 2016, Warlop et al., 2018). As such, the use of treadmill to calibrate for such populations may result in a misrepresentation of their gait during daily-life and should be considered with caution. Alternatively, a free-living protocol would be the ideal framework to provide a more ecologically valid measure of PA (Welk, 2005) in clinical populations.

The statistical approach adopted to translate activity counts and EE into cut-points could substantially impact the derived thresholds. For example, whilst linear regression has been most widely used (Aadland and Anderssen, 2012, Aadland and Steene-Johannessen, 2012, Motl et al., 2009, Sandroff et al., 2014a, Sandroff et al., 2014b, Serra et al., 2017, Weikert et al., 2011), it assumes that the relationship between activity counts and metabolic data (i.e. VO2, METs) is linear. To address this issue, recent calibration studies have incorporated more flexible statistical methods, such as ROC analysis, hierarchical models, and machine learning (Bassett et al., 2000, Crouter et al., 2011, Freedson et al., 2005, Montoye et al., 2017). However, in the context of clinical populations, it is pertinent to note that ROC analysis does not allow adjustment for clinical factors and may therefore not be an optimal approach.

Machine learning and pattern recognition have been identified as the optimal methods for classifying PA (Bonomi et al., 2009, Staudenmayer et al., 2015, Staudenmayer et al., 2009b, Welk, 2005). A recent systematic review highlighted the high predictive accuracy of laboratory-calibrated protocols using machine learning models (Farrahi et al., 2019), with hidden markov models (Pober et al., 2006), decision trees (Mathie et al., 2004) and artificial neural networks (Staudenmayer et al., 2009b) the most common models used to estimate PA from raw accelerometry signals. Indeed, the use of such models improved PA prediction, overcoming the inherent limitations of using static epoch lengths (Montoye et al., 2018). Whilst promising, the use of machine learning to estimate PA from raw accelerations is still in the early phases of development. Specifically, the reproducibility of machine learning approaches in free-living settings requires further investigation (Kerr et al., 2016). Additionally, machine learning models often require considerably sized training data sets, particularly deep learning, which might be a challenge when using indirect calorimetry (Mannini and Sabatini, 2010). Future studies calibrating accelerometry for clinical populations should consider using machine learning in order to achieve higher prediction accuracy and promote advancements in the field. However, it is noteworthy that even complex statistical approaches such as pattern recognition would still require an optimised calibration protocol in order to ensure high prediction accuracy. In addition, other statistical approaches should also be considered, such as probability analysis which has been employed to translate activity counts into PA behavioural data in mental illness patients (Chapman, 2017).

Cross-validation establishes the validity of the developed cut-points and verifies that the thresholds are applicable across any participant of similar age and health status to the sample it was generated from. Whilst it is recommended that a cross-validation should be conducted utilising an independent sample and different activities (Welk, 2005), the use of a leave-one-out-approach can also be considered. For example, Nero et al., 2015 used a leave-one-out approach to cross-calibrate the specific PD cut-points. Additionally, a measure of agreement should be performed in addition to a cross-validation (Lopes et al., 2009), and the cross-validation should be applied after developing the thresholds and not as a robustness check prior to the analysis (Aadland and Anderssen, 2012). Future studies should continue to cross-validate the disease-specific thresholds to ensure their reliability and validity across different protocols and clinical stages.

Outcome: MVPA cut-points

Disease-specific cut-points are essential in understanding and promoting PA in clinical populations. The majority of the MVPA cut-points developed for clinical populations were different to those previously developed for healthy adults (Freedson et al., 1998); disease-specific MVPA cut-points varied greatly, from 612 counts·min−1 to 6460 counts·min−1, even within the same condition. Indeed, Serra et al. (2017) developed Actical MVPA cut-points for stroke patients that were equivalent to LPA cut-points for general population. This large variability can be attributed to the occurrence of gait impairment at advanced stages of the disease, in addition to differences in treatments and medications. However, a control group is warranted in order to ascertain whether any variation in the cut-points is caused by the pathophysiology of the disease or differences in the calibration protocol. Indeed, whilst a control group is highly beneficial in the interpretation of the findings of each study, cut-points previously established for general populations could be used when necessary to investigate whether the use of the disease-specific cut-points enhances the predictive accuracy (Janssen et al., 2015, Trost et al., 2015).

Strengths and limitations

It is important to acknowledge that the search protocol was developed with a subject-specific librarian, following a rigorous iterative process. Specifically, initial pilot searches were conducted to assess the feasibility of the initial criteria and search terms. Revisions were subsequently made to the outcomes, risk of bias assessment and final analyses. Moreover, extensive screening was performed by the first author to capture all calibration studies, irrespective of healthy or clinical status, to ensure that no clinical calibration studies were missed. Whilst this review is associated with numerous strengths, there are, nonetheless, limitations. Firstly, only studies generating MVPA cut-points were included; whilst cut-points are still widely used in PA research, major limitations associated with this practice should be acknowledged. The large variability of intensity-related cut-points also occurs among general population (Reilly et al., 2008), causing what Trost (2007) described as the ‘cut-point conundrum’. This discrepancy is multifactorial, arising in part from the lack of standardization of calibration protocols and broad range of statistical approaches applied to reduce accelerometer data to cut-points. It is also important to acknowledge the high risk of bias encountered within the included studies which limits our conclusions. Finally, it is noteworthy that the present recommendations were based on a relatively small range of clinical conditions, further demonstrating the need for more population-specific calibration protocols.

Conclusion

This systematic review highlights the large variability in MVPA cut-points developed for clinical populations. Indeed, a lack of standardisation in the protocol design, as well as the statistical approach, makes it impossible to compare disease-specific cut-points to those generated for healthy populations. To ensure ecological validity, future calibration protocols should incorporate a large variety of free-living activities, of various intensities, instead of protocols composed predominantly of walking. Moreover, future research should ensure a robust measure of EE is adopted as the criterion measure for accelerometry, as well as a precise estimation of RMR. Studies incorporating a control group and utilising cross-validation of the developed clinical thresholds are warranted. Finally, whilst standardization is necessary, it is highly recommended that future studies consider the pathophysiology of the disease when designing the protocol.

Declarations

Ethics Approval and consent to participate: Not applicable.

Consent for publication: Not applicable.

Availability of data and material: Not applicable.

Authors’ contributions

MSB made substantial contributions to conception, design, systematic search, data analysis and interpretation, and drafted of the manuscript. MM and KM made substantial contributions to conception, design, systematic search, data analysis and interpretation, manuscript writing and critically revised the manuscript for important intellectual content. LL made substantial contribution to the conception of the initial protocol, development of methodology and input revising the manuscript critically. All the authors approved the final manuscript.

Funding

This review summarises independent research funded by the Cystic Fibrosis Trust UK under its programme grant for Strategic Research Centres (grant reference No. RP-PG-0108-10011). MSB is a funded PhD student by the Cystic Fibrosis Trust. The funder had no role in the conduct of the study, the writing of the manuscript, or the decision to submit it for publication.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.