Comparison of Self-Reported vs Objective Measures of Long-Term Community Ambulation in Lower Limb Prosthesis Users.

Objective: To determine normal variation in walking metrics in a population of lower limb amputees who use lower limb prostheses over a 6-month period and to provide a means to interpret clinically meaningful change in those community walking metrics. Design: Prospective cohort study monitoring walking behavior and subjective and objective measures of activity. Setting: Veterans Administration and university amputee clinics. Participants: 86 individuals with lower limb amputation who use protheses. Interventions: StepWatch activity monitor tracked subjects' walking for 24 weeks; Global Mobility Change Rating collected weekly. Main Outcome Measures: Association between change in Global Mobility Change Rating and change in any of the walking metrics.
Results: Walking metrics including step count, cadence, cadence variability, and walking distance in a population of lower limb prosthesis users were obtained. There was a high correlation in the walking metrics indicating higher function with higher functional classification level (K-levels) but also substantial overlap in all metrics and a very weak correlation between subject-reported activity level and objective measures of walking performance.
Conclusion: The overlap in walking metrics with all K-levels demonstrates that no single metric measured by StepWatch can determine K-level with 100% accuracy. As previously demonstrated in other populations, subjects' interpretations of their general activity level was inaccurate, regardless of their age or activity level. Objective measures of walking appear to provide a more accurate representation of patients' activity levels in the community than self-report. Therefore, objective measures of walking are useful in supporting K-level determinations. However, clinicians cannot rely on a single metric to determine K-level.

In the United States, nearly 1 in every 200 people have lost a limb (1.6 million). Much of rehabilitation after a lower limb amputation is focused on improving and maximizing walking ability, both at home and in the community. Community walking metrics such as steps per day and cadence can be helpful to rehabilitation clinicians in prescribing appropriate prostheses and designing appropriate rehabilitation interventions. However, little is known about the natural fluctuations in these walking metrics over time.

For example, the amount of change in walking metrics that corresponds to a clinically relevant change in walking function in unknown. Therefore, it is crucial when using community walking data to identify meaningful change rather than simple natural variations in walking pattern. This has profound implications for interpreting the effect of prosthetic components, rehabilitation, and other effects on walking. Without determining clinically relevant change in community metrics, the value of the data to inform patient care will be diminished.

As community-based metrics become more widely used by patients and clinicians, it will be important to understand how much variation is normal and expected in amputees of various functional classification level (K-levels) and what amount of variation should be considered clinically relevant. Previously, daily steps were used in patients with incomplete spinal cord injury to determine standard error of measurement and minimal detectable change. However, only steps taken during a 6-minute walk test and a 10-m walk test were evaluated. No community walking metrics were assessed for standard error of measurement, minimal detectable change, or clinically relevant difference. No study to date has focused on determining this in community walking metrics in an amputee population.

Studies in other populations have shown that patients are overall inaccurate at reporting their activity, and self-reported activity levels have poor correlation to objective measures.4, 5, 6, 7, 8 In fact, a systematic review found that self-reported measures of physical activity were both higher and lower than directly measured levels of physical activity. This is true when specifically measuring self-reported walking distances when compared with an accelerometer in individuals before and after joint arthroplasty and self-reported vs actual walking distance in individuals with multiple sclerosis. In lower limb prosthesis users, the majority of participants inaccurately self-reported low, medium, and high activity levels relative to objective measures of activity levels with no bias toward over- or underreporting. This may explain the growing support to incorporate objective outcome measures in addition to patient-reported outcome measures when evaluating the functional activity level of lower limb prosthesis users.

The StepWatch 3 Activity Monitor is a medical device and validated in a lower limb amputee populations and populations that include slow and/or abnormal walkers.,11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39 It measures not only daily steps but also peak performance index, walking distance, cadence, and cadence variability, as well as giving a functional-level assessment based on a proprietary algorithm that attempts to combine all of the metrics into one number to approximate the K-level (given as an number from 1 to 4). The functional-level algorithm was previously demonstrated to have good (R2=0.78) to excellent (r=0.96) agreement with clinically determined K-levels in cohorts of transtibial amputees. When comparing the metrics between week 1 and week 2 in the Godfrey et al study, there was an average of 5.3%±11% absolute change in the algorithm-derived K-level and 21%±15% absolute change in daily steps (mean and SD). Because of the potential broad differences in natural fluctuations of each community metric, it is anticipated that the amount each metric must change to represent a clinically relevant difference will vary depending on the metric.

The objective of this study was to determine normal variation in walking metrics in a population of lower limb prosthesis users over a 6-month period, to provide normative walking data in this population, and to provide a means to interpret clinically meaningful change in those community walking metrics.

Methods

After approval was obtained from the institutional review boards at the University of Utah and Salt Lake City VA and the Congressionally Directed Medical Research Program Human Studies committee, subjects were recruited from the University of Utah and Salt Lake City VA Amputee Clinics. When StepWatches were available for use, all patients who met inclusion criteria were invited to participate until study enrollment reached its target goal of n=100.

Inclusion criteria were individuals over the age of 21 who have at least 1 major lower limb amputation (defined as hip disarticulation, transfemoral amputation, knee disarticulation, transtibial amputation, or Syme amputation) from any cause, are at least 6 months post-amputation surgery, and use a lower limb prosthesis for transfers and/or walking. Exclusion criteria were anyone not able to read and understand English. After written informed consent was obtained, each subject had a StepWatch 3 Activity Monitor programmed for their walking gait and attached to their prosthesis just above the prosthetic foot in the lateral ankle region (figure 1). In addition, average stride length was measured by the subjects walking 10 strides, measuring the distance, and dividing the distance by 10 strides. K-level was taken from the subjects’ medical records as determined and recorded by their physical medicine and rehabilitation physician.

Fig 1

StepWatch activity monitor placed in typical location on prosthesis: lateral ankle region.

The subject was instructed to keep the StepWatch in that position. If the subject actively used more than 1 prosthesis for the same leg, a StepWatch was programmed for each prosthesis and the data were merged before generating the reports.

The StepWatch Activity Monitor records 50 days of data when collecting at step per minute intervals. Therefore, the subjects were mailed a new StepWatch each month with a shipping label to return the StepWatch with the previous month's data. Each subject's community walking metrics were averaged over each week. The algorithm required at least 5 days of usable data per recording period to calculate the metrics for a week. Five days per weekly recording period was the minimum necessary for the algorithm to calculate the metrics. Reasons for having less than 7 days was if the StepWatch was removed from the prosthesis or placed upside down during the week. At the end of each week, subjects were contacted by a study coordinator (via either email or phone call, depending on subject preference) to report whether their walking function changed compared to the previous week.

StepWatch measured the community metrics defined in table 1 daily steps, daily distance, cadence, cadence variability, and Modus Index. Cadence is measured as average daily steps per minute on the prosthetic limb; an increase in this metric indicates that the subject is walking at faster speeds and/or walker longer in continuous bouts. Cadence variability is measured as average daily standard deviation of each step per minute rate when walking (≥1 step per minute). An increase in cadence variability means that the subject has increased their range of walking cadences. The Modus Index is the only metric with a proprietary algorithm. It was previously validated to successfully distinguish functional K-levels in veterans with an amputation.

Table 1

Community metrics measured by StepWatch with Trex software

Metric: Definition
1) Modus Index: Overall walking function of the patient. It includes clinical observation K-level score, ambulation energy index, peak performance index, and cadence variability index.a. Ambulation energy: Algorithm that incorporates ambulation energy requirements and intensity of continuous walking bouts. b. Peak performance index: Algorithm that incorporates top 30 fastest 1-minute walking spurts achieved each day. c. Cadence variability index: Algorithm that incorporates proportion of walking at low (1-15 steps per minute), medium, (16-40 steps per minute), and high (≥41 steps per minute) cadence values.
2) Daily steps: Average daily steps taken with the prosthetic limb.
3) Daily distance: Estimated distance walked based on steps and user-defined stride length.
4) Cadence: Average daily steps per minute rate when walking. This is measured on the prosthetic limb only. Walking is defined as ≥1 step per minute. An increase in cadence indicates that the patient is walking at faster speeds and/or walking longer in continuous bouts.
5) Cadence variability: Average daily standard deviation of each step per minute rate when walking. Walking is defined as ≥1 step per minute. An increase in cadence variability means that the patient has increased their range of walking cadences.

Modus Index: Overall walking function of the patient. It includes clinical observation K-level score, ambulation energy index, peak performance index, and cadence variability index.

Subject perception of walking function change was assessed weekly via the Global Mobility Change Rating (GMCR). This rating states, “Since [last week], has there been any change in your mobility?” The response is made on a 15-point self-report Likert scale, from −7 to +7 based on recommendations for global measures of change. The subject's perceived reasons for change were asked via an open-ended question and also documented for context. Change in walking function was not limited to issues related to the prosthesis; illness, injury, or recovery of any kind were considered valid sources of mobility change if the subject thought they resulted in a small or substantial change in their walking. This approach was chosen because there was precedence in using GMCR for determining meaningful change in gait speed, Short Physical Performance Battery, and the 6-minute-walk test.

Each subject was tracked for 6 months (24 weeks). To discourage subject dropout, subjects were compensated $40 each month after retrieval of the StepWatch with the previous month's data and corresponding GMCR scores.

Statistical analysis

For each study subject, our data included up to 24 weekly measurements, of which 23 weeks could be used for computing change from the previous week. Most subjects provided data for fewer than 24 weeks. This study required at least 2 consecutive weeks of data to measure change, so 4 subjects who did not provide at least this much data were dropped from the analysis. Our approach to the missing weekly data was simply to analyze what was available. This was sufficient for our study objective, which was to measure the association between subjects’ recall of a change from the previous week with the change that actually occurred (community walking metrics such as Modus Index).

Our data set was a “clustered” data set, because there were multiple observations (2-23) for each study subject. Ordinary statistical methods, such as linear regression, assume that all observations in the data set are independent, which occurs when there is 1 observation (1 number for each of the variables) for each subject. To account for the lack of independence in a clustered data set, a mixed effects linear regression was required. This model computes the intracluster correlation coefficient (ICC), which measures the amount of lack of independence and then makes an appropriate correction based on the ICC so that the standard error, P values, and confidence intervals are correct. If the ICC turns out to be 0, the mixed effects model reduces to the ordinary linear regression model, so ordinary linear regression can be used.

For the descriptive analysis, this study computed the mean of the multiple observations for each subject, which reduces the data to a single value, and then analyzed these means as if they were a single observation per subject. This was necessary because with clustered data, a standard deviation is not a useful measure, because it is a composite of the variability from subject to subject as well as the variability of multiple observations within the same subject.

This study performed univariable regression models using 1 predictor variable at a time followed by a multivariable regression that included all of our predictor variables of interest. Next, the interaction terms were added between GMCR, our primary predictor, with each of the covariates in the model.

Results

A total of 100 subjects were recruited. Nine subjects dropped out (defined as being unresponsive to all attempts to gather weekly GMCR and never returned their StepWatch device). One subject was withdrawn by the investigators because of inability to give reliable answers to the GMCR related to cognitive deficits from dementia that became apparent during the study but had not been previously identified during screening. We required at least 2 consecutive weeks of data where the subject reported GMCR and provided StepWatch data, so that weekly change could be computed. Four subjects had partial data but did not meet this criterion and so were dropped from the data analysis. This left us with a sample size of n=86. Only 4 subjects of the n=86 provided the planned 24 weeks of data, of which a maximum of 23 weeks could be used to compute weekly change. The average number of weeks of data per subject was mean±SD of 14±6 weeks (table 2).

Table 2

Subject demographics (n=86)

Sex, n (%)*
Male	70 (92)
Female	6 (8)
Missing (n=10, (12%)
K-level, n (%)
1	9 (11)
2	18 (21)
3	36 (43)
4	21 (25)
Missing (n=2, 2%)
Amputation level, n (%)Unilateral amputation Transtibial Transfemoral SymeBilateral amputationTranstibial	49 (64)19 (25)1 (1)7 (9)
Missing (n=10, 12%)
Age, y
Mean±SD (min, max)	58±16 (21, 85)
Missing (n=0)
Number of weeks data were available for each study subject†
Mean±SD (min, max)	14±6 (2, 23)
Missing (n=0)
Number of weeks data were available for each study subject by K-levelMean±SD (min, max)
K-level: 1	13±6 (3, 22)
2	15±6 (3, 23)
3	15±6 (3, 23)
4	11±7 (2, 22)
Missing (n=2, (2%)

NOTES.

Percentages are percent of nonmissing data sample size; missing percentage is percent of study sample size (n=86).

This is the number of weeks for which both the GMCR and StepWatch data were recorded. GMCR could not be recorded the first week, because it represents change from previous week, so maximum is n=23, which 1 week less than the study's 24 weeks of follow-up.

Descriptive data on the most pertinent metrics are available in table 2 for all subjects and divided by K-level. Daily steps, cadence, cadence variability, and Modus Index are given for each K-level in table 3, as well as the standard deviation and the minimum and maximum of the observed values. Even with substantial overlap between K-levels in all measures, K-level highly correlated with the StepWatch metrics (table 3). The highest correlation was between the Modus Index and K-levels (rs=0.78, P<.001).

Table 3

StepWatch metrics descriptive statistics and linear trend test across K-levels

	(Average Experience)Simple descriptive statistics after collapsing data for all weeks into a single mean for each subject (so 1 measurement per subject)(n=86)							(Weekly Experience)Keeping data for all weeks for each subject and fitting a mixed effects linear regression model(n=1177 weekly observations with 2-23, 13.7 on average, observations per study subject)
	n	Min	Max	Mean	SD	Linear Trend Test P Value	Spearman RhoP Value	Mean	SE	Linear Trend TestP Value	ICC
Modus Index
Total sample	86	18.4	93.4	57.9	19.7			57.9	2.1		0.88
K-level 1	9	18.4	44.6	27.1	8.4	<.001	rho=0.77	27.1	3.8	<.001	0.70
K-level 2	18	26.7	64.0	41.9	12.9		P<.001	42.0	2.7
K-level 3	36	34.2	78.6	65.5	11.2			65.5	1.9
K-level 4	21	35.2	93.4	73.9	13.0			73.9	2.5
Daily steps
Total sample	86	16	6860	2019	1582			2018	169		0.84
K-level 1	9	16	1282	401	391	<.001	rho=0.62	402	411	<.001	0.76
K-level 2	18	82	3013	889	788		P<.001	894	290
K-level 3	36	335	5403	2429	1234			2433	205
K-level 4	21	141	6860	3132	1784			3129	270
Cadence
Total sample	86	1.6	21.7	9.0	4.0			9.0	0.4		0.85
K-level 1	9	1.6	6.0	4.3	1.5	<.001	rho=0.67	4.2	1.0	<.001	0.74
K-level 2	18	1.7	11.0	6.1	2.8		P<.001	6.2	0.7
K-level 3	36	4.1	17.3	10.1	2.9			10.1	0.5
K-level 4	21	2.4	21.7	12.0	3.8			12.1	0.6
Cadence variability
Total sample	86	1.2	20.2	8.1	3.3			8.1	0.4		0.81
K-level 1	9	1.2	7.2	4.3	1.7	<.001	rho=0.61	4.3	0.8	<.001	0.71
K-level 2	18	1.3	9.8	6.0	2.3		P<.001	6.0	0.6
K-level 3	36	2.8	13.3	9.0	2.2			9.0	0.4
K-level 4	21	2.5	20.2	10.5	3.5			10.5	0.6

NOTE. If we use the original StepWatch metric instead of weekly change, the K-level is highly correlated with the metrics.

There were very weak associations between change in GMCR and change in any of the walking metrics: GMCR vs Modus Index (r=0.15, P<.001), GMCR vs daily steps (r=0.08, P=.009), GMCR vs cadence (r=0.14, P<.001), and GMCR vs cadence variability (r=0.16, P<.001). The histogram shown in figure 2 shows the Pearson correlation coefficients correlating GMCR change with Modus Index change. Per this figure, it is evident that there are as many low correlations (−0.30

Fig 2

Histogram showing Pearson correlation coefficients correlating GMCR with Modus Index change.

The regression models determined that, of the demographic data, only amputation level had a small but significant influence on ability to predict Modus Index weekly change from GMCR change. The subjects with unilateral transfemoral amputation were slightly better (R2=0.10, P<.001) at predicting their mobility change than those with bilateral transtibial amputation (R2=0.05, P<.008), but neither group did it well based on the low R2 statistic.

The data from a few sample subjects are presented to reflect the variability among subjects. Subject 33 (figure 3) reported a drop in GMCR in week 4 due to a poor-fitting prosthetic socket leading to skin breakdown. This was reflected in a drop in all metrics for that week. On week 7, the subject reported an increased GMCR, stating that the skin breakdown had healed after socket modifications, which was reflected in an increase in all metrics for that week. On week 20, when Subject 33 reported having surgery, there was a drop in GMCR and all metrics. In contrast, on week 9, Subject 33 reported having a flare of back pain, and the data showed a drop in GMCR but the objective walking metrics were relatively stable.

Fig 3

Ambulation metrics versus change in GMCR for Subject 33.

In Subject 80 (figure 4), we see an opposite trend in the GMCR vs the metrics during weeks 1-7. This meant that in some weeks with lower GMCRs, the metrics showed higher levels of mobility. Then, in week 9, the subject was hospitalized for a toe amputation, and there was a corresponding drop in the objective metrics of walking. The amount of variation in GMCR also ranged in different patients. Subject 06 (figure 4) showed a wide variation in GMCR during weeks 1-7 with relatively stable walking metrics. In contrast, Subject 82 (figure 5) reported only a small decrease in function compared with the week prior (GMCR of −1) due to a “poor-fitting socket,” but his metrics revealed a substantial decrease in steps: an average of only 1 step per day indicating almost no walking with his prosthesis.

Fig 4

Ambulation metrics versus change in GMCR for Subjects 6, 80.

Fig 5

Ambulation metrics versus change in GMCR for Subjects 82, 98.

Even in the same subject, the correlation between GMCR and objective metrics was often poor. As an example, Subject 98 (figure 5) reported a GMCR of −5 on week 6 due to problems with prosthesis stability and again on week 11 due to an issue with prosthetic componentry. In week 6, we see a corresponding drop in metrics but no change in week 11, despite the same GMCR score.

Discussion

The purpose of this study was to evaluate the normal variation in community walking patterns in a cohort of lower limb prosthesis users. We hoped to identify the minimum clinically meaningful change in walking metrics; however, there were only very weak associations between change in GMCR and change in walking metrics in this study. The study does provide normative data for a relatively large study group of lower limb prosthesis users.

The lack of a strong association between the subjects’ subjective reports of change in walking ability (GMCR) and objective changes in walking metrics is consistent with previous studies in this and other populations,4, 5, 6,8, 9, 10 showing that patients are overall poor at self-reporting activity levels. Other populations with similar findings are as follows: multiple sclerosis, colon cancer, low back pain, and post joint arthroplasty. Our study confirms that prosthesis users are similarly inaccurate at self-reporting their activity levels and change in mobility from week to week. Clinicians must be aware of this when interviewing patients and recognize that approximately 50% of their patients may be inaccurate at reporting their activity level.

Recall may be challenging for patients for a variety of reasons. In this study, subjects were asked to compare mobility in 1-week blocks. Many of our subjects had difficulty recalling the prior week. They may have simply gotten confused on which week the change in mobility occurred, as is possible in Subject 80. Subject 80 reported the toe injury in week 8, but week 7 had even lower walking activity, indicating that the problem may have started in week 7. Alternatively, subjects may have had 1 sentinel event (either positive or negative) that week that colored their impression of the entire week. It is also possible that a subject could have had a week of increased activity, which may have caused them to feel fatigued or have residual limb pain, and that may have translated to reporting reduced GMCR when actual walking improved.

This study also provided community walking metrics for individuals at K1-4 levels. Though it is the opinion of the authors that K-level must remain a clinical decision based on the clinician's assessment of the patient, objective data can provide valuable information. Objective community walking metrics can also be useful when determining what is normal for an individual patient; a substantial change in walking metrics may trigger a clinician to investigate the cause. In this study, the subjects did not have access to their own data, but if patients were able to receive real-time feedback about walking data, it may help with self-report and add context to a perceived change in function to aid their recall.

As previously demonstrated in other studies on objective measures of walking function in amputees,,43, 44, 45 there is overlap in metrics between the K-levels. However, there is a strong correlation between higher function as measured by these metrics as the K-level increases. It is important to recognize that in this study, K-level was used from the medical record and, by definition, the K-level is based on the patients’ “ability or potential” to reach certain functional benchmarks. Because recruitment was for subjects at least 6 months post-amputation surgery, they may not have reached their potential yet, and some individuals may never reach that theoretical potential functional level if their rehabilitation is not optimized. This would have worsened the correlations between the objective metrics of walking and the K-level categories.

Although objective walking metrics do not provide a criterion standard K-level, and no metric is able to do so,, they can provide insight. For example, if a clinician is struggling to provide K-level to a third-party payor, these metrics may be helpful in supporting the decision. A previous publication provided guidance when using objective walking metrics to support K-level decisions.

In patients who accurately provide GMCR scores that correspond to changes in walking metrics, these objective data may be especially useful. In Subject 33, poor socket fit led to a decrease in all metrics, which then recovered after the socket was modified. This subject also saw a drop in metrics after a surgery. In clinical practice, having a baseline for a patient's normal walking patterns and expected variation may help a patient or the rehabilitation team set realistic and appropriate goals for improving function after injuries, illness, or surgeries. A drop in walking metric, in combination with other information, may also help to justify to third-party payors the prescription of a new prosthetic socket or other prosthetic changes.

Study limitations

Though this study did have a relatively large sample size compared to similar studies in this population, an even larger sample size may have allowed identification of significant associations in GMCR and walking metrics. The predominance of male subjects and subjects with higher K-levels may limit generalizability. Because recall of a prior week's function appeared difficult for some subjects, it may have been helpful to have subjects keep a daily log of their function rather than reporting weekly. Though the GMCR is a validated measure of subjective change in mobility, it requires comparison with a prior period of time, which may have been difficult for some subjects. Utilizing a measure of perceived walking function—a simple Likert score or a validated questionnaire—rather than perceived change may have been easier for some subjects to understand and might have given different results. However, to our knowledge, the GMCR is one of few measures that is validated to identify small changes in perceived mobility across small intervals of time.

Conclusions

When a clinician has prior data indicating the walking patterns of an individual patient, new changes in subjective and objective data may guide clinical decision making. In this study, some subjects reported wide variation in GMCR despite stable walking metrics, whereas others reported small decreases in function on the GMCR with substantial changes in walking metrics. Though clinicians cannot guess ahead of time which category an individual patient may fall into, a history of objective data may help reassure “overreporters” or identify “underreporters” who require intervention. Determining the normal variation in walking metrics for an individual patient may allow protective measures to be taken, by either the patient or the clinical team.

Future studies may build on this study by utilizing real-time data capture and analysis to determine whether that better correlates with subjective reports. As StepWatch and other validated devices develop the ability to immediately sync with a mobile apps and clouds for patient and clinician viewing in near real time, future research directions include investigating the effect of that data on patient behavior and clinical decisions by the rehabilitation team.

Suppliers

StepWatch Activity Monitor, Modus Health.