Smartwatches Can Detect Walker and Cane Use in Older Adults.

BACKGROUND AND OBJECTIVES: Clinicians commonly prescribe assistive devices such as walkers or canes to reduce older adults' fall risk. However, older adults may not consistently use their assistive device, and measuring adherence can be challenging due to self-report bias or cognitive deficits. Because walking patterns can change while using an assistive device, we hypothesized that smartphones and smartwatches, combined with machine-learning algorithms, could detect whether an older adult was walking with an assistive device. RESEARCH DESIGN AND METHODS: Older adults at an Adult Day Center (n = 14) wore an Android smartphone and Actigraph smartwatch while completing the six-minute walk, 10-meter walk, and Timed Up and Go tests with and without their assistive device on five separate days. We used accelerometer data from the devices to build machine-learning algorithms to detect whether the participant was walking with or without their assistive device. We tested our algorithms using cross-validation.
RESULTS: Smartwatch classifiers could accurately detect assistive device use, but smartphone classifiers performed poorly. Customized smartwatch classifiers, which were created specifically for one participant, had greater than 95% classification accuracy for all participants. Noncustomized smartwatch classifiers (ie, an "off-the-shelf" system) had greater than 90% accuracy for 10 of the 14 participants. A noncustomized system performed better for walker users than cane users. DISCUSSION AND IMPLICATIONS: Our approach can leverage data from existing commercial devices to provide a deeper understanding of walker or cane use. This work can inform scalable public health monitoring tools to quantify assistive device adherence and enable proactive fall interventions.

Many older adults are prescribed walker and canes to reduce their risk of falling; however, these devices are often used inconsistently or not at all. Smartwatches and smartphones, combined with machine-learning algorithms, can remotely monitor walker or cane use and trigger reminders to use these assistive devices.

More than one in three older adults falls each year, resulting in an estimated 29 million falls, 7 million injuries requiring medical attention, and $35 billion in direct medical costs.[1-3] Over the past several decades, multiple research initiatives and public health efforts have identified many factors that contribute to a high risk of falling, including a history of previous falls, cognitive impairments, gait problems, weakness, polypharmacy, and assistive device use.[4-7] Although our understanding of risk factors for falls has improved, the incidence of falls in older adults remains high.

An estimated one in four older adults over the age of 65 uses an assistive device, and the number of users has increased by nearly 50% in the past 15 years.[8] Walkers and canes are frequently prescribed to older adults to reduce fall risk, but are often used inconsistently or not at all.[9-11] Assistive devices may be used inconsistently or not at all due to factors such as accessibility, social stigma, or memory impairments.[9-11] In some studies, device nonuse has been associated with an increased risk of falling. One study found that 75% of falls at home occurred when an older adult was reportedly not using their assistive device.[12] Another study found that, of hospital patients who reported using an assistive device at home and fell in the hospital, 94% were not walking with a device at the time of their fall. Conversely, other studies have found that walkers and canes are associated with an increased risk of falls,[4,13] because they may interfere with a person’s balance control.[14] However, correlations like this may also relate to the fact that older adults with better mobility tend to not use assistive devices. While the intention of using a walker or cane may be to reduce fall risk, the evidence regarding their effectiveness in fall prevention remains mixed. Therefore, understanding assistive device adherence in the context of falls is critical to create better fall assessments and interventions.

A significant barrier to understanding fall mechanisms is the lack of objective, real-life falls data.[15-17] Clinicians and researchers frequently rely on self-report to understand a patient’s fall history and whether the patient was using their walker or cane. However, patients are prone to poor recall, especially when asked to recall events over longer periods of time.[18] Providing clinicians, researchers, and caregivers with clinically useful, objective measurements of assistive device adherence can lead to a greater understanding of factors that contribute to overall mobility, device adherence, and falls. Moreover, such information can assist clinicians in developing treatment plans that target assistive device use in an effort to decrease fall risk. Availability of such information may enable interventions to improve adherence and, hopefully, reduce fall risk.

Advances in machine learning have enabled activity recognition systems to monitor older adults’ mobility.[19] Activity recognition systems use information, oftentimes from movement sensors, to learn differences between activities (eg, walking, sitting, climbing stairs). Such systems perform best when the information they receive contains patterns that vary significantly among the activities. Therefore, sensor placement is critical for movement-related applications.[20] For example, sensors worn at the hip are often best for detecting walking-related activities, whereas sensors worn at the wrist are often best for detecting eating, grooming, and other activities involving the upper extremities.[20-22] With an appropriate design, activity recognition systems can make robust predictions about an older adult’s activity.

Wearables have become popular to monitor health because they have built-in sensors and processors, offering a platform readily available to clinicians, researchers, and the general public.[23,24] Usage of smart devices has almost doubled in the past 5 years: more than 75% of Americans own a smartphone,[25] and 15% of Americans own a smartwatch or fitness band.[26] As the population ages, the number of older adults using smart technology continues to rise.[27] The widespread use of smart technology has spawned numerous systems to detect falls to initiate faster emergency responses.[17,28,29] While rapidly responding to a fall improves outcomes,[30,31] it does not prevent the initial injury. In addition to reacting to a fall, smart technology may help prevent falls through proactive interventions that target modifiable risk factors.

In this study, we tested whether a smartphone or smartwatch, combined with machine-learning algorithms, could detect whether an older adult walked with or without their assistive device. Wearables, enabled by machine learning, may pick up on the changes in movement patterns to detect walker or cane use.[32,33] We hypothesized that smartwatches could detect walker and cane use more accurately than smartphones due to the change in arm swing when using an assistive device. We also hypothesized that a custom system (built for one individual) would perform better than a generic, “off-the-shelf” system. This study is innovative because it extends traditional activity recognition methods to detect the use of a walker or cane.

Research Design and Methods

Participants

We recruited 20 older adults from an adult day center in Evanston, IL. Participants were included if they were aged 60 or older, used a walker or cane, and could walk 10 m without their walker or cane with no physical assistance. Participants were excluded if there was an increase in pain when walking without their assistive device. We included individuals with cognitive impairments. To assess balance and cognition, participants completed the Berg Balance Scale, and we collected Mini-Mental State Exam scores from the adult day center. This study was approved by the Northwestern Institutional Review Board (STU00203678) and all participants provided written, informed consent with a third party witness.

Protocol

To train machine-learning algorithms to detect assistive device use, we collected sensor data under two conditions: (a) participants walking with an assistive device and (b) participants walking without an assistive device. Participants wore a smartphone and smartwatch while completing the six-minute walk, 10-meter walk, and Timed Up and Go tests with and without their assistive device on five separate days (Figure 1). A computer program randomized the order of the tests for each session to mitigate any bias in the order of data collection. A physical therapist recorded the participant's score and if the participant used their assistive device. Participants were allowed to take rest breaks at any time, and for as long as needed. The physical therapist monitored participants’ vital signs throughout the session and provided standby assistance to participants while walking to prevent falls.

Figure 1.

Experimental overview. (A) Older adults wore a smartphone and smartwatch to record accelerometer data. (B) Participants performed the following three physical therapy outcome measures on multiple days with and without their assistive device: Timed Up and Go (TUG), 10-meter walk test (10MWT), and six-minute walk test (6MWT). (C) We extracted features (f1, f2,…,fn) from accelerometer data and trained machine-learning classifiers to detect whether the participant was using their walker or cane. (D) We used cross-validation to evaluate how well our classifiers can predict assistive device use.

Sensor Measurements

Participants wore an Android smartphone (Nexus 5, Android version 6.0.1) with a custom app to record tri-axial accelerometer and gyroscope data at approximately 50 Hz. The location of the phone was unrestricted; at the start of each session, participants could place the phone in either pocket, in any orientation. In almost all cases, participants placed the phone in the same pocket every session. If a participant did not have pockets, they were provided a belt clip to wear the phone on their waist.

Participants also wore a wristwatch with a tri-axial accelerometer (Actigraph wGT3X-BT; Actigraph LLC, Pensacola, FL), sampled at 50 Hz. For participants who used a cane, the watch was worn on the same side as the cane. For participants who used a walker, they could place the watch on either wrist. Sensor data from the phone and watch were collected throughout the entire session, including rest breaks.

Data Preprocessing

Data processing and classification was performed offline using common procedures and parameters.[34-36] After data collection, we annotated the sensor signals using the recorded timestamps for the six-minute walk, 10 meter walk, and Timed Up and Go tests. We linearly interpolated the sensor signals to 30 Hz, separated the signals into nonoverlapping 3-second clips, removed clips with multiple labels, and calculated features for each clip (Table 1).

Table 1.

Features for Classifier

Description	Number
Minimum	4
Maximum	4
Mean	4
Standard deviation	4
Skew	4
Kurtosis	4
Interquartile range	4
Total per sensor	28

Notes: Each feature calculated for x, y, z and resultant signal per sensor. There were 56 features for the smartphone classifiers (accelerometer and gyroscope), and 28 features for the smartwatch classifiers (accelerometer).

Classification Algorithms

We tested a number of common machine-learning classifiers including random forests (RF), support vector machines (SVM), naïve Bayes (NB), logistic regression (LR), and linear discriminant analysis (LDA). For each classifier, we chose hyper-parameters through grid searches with across-user cross-validation on a different activity tracking data set.[37] Features were scaled to have mean 0 and unit variance (RF, NB, LR, LDA), or were scaled linearly to the range 0–1 (SVM) on the training set. Classifiers were trained and tested in Python (version 3.5.2) with scikit-learn (version 0.19.2).

Evaluation of Classification Algorithms

We performed four types of cross-validation to evaluate our classifiers (Supplementary Figure 1). Cross-validation is a process in which a data set is split into a training set and test set, the classifier is created using the training set, and evaluated on the unseen test data. Different types of cross-validation provide an indication of how well a classifier will perform in different use-case scenarios. Data from the six-minute walk test, 10-meter walk test, and Timed Up and Go were combined and used for all types of cross-validation.

User-specific, within-day cross-validation

We trained classifiers using data from all 5 days for one participant and tested the classifiers on all 5 days from that same participant. We used fivefold cross-validation, where the data set is divided into five equal sized pieces; the classifier is trained using four pieces, tested on the fifth piece, and the results averaged over the five possible divisions. If this type of cross-validation is accurate, it demonstrates that the classifier can detect assistive device use for the same participant on the same day. This resembles the unrealistic scenario where the user calibrates the system every single day.

User-specific, across-day cross-validation (80/20 train/test split)

We trained classifiers using data from 4 days for one participant and tested the classifiers on the remaining day for that participant. If this type of cross-validation is accurate, it demonstrates that the classifier can detect assistive device use for the same participant on different days. This resembles the scenario where a patient walks in a supervised setting (eg, a clinic) to calibrate the system before use.

User-specific, across-day cross-validation (20/80 train/test split)

We trained classifiers using data from 1 day for one participant, which is less data than in the previous case, and tested the resulting classifiers on the remaining 4 days for that participant. This test is important as it is expensive to obtain training data. If this type of cross-validation is accurate, it demonstrates that small amounts of user-specific training data are sufficient. This resembles the scenario where a patient comes into the clinic or research lab for 1 day for training, and no further training data are obtained.

User-generic, across-user cross-validation

We trained classifiers using data from all but one participant, and then tested the classifiers on that remaining participant. For example, to test a user-generic classifier for Participant 1, we built a classifier using data from Participants 2–14. Then, to test a user-generic classifier for Participant 2, we built a classifier using data from Participants 1 and 3–14. We repeated this process, using a new participant in the test data until we had evaluated a user-generic classifier on every participant. If this type of cross-validation is accurate, it demonstrates that the classifier can detect assistive device use for new individuals. This mimics the desirable use-case of a person buying the system “off-the-shelf,” with no additional calibration or training required.

Extending the Classifiers to Include Nonwalking Activities

People are not always walking, and a practical activity recognition algorithm must be able to detect assistive device use in the presence of nonwalking activities. Therefore, we extended our classifiers to include sitting, standing, sit-to-stand, and stand-to-sit activities, merged into a single class named “Not Walking.” Data for the “Not Walking” class was taken and labeled from participant rest breaks in between walking trials. We then reevaluated our classifiers using the same four types of cross-validation described above.

Combining Smartphone and Smartwatch Sensors

Classifiers often perform better when using data from multiple sensors.[21] We combined our data from the smartphone and smartwatch and repeated our previous analyses to test whether using both sensors improved classifier performance.

Statistical Analyses

Determining the amount of training data required for machine learning is notoriously difficult because classification performance depends on how separable the classes are for a specific application. We collected enough data to have at least hundreds, and more often thousands of samples to train our classifiers, an amount that is typical for activity recognition studies.[17,22,38] We compared the classification accuracy of the smartphone to the smartwatch for all types of cross-validation. We report median accuracies and used Wilcoxon signed-ranked tests for all comparisons (α = 0.05) because the data were not normally distributed (Shapiro–Wilk test, p < .05). However, in all cases we have training data across five time-points available, making the machine-learning analysis of this problem meaningful.

Results

Fourteen older adults met the inclusion criteria and completed the study (Table 2). Eight participants were walker users, and six participants were cane users. Participants wore a smartphone and smartwatch while they completed the six-minute walk, 10-meter walk, and Timed Up and Go tests with and without their assistive device. We used accelerometer data from a smartphone and a smartwatch (Figure 2) to train machine-learning classifiers to predict whether a participant was walking with or without their assistive device. We evaluated the performance of our machine-learning classifiers using four types of cross-validation mimicking different use cases. We present our results from the Random Forest classifiers because they were the most accurate. We found the same trends in performance for all state-of-the-art classifiers (not shown).

Table 2.

Participant Characteristics

No.	Sex	Age	BBS	MMSE	Assistive Device	Phone Location	Watch Location	Walking (Device), min	Walking (No Device), min	Not Walking, min
1	M	65	31	30	LBQC	R pocket	R wrist	52.4	27.2	160.3
2	M	73	47	30	LBQC	L pocket	R wrist	27.8	33.7	52.0
5	M	84	37	14	RW	R pocket	R wrist	37.3	39.1	138.2
6	F	62	48	30	SPC	L pocket	L wrist	36.9	36.2	72.2
7	M	78	37	24	SPC	L pocket	R wrist	34.3	33.4	91.3
8	F	83	42	18	RW	L belt	R wrist	40.9	34.8	101.5
9	M	86	32	12	RW	R pocket	R wrist	40.4	31.7	85.2
11	F	86	46	23	Rollator	L belt	L wrist	38.9	39.0	62.1
13	M	88	38	26	RW	L pocket	R wrist	39.7	37.1	111.1
14	F	83	36	24	Rollator	R pocket	L wrist	34.0	21.3	136.7
16	F	95	40	21	RW	R pocket	L wrist	36.5	23.4	97.2
18	M	86	54	27	SPC	L pocket	R wrist	37.4	36.3	84.7
19	M	79	42	10	RW	R pocket	L wrist	42.9	39.7	64.0
20	M	93	48	22	SPC	R pocket	R wrist	36.6	37.3	49.2
	9 M/5F	81.5 (9.2)	41.1 (6.4)	22.5 (6.6)	8 walkers 6 canes	7R/7L	9R/5L	38.2 (5.3)	33.6 (5.6)	93.2 (32.6)

Note: BBS = Berg Balance Scale; MMSE = Mini-Mental State Exam; R = right; L = left; LBQC = Large Base Quad Cane; SPC = Single Point Cane; RW = Rolling Walker.

Figure 2.

Examples of accelerometer data. (A) Smartphone and smartwatch accelerometer data when a participant (#7) walked with their cane, and without their cane. (B) Smartphone and smartwatch accelerometer data when a participant (#19) walked with their rolling walker, and without their rolling walker.

Smartwatch classifiers were superior to smartphone classifiers for all four types of cross-validation (p < .01 for all comparisons), supporting our first hypothesis.

The purpose of this type of cross-validation was to determine whether a classifier can detect assistive device use for the same participant on the same day. The smartphone system had a median classification accuracy of 92.9% (Figure 3A). The smartwatch system had a median classification accuracy of 99.7% accuracy (Figure 3B). As expected, both the smartphone classifiers and smartwatch classifiers had high accuracy because this type of cross-validation likely overestimates classifier accuracy for a real-world scenario.[39] While the classifiers are accurate, obtaining training data on the same day for every participant is unrealistic in real-world settings.

Figure 3.

(A) Classification accuracy when using smartphone sensors (accelerometer and gyroscope) to detect whether and older is walking with or without their assistive device. (B) Classification accuracy when using a smartwatch sensor (accelerometer) to detect whether an older adult is walking with or without their assistive device. The length of each bar represents the average accuracy across folds; error bars represent the standard deviation across folds.

The purpose of this type of cross-validation was to determine whether a classifier can detect assistive device use for the same participant on different days. The smartphone system had median classification accuracy of 64.2% (Figure 3A). The smartwatch system had a median classification accuracy of 99.6% (Figure 3B). Smartwatch classifiers maintained high classification accuracy across days, whereas smartphone classifiers failed.

The purpose of this type of cross-validation was to determine whether user-specific classifiers can make accurate predictions with data from a single session of data collection. The smartphone system had median classification accuracy of 57.6% (Figure 3A). The smartwatch system had a median classification accuracy of 99.2% (Figure 3B). Even though the amount of training data were limited to 1 day of data collection (less than 15 minutes spent walking), the smartwatch system continued to accurately detect assistive device use across days.

The purpose of this type of cross-validation was to determine whether a classifier can detect assistive device use for new users: participants for which the classifier had no prior information. We trained classifiers on data from all 5 days for all but one participant, and then tested the classifier on data from all 5 days for that remaining participant. The smartphone system had median classification accuracy of 54.4% (Figure 3A). The smartwatch system had a median classification accuracy of 98.2%. A closer examination of individual participant results revealed that smartwatches could detect assistive device use on some, but not all new users (Figure 3B). The smartwatch system had greater than 95% accuracy for 10/14 participants, but the smartphone system continued to fail (close to chance predictions) for all participants.

The four participants with the lowest smartwatch accuracies (<95%) were all cane users. We re-ran our across-user cross-validation for the smartwatch to determine whether device-specific (ie, walker specific or cane specific) classifiers improved performance. Classification accuracy improved slightly, however, the same four participants remained below 95% classification accuracy. In both our initial and post hoc analysis, new user classification accuracy was higher for participants who used a walker as opposed to a cane (Mann–Whitney U test, p < .05). Cane use was relatively hard to detect across participants.

We extended our classifiers to include nonwalking activities (ie, sitting, standing, stand to/from sit) to better evaluate their performance for real-world scenarios. Because the classifiers now include three classes with class imbalances, we present our results in the form of confusion matrices (Figure 4). Again, smartwatch classifiers outperformed smartphone classifiers for all four types of cross-validation (p < .01 for all comparisons). However, the smartwatch system had a small, but statistically significant decrease in classification accuracy when we included “Not Walking” activities for all user-specific classifiers (p < .05). There was no statistically significant difference for user-generic classifiers (p = .27). User-specific classifiers outperformed user-generic classifiers (p < .05 for all comparisons), supporting our second hypothesis. For user-generic smartwatch classifiers, the most common classification mistake was predicting that the participant was not walking, when they were actually walking with their device; however, these mistakes were participant dependent.

Figure 4.

Confusion matrices when classifiers were extended to include nonwalking activities. Higher percentages along the diagonal indicate better classifier performance. Higher percentages off the diagonal show where classifiers made incorrect predictions, or became confused.

When including nonwalking activities, combined smartphone and smartwatch classifiers were superior to smartwatch classifiers for all types of cross-validation (p < .01, Supplementary Figure 2) except user-specific, across-day cross-validation (20/80 train/test split).

Discussion and Implications

The purpose of this study was to determine whether smartphones and smartwatches, combined with machine-learning algorithms, could detect whether an older adult was walking with or without their walker or cane. Smartwatch classifiers could accurately detect assistive device use, but smartphone classifiers had close to chance accuracies. We found that if a user-generic smartwatch system did not perform well for a new user, an accurate user-specific system could be created with a single session of walking.

There is a growing body of evidence demonstrating that wearables such as smartphones and smartwatches can detect activities of daily living and falls.[17,24,28,40-43] We found that smartwatches provided considerably higher quality data for detecting walker and cane use compared with smartphones. Previous studies have demonstrated the importance of sensor placement for accurate classification.[37,44] Our results suggest that the change in arm swing (smartwatch) when using a walker or cane is greater than the change in hip or thigh motion (smartphone). User-specific classifiers failed when smartphone position was controlled via a belt clip for two participants (no pockets available), suggesting that restricting the smartphone position would not have substantially improved walker and cane detection. Although the smartphone was worse at detecting walking with versus without a device, it was better at detecting walking versus not walking for user-generic classifiers (Figure 4D). To detect walker and cane use, we recommend having at least one sensor located on the wrist, and, for user-generic classifiers, adding a second sensor located at the hip to provide the most accurate predictions (Supplementary Figure 2).

Existing activity recognition and fall detection systems can be augmented to detect walker or cane use in older adults. Because our approach uses common sensors, data processing techniques, and machine-learning classifiers, existing systems can extend their classifiers to predict whether an older adult is using their walker or cane. We found that a user-generic (“off-the-shelf”) smartwatch system could accurately detect device nonuse for most participants, especially those who used walkers. The system’s most common mistake was classifying “walking with a device” as “not walking” (Figure 4D). If walkers and canes reduce fall risk, we believe the most important class to detect is “walking without a device” to indicate that the user is at a higher risk of falling. A system with additional capabilities can remind the user to use their assistive device, or send a notification to a caregiver or clinician. A researcher or clinician can also use our approach to measure how often a person is “walking with a device” to assess assistive device adherence. In this case, we recommend using both a smartphone and smartwatch for a user-generic classifier. In all cases where a user-generic system fails, we recommend collecting additional data from the intended user to create a user-specific system, which should improve classification (Figure 4C). Alternatively, a large dataset including more than 14 participants may allow a better user-generic classifier.

The ability to detect walker and cane use can create new opportunities for researchers, clinicians, and caregivers. Research to understand and improve device adherence can be strengthened by quantitative, objective information about device usage patterns. For example, the ability to measure device use can enable researchers to evaluate the effectiveness of interventions targeting device adherence. Clinicians can also use such information to get feedback on how often their patients are using a device and modify their plan of care as appropriate. This may include changes to device fit, further gait training, or a home safety evaluation. Finally, family and caregivers caring for someone with cognitive impairments can remotely monitor whether the person is walking with their device at home.

There were several limitations to our study. First, different environments can affect recognition accuracy[45] and we did not validate our system in an older adult’s home. Although our data are from a familiar environment (adult day center), older adults can have different home layouts and compensatory techniques that affect their walking pattern (eg, furniture walking). Second, none of the older adults in our study used a standard walker; we do not know how accurate our approach is for standard walkers. Third, we used an Actigraph for data collection and have not validated our approach on commercial smartwatches (eg, Apple Watch). Fourth, we evaluated our classifiers on a small sample of participants. Although our sample size is small, it is typical for an activity recognition study.[17] Furthermore, we obtained several hours of recordings over multiple days for each participant. Future research should focus on making prospective predictions with a larger sample of participants to better understand how user-generic classifiers will perform on the general public and individuals with different movement impairments. Collecting data on more users also promises to lead to better user-generic classifiers. Finally, we performed our analysis offline and did not create real-time predictions. Much of the appeal of our technique is that real-time interventions become possible. With the advances in wearables and computing systems, extending our approach to a real-time system is an achievable next step.

In conclusion, we found that a smartwatch system could detect if an older adult is using their walker or cane. Our approach offers a tool for caregivers, clinicians, and researchers to monitor assistive device use in older adults.

Funding

This work was supported by the National Institutes of Health [5T32EB009406, R01NS063399]; the Northwestern University Data Science Initiative; and a Promotion of Doctoral Studies II Scholarship [to S.A.] by the Foundation for Physical Therapy Research.

Conflict of Interest

None reported.

Click here for additional data file.

Click here for additional data file.