Toward community-based wheelchair evaluation with machine learning methods.

INTRODUCTION: Upper extremity pain among manual wheelchair users induces functional decline and reduces quality of life. Research has identified chronic overuse due to wheelchair propulsion as one of the factors associated with upper limb injuries. Lack of a feasible tool to track wheelchair propulsion in the community precludes testing validity of wheelchair propulsion performed in the laboratory. Recent studies have shown that wheelchair propulsion can be tracked through machine learning methods and wearable accelerometers. Better results were found in subject-specific machine learning method. To further develop this technique, we conducted a pilot study examining the feasibility of measuring wheelchair propulsion patterns.
METHODS: Two participants, an experienced manual wheelchair user and an able-bodied individual, wore two accelerometers on their arms. The manual wheelchair user performed wheelchair propulsion patterns on a wheelchair roller system and overground. The able-bodied participant performed common daily activities such as cooking, cleaning, and eating.
RESULTS: The support vector machine built from the wrist and arm acceleration of wheelchair propulsion pattern recorded on the wheelchair roller system predicted the wheelchair propulsion patterns performed overground with 99.7% accuracy. The support vector machine built from additional rotation data recorded overground predicted wheelchair propulsion patterns (F1 = 0.968).
CONCLUSIONS: These results further demonstrate the possibility of tracking wheelchair propulsion in the community.

Introduction

Evidence suggests that wheelchair usage may induce abnormalities in joints of the upper limb[1-4] and increase the prevalence of upper limb pain.[5,6] Researchers have hypothesized that the increased chance of upper limb pain is due, in part, to repetitive overuse during wheelchair propulsion (WP).[5,7-10] Numerous examples from the biomechanics literature show that WP is a high-force daily task with a repetitive nature.[11-15] Shoulder pain and other upper limb injuries induce functional decline,[16,17] impede mobility and independence,[18] and reduce quality of life.[16,19,20] To prevent upper limb injuries and pain, researchers have theorized that reducing the amount of repetition and increasing the force efficiency in WP may reduce upper limb pain and injuries. The common WP patterns can be classified into four categories: arc, single loop over (SL), semicircle, and double loop over (DL) propulsion.[21-23] Researchers have found that certain WP patterns, such as the semicircular and DL patterns, are biomechanically more efficient.[13,24,25] Based on the kinematic and kinetic evidence, clinical practice guidelines (CPG) have been issued to suggest efficient propulsion to prevent upper limb injuries.[15,26]

Ecological validity of previous WP evidence is being challenged. Laboratory-based research assumes that wheelchair performance will most often reflect performance in the real world and that chronic overuse of the WP patterns performed in the laboratory is associated with upper limb injuries. However, a 3-year longitudinal study (n = 192) found that daily wheelchair usage measured through an odometer and self-report does not predict the chance of upper limb pain.[27] On the other hand, this study was limited in that self-report is often retrospective and prone to reporting errors; it also utilized odometers, which only measure the distance traveled throughout the day but do not report the context of daily activities such as WP patterns and the number of upper limb movements made. Therefore, a reliable tracking system for daily WP and activities is needed.

Several existing solutions may solve this problem of ecological validity. One solution is to utilize dynamometer-enhanced wheels, such as the SmartWheel, to capture force input and the trajectory of propulsion.[28,29] The SmartWheel has been used in numerous WP studies to find the most efficient way to propel a wheelchair.[11,24,30] The advantage of using the SmartWheel is its ability to capture numerous kinetic measures including contact force, tangential force, and overall force. The disadvantage of using the SmartWheel is that users can only push on the handrim where force can be measured. Many manual wheelchair users (MWUs) do not always push on the handrim; therefore, asking users to change their behavior may temper the assumption that we are measuring how MWUs perform in their daily lives. Furthermore, the cost of a SmartWheel prevents researchers from studying MWUs outside of the laboratory.

Another solution is to utilize numerous ambient monitors such as accelerometers,[31] odometers,[32] pressure sensors,[33] and location sensors[34] (see review, Tsang et al.[35]). However, most of these methods do not provide information regarding the context of daily living such as the types of activities the participant engages in, the quality of his or her WP, or the number of times each activity is performed. Context of the activity is important; ignoring the context may result in over-estimating the amount of energy expended during propulsion and physical activity levels in a real-world scenario, in which activities are performed intermittently throughout the measurement.

Activity recognition using machine learning (ML) and wearable sensors has been established over the last 20 years.[36,37] Several papers have demonstrated the use of ML to recognize different WP patterns and other related activities.[38-41] However, some studies are limited to WP patterns,[38] some studies only show crude contexts of activity (e.g., lying on a bed or sitting),[39,41] and others show limited activity predictions (e.g., desk work, pushing or being pushed in a wheelchair).[40] Furthermore, generalized ML recognition often suffers from lower accuracy due to between-subject variability in movement patterns. On the other hand, although subject-specific (i.e., individualized) ML recognition provides higher accuracy in activity recognition, recording activities for each individual may be a burden.

In this pilot study, we aim to further demonstrate the possibility of standardizing a subject-specific, ML-based daily wheelchair usage monitoring system using wearable accelerometers and tracking simulated daily activities. The goal is to overcome the recording burden and utilize a subject-specific ML model to recognize propulsion patterns with high accuracy. We tested the feasibility in two ways: (1) by building ML algorithms within a WP training session with a stationary roller system to identify overground WP performance, and determining the minimum amount of data required to achieve high accuracy in propulsion pattern prediction (Study 1), and (2) by building ML algorithms to identify WP patterns in simulated scenarios with different daily activities involved, and determining the benefit of using inertial measurement units (IMUs) that include rotation data in addition to the acceleration data, as per Study 1 (Study 2).

Study 1 methods

Participants

One experienced adult MWU who has transverse myelitis participated in this study. The experienced MWU is a trainer of WP and is experienced with the CPG. Ethical approval was granted by the Institutional Review Board (#20170447) at Washington University in St. Louis School of Medicine. The participant gave written, informed consent.

Equipment

The WheelMill System

The WheelMill System (WMS) is a stationary roller system that can simulate different terrains such as uphill and cross-slope[42] (Figure 1). The WMS can also apply different resistances to simulate the propulsion experience of carpet or tile. The benefit of using the WMS is that the MWU can focus on practicing WP without environmental distractions.[43] In our first experiment, we tested the feasibility of tuning an ML algorithm using data collected during the WP training session on the WMS[43] and using this WMS-tuned algorithm to predict overground WP.

Figure 1.

The WheelMill System.

Accelerometers

Two GT3x (ActiGraph, Pensacola, FL) accelerometers were fitted to the dominant arm of the participant. The accelerometers collected three-axle acceleration data at a 30 Hz sampling rate.

Procedures

The participant wore two accelerometers, one around the humeral lateral epicondyle and one around the dorsal wrist of the dominant arm. The participant performed four different WP patterns—arc, SL, semicircle, and DL—on two different surfaces: (1) the WMS and (2) a smooth, flat overground surface. The WMS trials simulated propulsion on a flat, smooth surface. The overground trials were conducted in a 150 foot indoor garage with a concrete flooring surface. The participant was told to go back-and-forth in oval-shaped laps; each direction was 60 feet long, for a total of 120 feet per lap. During the “U-turn,” participants were instructed to turn with the propulsion pattern they currently perform. The participant was asked to continuously perform each propulsion condition (i.e., arc, SL, semicircle, DL) for 5 min, with 2 min breaks between each condition. The total data recording time was 20 min. During each trial, a research staff member observed and confirmed that all propulsion repetitions were performed using the requested pattern.

Data analysis

Preprocess

All data analyses were performed with R version 3.3.0.[44] The caret package was used extensively throughout data analysis.[45] Each type of propulsion data was first cut into various epochs ranging from 1 s to 4 s. Feature variables, which were used to feed into the ML algorithm, were created with each epoch as one data point. Feature variables were generated with each axle of data (i.e., x, y, and z) by the time-domain features.[36] These features were created according to French et al.[38]; however, no frequency-related features were included, as suggested by Holloway et al.[46] Collinearity was removed, if any. To ensure that there were no mistakes in participants' performance of desired propulsion patterns, researcher observation was used to mark any mistakes made by the participant during each propulsion trial. Any mistakes would be timestamped and removed from the feature space. The participant performed the desired propulsion pattern even when turning, and no observable error was found during the trials. Therefore, we proceeded to label data accordingly to tune the ML algorithm and to evaluate the algorithm.

Machine learning

To tune a supervised ML algorithm, it is ideal to collect separate training data and testing data. Training data are used for tuning the ML algorithm to associate movement patterns into specific activity bins. To independently evaluate the result of the tuning, a testing dataset, which ideally should be a separate recording, is used. The references are created from human observation as the truth and compared with the prediction from the ML algorithms. WP movement patterns recorded from the WMS trial were used as the training dataset to tune both the k-nearest neighbor (kNN) and the support vector machine (SVM) with linear kernel. The tuned ML algorithms were then used to test data recorded from the overground WP movement patterns. The prediction of the ML algorithm then was compared against human observation. The comparison method was multi-class, one against all. To ensure that the protocol provided the greatest accuracy with the least amount of training recording, the accuracy of the ML algorithm was compared with varying minutes of data needed and varying cutting windows (i.e., epochs). This ranged from 1 min of each propulsion condition to 5 min. The statistical measures of algorithm performance were calculated including specificity, precision, sensitivity, and F1 measures. The F1 measure was calculated with the following formula:[47,48]

Because the F1 score is the harmonic mean of precision and recall, it is a comparative score that has been used throughout ML literature.

Study 1 results

No statistical significance test was compared. When comparing the accuracy of the two different models, the highest accuracy for the kNN method was 94.9%, and the highest accuracy for the SVM method was 99.20% (Tables 1 and 2). SVM modeling also had better overall accuracy across different variations of epoch and amounts of data used, and higher accuracy when there was a shortage of training data. The optimal prediction accuracy for SVM is 5 min of data for each propulsion pattern with 4 s of epoch window, with 99.7% accuracy. However, because the goal was to find a balance between the training burden and optimal accuracy, we thought the second-best accuracy (99.2% accurate) with a 2 s epoch window and that only requires 3 min of data collection was our best option. The third-best option with 3 min data was with a 3 s epoch window, with 99.0% accuracy. Tables 3 and 4 compare the differences in specificity, precision, sensitivity, and F1 scores of the second- and third-best options. The conclusion is that both of the 3 min recordings were viable options with optimal accuracy.

Table 1.

Percentage accuracy of kNN algorithm with different amounts of data for each propulsion condition in different epoch windows.

kNN	1 min	2 min	3 min	4 min	5 min
1 s	0.751	0.832	0.843	0.850	0.859
2 s	0.867	0.897	0.914	0.938	0.944
3 s	0.894	0.937	0.944	0.942	0.949[a]
4 s	0.882	0.916	0.923	0.923	0.933

kNN: k-nearest neighbor.

Highest accuracy among varying time windows.

Table 2.

Percentage accuracy of linear SVM algorithm with different amounts of data for each propulsion condition in different epoch windows.

SVM	1 min	2 min	3 min	4 min	5 min
1 s	0.963	0.980	0.976	0.962	0.967
2 s	0.857	0.968	0.992	0.987	0.981
3 s	0.712	0.758	0.990	0.990	0.987
4 s	0.916	0.741	0.879	0.872	0.997[a]

SVM: support vector machine.

Highest accuracy among varying time windows.

Table 3.

The evaluation of SVM predictions with 3 min of data for each propulsion condition and 2 s epoch window.

2 s	Specificity	Precision	Sensitivity	F1
Arc	1.000	1.000	0.987	0.993
DL	0.993	0.980	1.000	0.990
SC	1.000	1.000	0.980	0.990
SL	0.995	0.987	1.000	0.993

DL: double loop over propulsion; SC: semicircle; SL: single loop over propulsion.

Table 4.

The evaluation of SVM predictions with 3 min of data for each propulsion condition and 3 s epoch window.

3 s	Specificity	Precision	Sensitivity	F1
Arc	1.000	1.000	1.000	1.000
DL	0.987	0.960	1.000	0.980
SC	1.000	1.000	0.960	0.980
SL	1.000	1.000	1.000	1.000

DL: double loop over propulsion; SC: semicircle; SL: single loop over propulsion.

Study 2 methods

Participant

One experienced adult MWU who has transverse myelitis participated in this study. The experienced MWU is a trainer of WP and is experienced with the CPG. One able-bodied individual who has no cognitive or motor deficits also participated. Ethical approval was granted by the Institutional Review Board (#20170447) at Washington University in St. Louis School of Medicine, and both participants gave written, informed consent.

Inertial measurement unit

Two IMUs, BPMpro (270 Vision Limited, Chilbolton, UK), were fitted onto the same arm position as the accelerometers in Study 1. However, unlike in Study 1, the IMUs included three-axle acceleration and three-axle rotation data. The IMUs recorded both acceleration and rotation inertia with a 100 Hz sampling rate.

Both participants wore two IMUs, one around the humeral lateral epicondyle and one around the wrist of the dominant arm. The MWU performed two different WP patterns—SL and DL—overground. The MWU also maneuvered through ramps to simulate pushing uphill, rolling downhill, and navigating a cross-slope. The able-bodied participant performed eight different daily activities in a simulation laboratory similar to a studio apartment, including a kitchen, a bedroom, and a bathroom. The eight common daily activities performed included: (1) grabbing and reaching to upper cabinet, (2) grabbing and putting items into lower cabinet, (3) cleaning vertical closet door with cloth, (4) cleaning table with cloth, (5) mixing nuts and ice cream, (6) eating ice cream, (7) folding clothes, and (8) stirring food in a pan. The MWU did not perform activities of daily living because we were trying to reduce the burden of the MWU in this pilot study. Both participants were asked to perform each activity for 3 min, with 2 min breaks between each activity condition. The recording time of 3 min was determined based on Study 1 results.

All data analyses were identical to that of Study 1, except (1) rotation data were used to compare the benefit of utilizing rotation information in addition to the acceleration-only (as per Study 1), and (2) due to the limited amount of data recorded, a 10-fold stratified cross-validation technique was used. Training and testing datasets were divided 10-fold. Each time, 1/10 of the data points for each activity were used as testing data, and 9/10 of the data points for each activity were used to tune the ML algorithm; this was repeated 10 times with different data being used as the training set. The statistical measures of algorithm performance were calculated including specificity, precision, sensitivity, and F1 measures.

Study 2 results

No statistical significance was compared due to limited sample size. Overall, rotation information helped increase the accuracy of several activities (Table 5, Figure 3). There was no obvious difference between 2 s and 3 s epoch windows. The SVM algorithm was able to dissociate maneuvers and daily activities with fairly high accuracy. A 2 s epoch showed slightly higher accuracy than the 3 s epoch for detecting wheelchair maneuvers. The 3 s epoch showed slightly higher accuracy than the 2 s epoch for detecting the eight different daily activities. The details of the performance for each activity can be found in the normalized confusion matrices (Figures 2 and 3). Because maneuvering cross-slope and uphill both utilize an arc propulsion pattern, cross slope and uphill wheelchair maneuvers were often confused by the model. The folding clothes activity received lower accuracy when rotation information was not included in the model (Figure 4). The mixing food activity received relatively lower accuracy compared to other activities. Detailed statistical evaluation of the IMU-based model can be found in Tables 6 and 7.

Table 5.

Average F1 measures of the activity.

F1 measure	2 s IMU	3 s IMU	2 s Acc.	3 s Acc.
Average maneuver	0.968	0.965	0.936	0.944
Average daily activity	0.979	0.989	0.927	0.940
Average all activity	0.975	0.980	0.930	0.941

This table compares the epoch windows of 2 s and 3 s with or without rotation data.

Acc.: accelerometer; IMU: inertial measurement unit, records acceleration and rotation information.

Figure 3.

Two-second epoch window normalized confusion matrix for (a) accelerometer and (b) IMU.

Figure 2.

Three-second epoch window normalized confusion matrix for (a) accelerometer and (b) IMU.

Figure 4.

F1 score comparison between different sensors and different epoch windows.

Table 6.

Evaluation of the algorithm using both acceleration and rotation data with 2 s epoch.

2 s IMU	Specificity	Precision	Sensitivity	F1
Reaching up	0.999	0.992	0.992	0.992
Reaching low	1.000	1.000	1.000	1.000
Closet wiping with cloth	0.999	0.992	0.992	0.992
Table wiping with cloth	0.999	0.989	0.989	0.989
Mixing food with spoon	0.995	0.891	0.980	0.933
Eating food	1. 000	1.000	0.996	0.998
Folding clothes	0.998	0.956	0.977	0.966
Stirring food	0.999	0.990	0.942	0.966
Cross-slope	0.999	0.980	0.943	0.962
Pushing uphill	0.999	0.977	0.956	0.966
Rolling downhill	0.999	0.960	1.000	0.980
Single loop	0.996	0.957	0.965	0.961
Double loop	0.995	0.968	0.973	0.970

Table 7.

Evaluation of the algorithm using both acceleration and rotation data with 3 s epoch.

3 s IMU	Specificity	Precision	Sensitivity	F1
Reaching up	1.000	1.000	1.000	1.000
Reaching low	1.000	1.000	1.000	1.000
Closet wiping with cloth	1.000	1.000	1.000	1.000
Table wiping with cloth	1.000	1.000	1.000	1.000
Mixing food with spoon	0.995	0.892	1.000	0.943
Eating food	1.000	1.000	1.000	1.000
Folding clothes	1.000	1.000	1.000	1.000
Stirring food	1.000	1.000	0.942	0.970
Cross-slope	0.998	0.941	0.914	0.928
Pushing uphill	0.998	0.933	0.933	0.933
Rolling downhill	1.000	1.000	1.000	1.000
Single loop	0.997	0.974	0.987	0.981
Double loop	0.997	0.984	0.984	0.984

Discussion

In this study, we demonstrated with high accuracy an individualized WP tracking system using a subject-specific ML algorithm and wearable sensors. We were able to record training data during a WP practice session and use it to predict overground propulsion. We found that, for detecting specific WP patterns, one needs only 3 min of data for each type of propulsion pattern to achieve high accuracy. This short recording session can be easily implemented into an inpatient rehabilitation session for MWUs. We also established knowledge between different prediction accuracies in terms of choice of ML algorithm, type of sensors, and epoch windows. Importantly, we demonstrated that it is possible to distinguish daily activities from manual wheelchair usage.

The results of our study follow the ML literature that WP data can be recorded in the laboratory in a controlled setting. French et al. demonstrated, with a single-subject study, the possibility of using ML to classify four different propulsion patterns with fairly high accuracy on a dynamometer.[38] However, the accuracy of recording on other surfaces (e.g., tile or asphalt) drops below 90%. They also found that kNN predicts propulsion patterns better than the SVM model (in which they used a radial kernel). In our study, we demonstrated not only the possibility of predicting WP with high accuracy, but also the feasibility of integrating the protocol into a WP training program using the WMS. Other studies have attempted to document different wheelchair activities but did not include data as rich in context and type of propulsion as our current study.[35,46,49] In human activity recognition literature, several researchers have already documented numerous methods of predicting daily activities.[36,37,40] However, most of these daily activities focus on dissociating the posture of the user (e.g., standing, sitting, lying) from movement (e.g., ambulatory or sedentary). Our study is innovative, because we provide higher granularity of the type of activities and demonstrate the potential separation of wheelchair-related activities from all other activities.

There are several limitations to this study. First, this is a single-subject study. It is very possible that optimal settings may change across different subjects. However, this type of design is common in initial feasibility testing in human activity recognition research.[36,38,50] Because the main goal of this pilot study was to test the feasibility of implementing ML algorithms and to determine parts of the experimental protocol, we believe that this study provides important information for further development. Second, in Study 2, we collected daily activity movement patterns from an able-bodied individual instead of an MWU. The reason for this was to reduce the burden on the MWU in this study. It is very possible that the movement patterns of an MWU are different from those of able-bodied individuals when performing the eight daily activities. However, the objective was to test the possibility of dissociating different activities from WP. These results provide proof-of-concept for further investigation. Another limitation to this study was the limited number of activities included. It is possible that we have not encountered activities that will increase the chance of error in monitoring wheelchair maneuvers. In future research, we will include more defined activities and a null category in which all non-defined activities will be recorded. We will also further investigate the possibility of recording ML data from different propulsion patterns before and after a WP training program. The results of the future study can inform us how to implement this method into inpatient rehabilitation and further the possibility of tracking post-rehabilitation progress for MWUs.

In conclusion, the current study demonstrates the potential for using ML and wearable sensors to track WP. Current literature has already shown laboratory-based evidence for improving WP. However, wheelchair maneuvers often change depending on environmental factors such as terrain, ramps, and cross-slopes. It is imperative to provide more evidence of community-based WP and wheelchair usage to fully understand the prevalence of upper limb injury. This study further establishes a tool to track wheelchair usage in the lived environment, enabling researchers to further understand the topic. Further development of this tool is currently underway. Our hope is to provide a tool to not only understand but eventually provide feedback through mobile devices as a wheelchair usage intervention.