Machine Learning Improves Functional Upper Extremity Use Capture in Distal Radius Fracture Patients.

Current outcome measures, including strength/range of motion testing, patient-reported outcomes (PROs), and motor skill testing, may provide inadequate granularity in reflecting functional upper extremity (UE) use after distal radius fracture (DRF) repair. Accelerometry analysis also has shortcomings, namely, an inability to differentiate functional versus nonfunctional movements. The objective of this study was to evaluate the accuracy of machine learning (ML) analyses in capturing UE functional movements based on accelerometry data for patients after DRF repair. In this prospective study, six patients were enrolled 2-6 weeks after DRF open reduction and internal fixation (ORIF). They all performed standardized activities while wearing a wrist accelerometer, and the data were analyzed by an ML algorithm. These activities were also videotaped and evaluated by visual inspection. Our novel ML algorithm was able to predict from accelerometry data whether the limb was performing a movement rated as functional, with accuracy of 90.4% ± 3.6% for within-subject modeling and 79.8% ± 8.9% accuracy for between-subject modeling. The application of ML algorithms to accelerometry data allowed for capture of functional UE activity in patients after DRF open reduction and internal fixation and accurately predicts functional UE use. Such analyses could improve our understanding of recovery and enhance routine postoperative rehabilitation in DRF patients.

Takeaways

Question: Do ML algorithms reliably capture functional upper extremity use data from accelerometry analysis in DRF patients following surgery?

Findings: Six patients treated surgically for DRF donned a wrist accelerometer, and the data were analyzed by ML algorithms. Our novel ML algorithm predicted from accelerometry data whether the limb was performing a movement rated as functional, with accuracy of 90.4% ± 3.6% for within-subject modeling and 79.8% ± 8.9% accuracy for between-subject modeling.

Meaning: Using ML algorithms to analyze accelerometry data helps to more accurately capture functional upper extremity activity in DRF patients following surgery, which can provide insight into optimizing postoperative rehabilitation with patient-specific granularity.

INTRODUCTION

Distal radius fractures (DRFs) are the most common upper extremity (UE) fracture, and the overall incidence is increasing worldwide.[1] Deciding on appropriate treatment after DRF remains a challenge, with multiple high-quality studies providing incongruous results that may indicate a lack of adequate patient-specific granularity in treatment decision-making.[2-5] Although there are several outcome measures, including strength/range of motion testing, patient-reported outcomes (PROs), and motor testing, they ultimately all fall short in accurately describing functional use of the UE.[6,7] Accelerometry analysis is an appealing alternative to better characterize UE use in daily living, but previous studies have failed to validate it as a reliable tool for differentiating functional versus nonfunctional use of the UE.[8-10] We describe our use of accelerometry analysis with machine learning (ML) algorithms to reliably track UE functional movements in DRF patients.

PATIENTS AND METHODS

This study was conducted between December 2020 and October 2021. Institutional review board approval was obtained. Patients with a DRF who underwent open reduction and internal fixation (ORIF) by hand surgeons in a regional group of tertiary care centers were asked to participate in the study. Patients were excluded if they had a baseline deficit in activities of daily living before DRF, history of abnormal bilateral UE function before DRF, or conditions that would impact their ability to ambulate or use their UE. Written informed consent was obtained.

Patients were asked to complete various PROs, including Michigan Hand questionnaire, Activity Card Sort Test, PROMIS UE short form 7a, Wong-Baker-faces pain scale, patient perception of change, and Simple Hand Score. Patients donned wrist accelerometers and performed an activity script of unstructured activities, such as laundry and shopping, while being videotaped in a supervised setting to simulate a typical home environment. Patients were counseled to perform activities as they would normally in an unsupervised setting.

Data Analysis

A full description of the approach has been reported elsewhere.[10] Briefly, each frame of the video was annotated by humans into functional and nonfunctional categories, and these ground truth labels were transferred to accelerometry data. We then trained a random forest classifier, an ensemble ML algorithm to classify data, on the accelerometry data. For the intrasubject models, stratified fivefold cross validation was used and reports intrasubject accuracy. For the intersubject models, leave-one-out cross validation was used since it is approximately unbiased and the sample size was small; it reports intersubject accuracy. Correlations between percent functional use from video versus from ML and between percent functional use and PRO data were analyzed using Pearson correlation coefficient.

RESULTS

A total of six patients with DRF who underwent ORIF completed functional testing and completed PROs 4–8 weeks following surgery. The mean patient age was 45.2 ± 8.2 years. All six patients sustained dorsally angulated DRFs. Two underwent concomitant carpal tunnel release and one underwent concomitant distal ulnar ORIF. Five patients were internally fixed with a volar locking plate, and one was fixed with a dorsal plate. On average, 19.5 days elapsed from surgery to therapy initiation and 34.2 days elapsed from surgery to study visitation.

The average functional use of the impaired UE determined via video analysis was 65.4% ± 17.9% (Table 1). Predicted functional use from the stratified fivefold cross-validation algorithm was 68.7% ± 21.5% for intrasubject modeling and 88.3% ± 4.8% for intersubject modeling from the leave-one-out cross-validation algorithm (Fig. 1). The average accuracy of the new adjunct ML algorithms in predicting functional use within and between DRF patients was 90.4% ± 3.6% and 79.8% ± 8.9%, respectively. The average correlation between functional use and PROs was 0.58 ± 0.19 (Table 2; Fig. 2). Correlation between intrasubject predictions and proportion of functional use from video analysis was 0.994.

Table 1.

Intrasubject and Intersubject Modeling of Functional Use

	Percentage of Functional Use from Video Analysis	Intrasubject		Intersubject
		Accuracy	Predictions	Accuracy	Predictions
Subject 1	0.344	0.891	0.295	0.644	0.850
Subject 2	0.624	0.890	0.665	0.768	0.845
Subject 3	0.679	0.864	0.745	0.779	0.862
Subject 4	0.725	0.889	0.788	0.847	0.921
Subject 5	0.656	0.923	0.692	0.868	0.963
Subject 6	0.894	0.968	0.939	0.884	0.857

Fig. 1.

Association between functional use by video analysis and functional use by machine learning.

Table 2.

PRO Scores and Correlation with Functional Use of UE

Participant	Michigan Hand Questionnaire	ACS Test	PROMIS UE Score	Simple Hand Score	Patient Perception of Change	%Functional Use
Subject 1	39.58	28	25	20	80	0.343918
Subject 2	35.41	29.5	25	30	64	0.62412
Subject 3	37.5	28.5	23.9	25	75	0.678797
Subject 4	45.83	33.5	23.9	40	79	0.725437
Subject 5	41.67	45	26.1	40	69	0.655685
Subject 6	47.9	34	33.9	60	65	0.894273

The average correlation between PROs and %functional use was 0.578. Michigan Hand questionnaire is a self-reported, hand-specific outcome measure to evaluate overall hand function, ADL, pain, and work performance. ACS is a self-reported, activity participation outcome measure that evaluates activity in four domains. PROMIS-UE Score is a self-reported UE function outcome measure that evaluates participants’ ability to perform activities that involve the UE. Simple hand score is a self-reported PRO in which patients rate hand function on a scale from 0 to 100. Patient perception of change is a patient-reported outcome of UE recovery in which patients use a 7-point Likert scale to rate the functional status of their affected arm.

ACS, Activity Card Sort Test; ADL, activities of daily living.

Fig. 2.

Association between functional use by video analysis and the simple hand PRO measure. The simple hand score is a self-reported PRO in which patients rate hand function on a scale from 0 to 100.

DISCUSSION

Our pilot study demonstrates an innovative ML algorithm for accelerometry analysis. This approach aims to reliably obtain functional data regarding UE use using data science enhanced accelerometry. Our preliminary results suggest that ML algorithms of accelerometry analysis can accurately predict functional UE use in patients after DRF ORIF.

According to the literature, many contemporary outcome measures, such as PRO scores, strength/range of motion testing, and motor testing, do not provide insight into functional use of the UE in DRF patients.[6,9,10] PROs have become the central focus of outcome studies across a variety of specialties and conditions; however, especially in hand surgery, the available instruments are all limited by some combination of ceiling/floor effects, lack of granularity, inconsistent or unreliable completion, and others that limit accuracy especially in differentiating between more subtle variations.[6,11-13] Accelerometry was introduced to better characterize unsupervised functional outcomes, although, in silo, accelerometry is unable to differentiate functional movements from nonfunctional ones. The optimization of accelerometry analysis via data science enhancement has been previously conducted to evaluate functional use of the UE in stroke patients.[10] In amputees, ML models applied to inertial wrist-worn sensors were shown to reliably separate functional movement from walking to greater than 95% accuracy for intrasubject modeling and greater than 80% accuracy for intersubject modeling, highlighting real-world application of these models for clinical rehabilitation.[14] Our application of ML algorithms of accelerometry data was able to predict with an accuracy of 90.4% ± 3.6% in within-subject modeling and 79.8% ± 8.9% in between-subject modeling compared with video analysis of functional use, similar to what has been previously described in other patient populations.

Our application of ML algorithms to accelerometry analysis yielded accurate predictions when compared with video analysis. Nevertheless, we acknowledge the limitations of our study, especially in the setting of a small sample size, no controls, and with tasks performed under observation and not at home. As is usually the case, especially with early pilot testing, performance of intrasubject is better than that of intersubject; however, with additional data and refinement, we believe that this gap will narrow substantially.

CONCLUSIONS

Our application of ML algorithms to accelerometry analysis was able to reliably capture functional UE use in DRF patients after ORIF when compared with video analysis. With refinement and more robust testing, this novel data science enhancement to accelerometry analysis can be implemented as a component of postoperative evaluation of patients with DRF to better understand and track their progress when in unsupervised, nonclinical settings.