A validation study of a smartphone application for functional mobility assessment of the elderly.

BACKGROUND: To minimize the reaction time and position judgment error using stopwatch-timed measures, we developed a smartphone application to measure performance in the five-time sit-to-stand (FTSTS) and timed up-and-go (TUG) tests.
OBJECTIVE: This study aimed to validate this smartphone application by comparing its measurement with a laboratory-based reference condition.
METHODS: Thirty-two healthy elderly people were asked to perform the FTSTS and TUG tests in a randomized sequence. During the tests, their performance was concurrently measured by the smartphone application and a force sensor installed in the backrest of a chair. The intraclass correlation coefficient [ICC(2,1)] and Bland-Altman analysis were used to calculate the measurement consistency and agreement, respectively, between these two methods.
RESULTS: The smartphone application demonstrated excellent measurement consistency with the lab-based reference condition for the FTSTS test [ICC(2,1) = 0.988] and TUG test [ICC(2,1) = 0.946]. We observed a positive bias of 0.27 seconds (95% limits of agreement, -1.22 to 1.76 seconds) for the FTSTS test and 0.48 seconds (95% limits of agreement, -1.66 to 2.63 seconds) for the TUG test.
CONCLUSION: We cross-validated the newly developed smartphone application with the laboratory-based reference condition during the examination of FTSTS and TUG test performance in healthy elderly.

Introduction

Many smartphone applications for medical purpose have been developed and widely used by healthcare providers to facilitate medical diagnosis, evaluation, patient education, and treatment [1,2]. It is not surprising that nearly 80% of medical students and 75% of postgraduate trainees own smartphones [3]. This trend is expected to grow as this handheld device allows portable computation of the data obtained from inbuilt sensors. A smartphone application can also be custom-made because it is capable of running third-party software.

Aging is a global health issue and the functional mobility level has been associated with fall risk [4], rehabilitation outcome [5], disability level [6], quality of life [7], and mortality [8] in the elderly. Many clinical tests exist to measure the functional mobility in the elderly population. Among them, the five-time sit-to-stand (FTSTS) test and the timed up-and-go (TUG) test are two of the most common tools used for clinical evaluation.

The FTSTS test was initially designed to measure the functional strength of the lower extremities [9]. It is also widely used to assess the rehabilitation progress, balance dysfunction, and functional performance in elderly people with musculoskeletal or neurological conditions clinically [10-13]. The TUG test is a commonly used functional test to evaluate basic mobility [14], self-independence in daily life [15], and fall risk [16] in elderly people. The performance of both tests is clinically quantified by a rater using a stopwatch. In contrast to stopwatch-timed measures, quantitative measurement using the inbuilt sensors in the smartphone would eliminate potential human errors, which include the reaction time delay in using a stopwatch [17] and human error for position judgment. Hence, the measurement accuracy could be enhanced by a robust measurement using a custom-made smartphone application.

Therefore, this study aimed to examine the measurement consistency and agreement of a newly established smartphone application with respect to a laboratory-based reference condition. We hypothesized that the measurement using smartphone application would be comparable to the findings obtained from the laboratory-based reference condition.

Materials and methods

Participants

Thirty-two participants (21 women; age, 70.7 ± 6.5 years) were recruited from a local elderly centre. All participants were independent in all activities of daily living and they did not need any walking aids during locomotion. The experiment procedures were approved by the Departmental Research Committee, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University. Written consent was obtained from all participants before the test. The sample size was justified by the method proposed by Liao [18]. In brief, assuming no discordant pair of measurements was allowed and using alpha and beta at 0.05 and 0.8, respectively, 32 participants were required for this study.

Testing procedures

All participants were evaluated for their performance in the FTSTS test and TUG test in a randomized order. The random sequence was generated by an online program (www.random.org). For each test, a demonstration was provided and two practice trials were allowed before the actual test began [19].

The FTSTS test measured the time taken to complete five repetitions of the sit-to-stand manoeuvre as quickly as possible. All participants were asked to sit on an armless chair at 43 cm in height [6]. Before the test, participants crossed their arms over their chest, sat upright with their back in contact with the backrest of the chair. The correct manoeuvre was demonstrated and included coming to a full stand (defined as an upright trunk with the hips and knees extended). The participants had to lean their back against the backrest at the end of each repetition. The TUG test measures the time it takes for a participant to stand up from a chair with the armrests at 46 cm in height, walk for 3 meters to a mark on the floor, turn around, return to the chair, and sit down [20]. The task should be performed at a self-paced comfortable walking speed. The test ended when the participants resumed the starting position [21]. During the test, a participant’s performance was concurrently measured by the smartphone application and the force sensor.

The algorithm of the smartphone application was based on the data collected from the three-dimensional inertial measurement unit (IMU) built in an android-based smartphone (Galaxy Note II; Samsung Electronics Co. Ltd, Suwon, Korea). The phone was securely affixed onto a participant’s chest by Velcro straps during the test. Before actual data collection, we calibrated the starting position in the FTSTS test and TUG test by obtaining the three-dimensional IMU data for 5 seconds. The static standing position was also collected for 5 seconds for the FTSTS test.

A beep sound followed by an audio script of “3 … 2 … 1” cued the participant to start. In the TUG test, the time began to be counted after the beep and was stopped when the smartphone returned to its original position. In the FTSTS test, the smartphone continued time-counting until it detected the last cycle of the stand-to-sit manoeuvre. The application could be downloaded via the quick response (QR) code at Appendix 1.

A force sensor (YZC-516; Guangzhou Electrical Measuring Instruments Factory, Guangzhou, China) was installed at the backrest of the test chair. In the FTSTS and TUG tests, time was measured from the moment when the body lifted off from the backrest until the time when the force sensor detected contact. Before starting each trial, the sensor was calibrated. The measurement collected from this reference condition was regarded as the gold standard in this study.

Statistical analysis

Measurement consistency between the smartphone application and the reference condition was compared using two-way random-effects intra-class correlation [ICC(2,1)]. Bland–Altman analysis was used to assess the agreement between two measuring methods [22,23]. A zero bias represented no difference between the estimated time and the reference time; a negative bias (i.e., the time measured using the smartphone application minus the reference time) indicated an underestimation of the time by the smartphone application; and a positive bias corresponded to an overestimation of the measured time. Statistical analyses were performed using SPSS version 17.0 (SPSS Inc., Chicago, IL, USA) and GraphPad Prism version 5.01 (GraphPad Software Inc., La Jolla, CA, USA).

Results

The measurement consistency between smartphone application and reference condition was excellent: the ICC (2,1) in the FTSTS and TUG test were 0.988 (95% confidence interval, 0.976–0.994) and 0.946 (95% confidence interval, 0.889–0.973), respectively. Bland–Altman analysis showed a positive bias of 0.27 seconds (95% limits of agreement, from −1.22 seconds to 1.76 seconds; Figure 1) for the FTSTS test and 0.48 seconds (95% limits of agreement, from −1.66 seconds to 2.63 seconds; Figure 2) for the TUG test. The results demonstrated that smartphone application overestimated the time required during the selected physical tests.

Figure 1.

The 95% limits of agreement for the five-time sit-to-stand test.

Figure 2.

The 95% limits of agreement for the timed up and go test.

Discussion

This study validated a newly developed smartphone application for assessing the performance of FTSTS and TUG tests in elderly individuals. We found excellent measurement consistency between the smartphone application and the reference condition. However, the result demonstrated that a positive bias of 0.27 seconds and 0.48 seconds for the FTSTS test and the TUG test, respectively, in the smartphone application. Such overestimation in the measurement could be explained by the time lag existing between the audio cue and the initiation of movement. Another explanation of the measurement difference may be attributed to the different mechanism used to detect the test end point.

The reaction time between the audio feedback and participant’s actual movement was included in the smartphone application. By contrast, the measurement by force sensor only accounted for the time spent in the physical tasks. Thus, the influence of participants’ reaction time was eliminated in the reference condition. Furthermore, the original sitting position was recorded in the smartphone application before the test. Participants may lean on the backrest naturally during calibration but that position may be different with the position where the body was just in contact with the backrest. Positive biases therefore occurred for these reasons. The reaction time for healthy elderly ranged from 0.27 seconds to 0.35 seconds [6]; therefore, the true biases would be even less than the findings reported in the Bland–Altman analyses. A mean difference of ≥ 0.63 seconds in the TUG test may indicate higher fall risk in the elderly [24]. Taken together with the minimal detectable differences in the FTSTS (0.54 seconds) and TUG tests (0.63 seconds) [24,25], the average measurement biases presented in our smartphone application should be clinically negligible. However, if we considered the greatest possible biases (i.e. 1.76 seconds and 2.63 seconds in the FTSTS and TUG tests respectively), the accuracy of the smartphone application may need further improvement.

During the early phase of the sit-to-stand or stand-to-sit manoeuvre in a healthy elderly person, the trunk must bend forward for ~40° [26,27]. Because this smartphone application measures the test performance by using the orientation of the trunk, our smartphone application algorithm in measuring performance of frail elderly people remains unanswered because frail elderly people have a lower change in sagittal plane acceleration during the sit-to-stand or stand-to-sit in comparison to healthy elderly people [28].

Limitations

This study had several limitations. First, we did not test the smartphone application in any disease population and the generalization of our findings is therefore limited. Second, we did not compare the measurement agreement between the smartphone application and the clinical method. It remains unknown whether the smartphone application promotes a more robust measurement in the selected physical tests. Third, this smartphone application was developed using android codes. Smartphone of other operating platforms are therefore incompatible.

Conclusion

In this study, we cross-validated the newly developed smartphone application with laboratory-based reference condition during examination of FTSTS and TUG test performance in healthy elderly individuals. The positive bias presented by the smartphone application was not clinical significant.