Validity of the Samsung Phone S Health application for assessing steps and energy expenditure during walking and running: Does phone placement matter?

BACKGROUND: The emergence of mHealth and the utilization of smartphones in physical activity interventions warrant a closer examination of validity evidence for such technology. This study examined the validity of the Samsung S Health application in measuring steps and energy expenditure.
METHODS: Twenty-nine participants (mean age 21.69 ± 1.63) participated in the study. Participants carried a Samsung smartphone in their non-dominant hand and right pocket while walking around a 200-meter track and running on a treadmill at 2.24 m∙s-1. Steps and energy expenditure from the S Health app were compared with StepWatch 3 Step Activity Monitor steps and indirect calorimetry.
RESULTS: No significant differences between S Health estimated steps and energy expenditure during walking and their respective criterion measures, regardless of placement. There was also no significant difference between S Health estimated steps and the criterion measure during treadmill running, regardless of placement. There was significant differences between S Health estimated energy expenditure and the criterion during treadmill running for both placements (both p < 0.001).
CONCLUSIONS: The S Health application measures steps and energy expenditure accurately during self-selected pace walking regardless of placement. Placement of the phone impacts the S Health application accuracy in measuring physical activity variables during treadmill running.

There were 6.9 billion mobile subscriptions worldwide in 2014 which equated to around 4.5 billion mobile users worldwide.[1] Smartphone shipments worldwide were just over 1 billion units in 2014 and are projected to be 1.7 billion in 2017 (71% of the Smart connected device market).[1] Mobile health (mHealth) has emerged as an important field for disease management, health behavior assessment, and health behavior interventions.[2-5] mHealth is a term used for the practice of medical and public health supported by mobile devices and is used most commonly in reference to using mobile communication devices for health services and information.[6] Mobile and connected technologies have been adopted for these purposes because they offer novel approaches to measurement and intervention methodologies.[4] The highest mobile phone use in the USA occurs in adolescents, young adults, socioeconomically disadvantaged populations, people who rent their homes or frequently change address.[5] These populations are frequently overlooked in population based physical activity research. Because mobile phones are more likely to be carried throughout the day, the value of smartphones as a medium for measuring and influencing physical activity in real time has been amplified.[3,4,7,8] Advantages of smartphone use in physical activity research include increasing participant recruitment and compliance; transmission of data wirelessly over long distances and from remote locations without retrieval of a device; and an ability to program smartphone software to the researcher’s specifications.[8]

The Samsung Electronics Company has led the smartphone industry as top vendor, most shipments, and percent market share in 2012 and 2013 with its nearest competitor being Apple Incorporated.[9] Samsung smartphone devices use an Android operating system developed by Google as opposed to Apple smartphone devices, which use an iOS operating system developed by Apple Incorporated. According to Middleweerd et al.,[3] smartphone applications (apps) that promote physical activity have become popular: of the 875,683 active apps available in iTunes and the 696,527 active apps in Google Play, 23,490 and 17,756 were categorized as health and fitness, respectively. The popularity of these apps demonstrates that people are looking for ways to promote physical activity and monitor their behavior through smartphones.[6] Samsung smartphone devices were the first to include an app (S Health) that allowed users to manage their overall health and well-being through capturing and tracking health-related information and metrics that shared proprietary rights with the manufacturer. The S Health app provides users with physical activity information related to calories burned, distance covered during activity, and steps taken. Recent literature reviews examining mHealth and smartphone applications have indicated that evidence for its use is developing and that more research is needed to validate the measurement of physical activity to other criterion measures being used in the literature.[2,4,7] To date, there are limited studies that have examined health apps’ ability to measure components of physical activity in Android devices and no studies that have included the S Health application. Therefore the primary purpose of this study was to examine the validity of the Samsung S Health app in measuring steps and energy expenditure during walking and treadmill running. The study also wanted to determine if smartphone placement impacted the app’s measuring capabilities.

Methods

Participants

Twenty-nine (15 females and 14 males) young healthy adults mean age (21.69 ± 1.63) participated in the study. All participants completed a Physical Activity Readiness Questionnaire (PAR-Q) to determine if there were any contraindications or physical limitations affecting their ability to participate in the study. A positive response to any of the PAR-Q questions excluded a participant from the study. No participants were excluded based on PAR-Q responses. Participants were recruited from the university community and had to be ambulatory as well as be able to run for at least 15 minutes at 2.24 m·s−1 on a treadmill. Height and weight were assessed in light clothing and without shoes to the nearest 0.5 cm and 0.1 kg, respectively. Body mass index was calculated from height and weight measurements for all participants. Prior to participation, all participants had the research study and its potential risks and benefits explained fully before providing written informed consent. The Institutional Review Board approved all procedures. Physical characteristics of study participants are provided in Table 1.

Table 1.

Demographic characteristics of participants.

	Female (n = 15)	Male (n = 14)	All participants (N = 29)
Age (years)	21.2 ± 1.6	22.2 ± 1.6	21.7 ± 1.6
Height (cm)	166.8 ± 5.3	180.0 ± 5.8	173.1 ± 8.7
Weight (kg)	64.4 ± 4.6	82.3 ± 10.7	73.0 ± 12.1
BMI (kg·m−2)	23.1 ± 2.0	25.8 ± 3.1	24.4 ± 2.9
Self-selected pace (m·s−1)	1.5 ± 0.2	1.3 ± 0.1	1.4 ± 0.3

±: standard deviation; cm: centimeter; kg: kilogram; kg · m−2: kilogram per meter squared; m · s−1: meter per second; BMI: body mass index

Instruments

Two Samsung Galaxy S4 smartphones were used in this study. The dimension of the smartphones were 136.6 mm × 69.8 mm × 7.9 mm and they weighed 130 g. The phones were 4G LTE with built-in accelerometers and came loaded with the S Health app (version 2.0.0.009). For the duration of the study, all notifications for the S Health app updates were denied to ensure that both phones used the same app version. The S Health app was accessed from the smartphone app home screen. Participant profiles were created each time and required participants’ identification number, gender, date of birth, height, and weight. The app profile also requires the selection of an activity level (five levels) ranging from little to no activity to very heavy activity. For the purposes of this study, the moderate activity level was selected for all participants. After participant profiles were created, the pedometer option from the main screen was selected to access the S Health application’s built-in step counter. This option allowed the researchers to start and stop the smartphones during testing and also allowed for estimated steps and calories to be recorded.

The StepWatch 3 (version 3.4) activity monitor (Modus Health LLC, Washington, D.C., USA) was used as the criterion measure of steps taken because it is the most accurate pedometer ever designed for ambulatory movement, and it is capable of detecting the number of actual steps taken to within ±3% for speeds ranging from 1 to 5 mph.[10,11] The StepWatch 3 has also been used and supported as a criterion measure of steps taken in previous research in both laboratory and free-living settings.[10,12-15] It is an ankle-worn, microprocessor-controlled step counter. It is secured around the lateral side of the ankle with an elastic band and Velcro. The device measures 75 mm × 50 mm × 20 mm and weighs approximately 38 g. The StepWatch3 was programmed with participants’ gender, height, weight, and age. For this study we had the device record data in 15 s epochs, which were then downloaded to a computer using a docking station for determination of steps taken. The StepWatch 3 only records steps taken on one leg, therefore the values were multiplied by two to determine total steps. All participants wore two StepWatches (“Calibrated” option on left ankle and “Quick Stepping” option on right ankle). The “Calibrated” option allows researchers to set subject preferences that include walking speed, range of speed, and leg motion. The “Quick Stepping” option allows researchers to disable the subject preference features mentioned above and is recommended by the manufacture to accurately identify steps taken during running, jogging, or more vigorous activity.

Both StepWatch options were used in this study to determine their level of performance for running due to dearth of research available on either option. Prior to participants performing the two tasks in the protocol procedures, each participant was instructed to walk on a treadmill with gradient set at 0% for 30 s at 1.34 m·s−1 while researchers counted actual steps taken (via tally counter) and steps taken (via red LED flashing lights) for the StepWatch “Calibrated” option. After the walking task, each participant straddled the treadmill and researchers increased the speed to 2.24 m·s−1. Participants were then instructed to run on the treadmill with gradient set at 0% for 30 s at this speed while researchers counted actual steps taken (via tally counter) and steps taken (via red LED flashing lights) for the StepWatch “Calibrated” option. The StepWatch “Calibrated” steps were time stamped and recorded. After the completion of these tasks, the time stamped StepWatch “Calibrated” option worn on the left ankle was used to synchronize corresponding data from the StepWatch “Quick Stepping” option worn on the right ankle for each participant. This allowed researchers to ensure that recorded steps from both monitors were aligned for analysis. Mean absolute percent error (MAPE) was calculated for actual steps taken and the Stepwatch “Calibrated” option worn on the left ankle for walking (0.12 ± 0.02%) and running (8.79 ± 2.56%). MAPE was also calculated for actual steps taken and Stepwatch “Quick Stepping” option worn on the right ankle for walking (4.2 ± 0.88%) and running (0.09 ± 0.03%). Lowest MAPE StepWatch option was used for criterion and subsequent analysis for walking at self-selected pace and treadmill running.

Energy expenditure was measured by indirect calorimetry using the Cosmed K4b2 (Cosmed Pulmonary Function Equipment, Rome, Italy). The Cosmed K4b2 is a lightweight device, which has been shown to be valid for measuring VO2 and VCO2, compared with the Douglas Bag method, during cycle ergometry.[16] Prior to each test the oxygen and carbon dioxide analyzers and the flow turbine were calibrated according to the manufacturer’s instructions, which consisted of a room air calibration, gas calibration, volume calibration, and a delay calibration. The Cosmed was worn on the torso and connected to a mask which was used to measure oxygen consumption by analyzing the concentration of gases inhaled and exhaled. Data from the Cosmed was stored in memory and downloaded to a computer at the completion of each test. Breath-by-breath data were collected by the Cosmed K4b2, which were averaged over a 15-s period.

Protocol

Two Samsung Galaxy S4 phones with the S Health app recorded step counts and energy expenditure while walking on an indoor 200-m track and running on a motor-driven treadmill at 2.24 m·s−1. StepWatch 3 activity monitor and indirect calorimetry served as the criteria for step counts and energy expenditure, respectively. The testing session lasted almost 90 min per participant. All participants wore either athletic shorts or pants with pockets and comfortable athletic shoes. Participants were first asked to complete a 15 minutes 200-m walk around an indoor track at self-selected pace with one pre-programmed Samsung smartphone being held in their non-dominant hand (SH-H) and the other pre-programmed smartphone placed in their right pocket (SH-P). Participants stopped at the initial starting point on the track, the pre-programmed smartphones were initiated by the researchers and proper phone placement was ensured. Participants were then instructed to walk the 200-m track at a self-selected pace for 15 min. At the end of the 15 min, the participants were told to stand still, and the number of steps, distance traveled, and energy expenditure detected by the two smartphones was recorded. Researchers recorded the start and end times for the trials using a wrist worn sports watch.

Participants were then instructed to run on a treadmill at 2.24 m·s−1 with gradient set at 0% for 15 min. Participants stood still (straddling the treadmill belt) to allow the pre-programmed smartphones to be initiated by the researchers and to ensure proper phone placement. Participants were then instructed to run on the treadmill for 15 min. At the end of the 15 min, participants stood still (straddling the treadmill belt), and the number of steps, distance traveled, and energy expenditure detected by the two smartphones was recorded.

Statistical analysis

Means and standard deviations were reported for descriptive data. One-way within subjects repeated measure analyses of variance (ANOVAs) were used to assess significant differences between criterion steps taken/energy expenditure and step counts/energy expenditure registered by the two smartphones placements (SH-H and SH-P) during self-selected pace walking and running at 2.24 m·s−1 on a treadmill. Post-hoc analysis for the ANOVA procedures were performed if significance were found using pairwise comparisons with Bonferroni adjustments. The ANOVAs for self-selected pace walking and running on a treadmill examining differences between the StepWatch 3 and both smartphone placements only used 28 participants’ data due to missing values. The ANOVA for self-selected pace walking examining differences between indirect calorimetry and both smartphone placements only used 27 participants’ data due to missing values. Missing values were due to device programming errors (SH-P app did not register steps) and equipment malfunction (Cosmed sampling line became unplugged during walking activity). Pearson product moment correlations were analyzed to examine the relationship between the S Health apps estimated steps/estimated energy expenditure by phone placement and the criteria (StepWatch 3 and indirect calorimetry). Bland–Altman plots were also used to demonstrate level of agreement between criterion measures and estimated step counts/estimated energy expenditure registered by the S Health app’s two smartphone placements during self-selected pace walking and running at 2.24 m·s−1 treadmill speed.[17] These plots provide a visual illustration of mean error score and 95% prediction interval. Prediction equations that show a tight prediction interval around zero are deemed more accurate. Data points below zero signify overestimations, while points above zero signify underestimations. Statistical analyses were performed using SPSS 18.0 (Statistical Package for the Social Sciences, Inc., Chicago, IL, USA). For all analyses, a p value of 0.05 was used to indicate statistical significance.

Results

There was no statistical difference for steps taken during self-selected pace walking (F (2, 54) = 0.88, p = 0.42) among the variables (smartphone placements and StepWatch 3). There was no statistical difference for steps taken during treadmill running at 2.24 m·s−1 among the three variables. However the SH-P step counts were approaching a statistical difference (p = 0.051) during this condition. There was also no statistical difference for energy expenditure during self-selected pace walking (F (2, 52) = 0.36, p = 0.70) among the variables (smartphone placements and indirect calorimetry). During treadmill running at 2.24 m·s−1, a significant difference was found for energy expenditure among the variables (F (1.100, 30.794) = 52.64, p < 0.001). Post-hoc analysis revealed that both smartphone placements were significantly different from indirect calorimetry (p < 0.001). Table 2 presents the estimated step counts and energy expenditure registered for SH-H and SH-P during self-selected pace walking and running on a treadmill.

Table 2.

Step counts and EE registered by the two Samsung devices by placement, for over-ground walking at a self-selected pace and treadmill running at 2.24 m·s−1.

	All participants (N = 29)
SSP SH-H steps	1730 ± 117
SSP SH-P steps	1767 ± 130
SSP StepWatch 3	1752 ± 92
2.24 m·s−1 SH-H steps	2389 ± 91
2.24 m·s−1 SH-P steps	2406 ± 79
2.24 m·s−1 StepWatch 3	2308 ± 179
SSP SH-H EE (kcal)	69.50 ± 14.53
SSP SH-P EE (kcal)	69.85 ± 13.94
SSP indirect calorimetry (kcal)	68.57 ± 14.86
2.24 m·s−1 SH-H EE (kcal)	189.59 ± 42.33
2.24 m·s−1 SH-P EE (kcal)	194.48 ± 43.96
2.24 m·s−1 indirect calorimetry (kcal)	143.96 ± 33.17

Values in bold indicate significant differences from criterion.

±: standard deviation; m · s−1: meters per second on treadmill; SSP: self-selected pace; SH-H: Samsung smartphone held in non-dominant hand; SH-P: Samsung smartphone placed in right pocket; EE: energy expenditure; kcal: kilocalorie

Figure 1 displays the Bland–Altman plots for SH-H and SH-P estimated step counts for self-selected pace walking and treadmill running. The Bland–Altman plots showed a high agreement between criterion steps and SH-P estimated steps during walking (Figure 1(b)). The mean bias was −12 ± 61 steps (95% limits of agreement = −132 to 107 steps). The remaining Bland–Altman plots showed a moderate–high agreement between criterion steps and smartphone placement. Figure 2 displays the Bland–Altman plots for SH-H and SH-P estimated energy expenditure for self-selected pace walking and treadmill running. The Bland–Altman plots showed a moderate agreement between indirect calorimetry and SH-P estimated energy expenditure during walking (Figure 2(b)). The mean bias was −1.5 ± 8.5 kcal (95% limits of agreement = −18 to 15 kcal). The remaining Bland–Altman plots showed a poor–moderate agreement between criterion energy expenditure and smartphone placement.

Figure 1.

Bland–Altman plots depicting error scores (StepWatch 3 steps – estimated step) for the S Health app by phone placement (SH-H: Samsung smartphone held in non-dominant hand; SH-P: Samsung smartphone placed in right pocket) during (a) self-selected pace SH-H, (b) self-selected pace SH-P, (c) treadmill running at 2.24 m · s−1 SH-H and (d) treadmill running at 2.24 m · s−1 SH-P; dashed lines represent 95% limits of agreement.

Figure 2.

Bland–Altman plots depicting error scores (actual – estimated energy expenditure) for the S Health app by phone placement (SH-H: Samsung smartphone held in non-dominant hand; SH-P: Samsung smartphone placed in right pocket) during (a) self-selected pace SH-H, (b) self-selected pace SH-P, (c) treadmill running at 2.24 m · s−1 SH-H and (d) treadmill running at 2.24 m · s−1 SH-P; dashed lines represent 95% limits of agreement.

EE: energy expenditure

Pearson product moment correlations involving SH-H and SH-P for step counts revealed a positive correlation for SH-P during self-selected pace (r = 0.90, p < 0.001) and a non-significant positive correlation for SH-H during self-selected pace (r = 0.33, p < 0.08). Pearson product moment correlations involving SH-H and SH-P for energy expenditure revealed positive correlations for both placements during self-selected pace walking (r = 0.63, p < 0.001 and r = 0.83, p < 0.001, respectively). During treadmill running at 2.24 m·s−1, both SH-H and SH-P had a positive relationship with energy expenditure (r = 0.58, p = 0.001 and r = 0.63, p < 0.001, respectively).

Discussion

The primary purpose of this study was to examine the validity of the Samsung S Health app in measuring steps and energy expenditure during walking at self-selected pace and during treadmill running at 2.24 m·s−1. The study also examined whether smartphone placement (SH-H and SH-P) impacted the S Health app accuracy during both conditions. Smartphone placement did not impact the S Health app’s ability to estimate steps and energy expenditure when compared with the criteria during self-selected pace walking. Further examination of the Bland–Altman plots and Pearson product moment correlations indicate that SH-P is the most accurate in detecting steps and energy expenditure during self-selected pace walking.

The S Health app’s ability to estimate steps during treadmill running by smartphone placement were not statistically different from the criterion. However, further examination of the Bland–Altman plots and Pearson product moment correlations indicate that these placements typically overestimated steps and had a negative relationship with the criterion. Smartphone placement had a negative impact on the S Health app’s ability to estimate energy expenditure when compared with the criterion during treadmill running. Further examination of Bland–Altman plots and Pearson product moment correlations for estimated energy expenditure from both smartphone placements indicated a greater tendency of overestimation and positive relationships with the criterion.

There have been very few studies that have examined the accuracy of smartphone devices with commercially available or privately developed apps to measure physical activity (steps or energy expenditure) and all have used the iPhone smartphone device.[8,14,18] Bergman et al.[14] used the iPhone 3G with three placements (pocket, waist, and arm) and a free pedometer app (iPedometer) to examine the accuracy of estimated steps during five treadmill walking speeds (0.9 m·s−1; 1.12 m·s−1; 1.33 m·s−1; 1.57 m·s−1; 1.78 m·s−1). The authors found that the iPedometer app with the phone placed in the right pocket during treadmill walking at 1.78 m·s−1 was the only one not significantly different from tally counted steps and concluded that this app was not valid in monitoring activity during treadmill walking. These results differ from the current study, which showed no statistical difference between SH-H and SH-P during self-selected pace walking at an average speed of 1.37 ± 0.28 m·s−1. One possible reason for the differences in results could be attributed to the shared proprietary rights of the S Health app and Samsung device accelerometer, which would lend itself to better accuracy when compared with commercially available apps that do not share these rights and have to rely on pre-determined algorithms that may or may not align with smartphone device hardware.[14]

Manohar et al.[18] and Nolan et al.[8] used the iPhone 3 G and iPod Touch, respectively, with various placements (middle of back, waist, arm, hand, trouser pocket, handbag, and backpack) and personally developed apps to examine the accuracy of estimated energy expenditure during treadmill walking (range 0.22 m·s−1 – 2.01 m·s−1) and running (2.25 m·s−1; 2.46 m·s−1; 2.70 m·s−1; 2.92 m·s−1; 3.14 m·s−1). Manohar et al.[18] concluded that their app demonstrated significant linear response (intraclass correlation coefficient >0.9, p < 0.001) while detecting walking energy expenditure and that placement of the iPhone impacted this relationship. Our study reported similar correlation data with energy expenditure in regard to placement (SH-H and SH-P), however, it should be noted that high correlation does not mean that two methods agree.[17] Nolan et al.[8] found that their app trended towards overestimation of energy expenditure at slower speeds and underestimation of energy expenditure at higher speeds for both treadmill walking and running with the iPod Touch placed on the back. These trends were not observed in our study, there was an overestimation of energy expenditure during treadmill running regardless of phone placement and mixed results for energy expenditure during self-selected pace walking. Possible reasons for the differences in results could be device placement (back versus SH-H versus SH-P) and device used (iPod Touch versus Samsung Galaxy S4).

When comparing our results with pedometer and accelerometer research that have examined measurement accuracy of step counts and energy expenditure, the S Health app performs as well during self-selected pace walking for both.[19-23] Unlike some accelerometers that improve in accuracy as movement speed increases, the S Health app step count accuracy appears susceptible at faster ambulatory movements when carried in the non-dominant hand or placed in the right pocket while running on a treadmill. Future studies should examine if other phone placements impact the accuracy of the app in registering steps at faster ambulatory movements. Also, because the average self-selected pace was 1.37 ± 0.28 m·s−1, future studies should examine whether the S Health app is susceptible to slower walking paces similar to pedometers and accelerometers.

Limitations of this study included the S Health app version (2.0.0.009) used to estimate steps and energy expenditure. Only one new updated version of the app was released during the testing of the app version used in this study. The researchers are not aware of the effects that an updated app version would have on the estimate of steps and energy expenditure. Another limitation of this study included the researchers’ selection of the “moderate activity” level option for the S Health app. The researchers did not determine whether any adjustments are made to energy expenditure estimates based on selection of activity type. This study only examined self-selected pace walking and treadmill running at a low speed, future research should examine how the S Health app performs at faster treadmill running speeds. With increased interest in measuring sedentary and light activities, future studies should examine how the S Health app performs during other common activities of daily living. Despite the researchers demonstrating the appropriateness for using the StepWatch 3 as the criterion for measuring steps during treadmill running in this study, further research is needed to examine how step rate impacts the StepWatch 3 step count accuracy during various treadmill running speeds.

It would be a futile endeavor to validate the enormous amount of physical activity apps available for smartphone devices. In an attempt to focus future research endeavors, apps that share proprietary rights with the smartphone device may be a good starting point. Recently, iPhone has released a Health app that allows for tracking of physical activity measures similar to the S Health app, with the exception that additional accessories have to be purchased to get results. Given that Samsung and Apple are the leading smartphone manufacturers, a logical next step would be to include these smartphone devices and compatible physical activity apps for validation studies across multiple populations that exhibit various ambulatory activity.

mHealth and the utilization of smartphone devices to track health and physical activity behaviors is an emerging field that will continue to grow along with technological advances. It has been suggested that the greatest potential of smartphone devices in physical activity research is the ability to reach populations with restricted access to interventions or healthcare information.[5,7] Because of the plethora of physical activity apps available, this field of study will require guidelines for monitoring and for standard content across applications that are supported with evidence of validity and that remain user friendly. This study provides preliminary evidence of validity for the S Health app and Samsung smartphone device in measuring physical activity in young adults where studies are lacking.