Development and Usability of a Life-Logging Behavior Monitoring Application for Obese Patients.

BACKGROUND: Obesity is a global epidemic. Behavior change monitoring using a smartphone application (app) can support weight management in obese patients. These apps must undergo usability testing, which is an important step in mobile healthcare app development. The current study aimed to develop a mobile app for behavioral monitoring and to test its usability including technical effectiveness, user efficiency, and user satisfaction for obese adults.
METHODS: Development of the Dr. Youth app components included information on behavioral monitoring indicators and their obesity subtypes. The usability of the app was tested with 50 obese adults in a university hospital. Participants were asked to complete eight tasks for evaluating the technical effectiveness of the app. The time to complete each task was measured to test user efficiency. To explore user satisfaction, each participant completed the System Usability Scale (SUS). Descriptive statistics were used to examine the mean user efficiency and SUS scores.
RESULTS: Fifty adults (14 men and 36 women, aged 20-59 years) who are obese (body mass index ≥25 kg/m2) were recruited. The mean age of participants was 42.6 years (standard deviation [SD], 10.8 years); their mean body mass index was 29.6 kg/m2 (SD, 5.7 kg/m2). The tasks were completed with a 99% success rate. The overall mean SUS score was 76.65 (SD, 15.43).
CONCLUSION: The Dr. Youth app shows acceptable technical effectiveness, user efficiency, and user satisfaction. Future study is warranted to establish the app's clinical efficacy.

INTRODUCTION

The prevalence of obesity has increased worldwide and is now considered a global epidemic. Globally, the number of obese people is 650 million1 and the rate of increase of obesity in Korea is similar to the global increase.2 Obesity is an important risk factor for chronic diseases such as arthritis, diabetes, hypertension, dyslipidemia, cardiovascular disease, and cancer.3 The socioeconomic costs of obesity in Korean adults are substantial.4 In order to control the socioeconomic burden attributable to obesity, effective strategies for managing obesity should be established and implemented.4

Since approaches targeting behavioral changes are important in obesity management, cognitive behavioral interventions, and lifestyle improvement programs are needed to meet users’ emotional needs. Moreover, a recent study of adult weight management has suggested that mobile health interventions may be effective.5 Because of their widespread use and accessibility, smartphone applications (apps) offer a substantial opportunity to support health behavior changes and weight management. Obese patients can manage their obesity efficiently with a smartphone app by managing behaviors, setting goals, and tracking information.

Usability describes a system’s capacity to allow users to carry out their tasks safely, effectively, efficiently, and enjoyably.5 Usability testing is an important step towards refining the development of a mobile healthcare app that can be used in the treatment of obesity. Users’ experiences with mobile apps vary depending on the type of device used; users often report difficulty using mobile apps. It is vital that the end user is considered throughout app development (particularly where the app is to be used in clinical cohorts) and that testing for both technical and clinical effectiveness is completed so that functionality can be optimized.6

The primary purpose of this app was not only to monitor lifestyle logging behaviors, but also to facilitate self-management by obese patients in hospital clinics and assist clinicians toward the effective counseling of patients with regard to cognitive behavioral therapy. A few studies have investigated whether the use of obesity-related behavioral monitoring apps helps patients as well as clinicians in time-limited hospital clinics. Moreover, if apps for weight management do not engage participants enough for them to want to continue using the app, the intended weight loss and maintenance effects might not be achieved.7 Therefore, we first evaluated the usability of the app in this study before conducting a full-scale effectiveness assessment.

METHODS

Development of the Dr. Youth app

The app’s overall content and usability were formulated and developed via qualitative investigation and supported by a team approach. The study team consisted of two family physicians, one neuropsychiatric specialist, and a nutritionist. The team met to discuss the project plan, and an app developer was employed to write the software for the app.

Obesity represents a heterogeneous condition according to demographic, clinical, and behavioral factors. Based on a study of the demographic, clinical, and behavioral typology of obesity in the United States,8 we paid particular attention to the content that differences in the demographic characteristics, clinical conditions, and behavioral factors create distinct subtypes of obesity. Hence, stratifying obesity by subtypes or by assessing different risks across distinct groups may benefit weight management. Obesity subtyping helps to identify which behaviors need to be targeted and in whom, and who will be helped by the target obesity management program. A study used data from the Yorkshire Health Study to investigate six distinct groups of obese individuals, namely heavy drinking males, young healthy females, affluent and healthy elderly, physically sick but happy elderly, unhappy and anxious middle-aged persons, and a cluster of people with very poor health.9 Considering clinical experiences as well as several studies using obesity subtypes, we defined seven categories of obesity subtypes according to demographic factors and behavioral factors to better understand the lifestyle differences of individuals and to provide personalized lifestyle logging and feedback for each participant. The following obesity subtypes were suggested for a hospital weight management clinic setting: heavy drinking, stress-related eating, sedentary lifestyle, depression or sleep problems, age-related weight gain, shift work-related weight gain, and binge eating patterns (Table 1). The main reason for subtyping obesity was to focus on the kind of behavior monitoring that should be emphasized for each type of obese participant.

Table 1

Obesity subtypes according to the obesity-related behavioral components

Category	Obesity subtype	Behavioral component
1	Frequent drinking/company night out	I frequently eat and drink with my coworkers or friends after work.
2	Stress	I frequently overeat when I feel like I’ve been stressed out/under stress.
3	Work at night/night shift	I work late at night or in night shifts and thus eat late.
4	Sedentary lifestyle	I eat a lot but don’t exercise much.
5	Depression or sleep problem	I eat too much when I feel depressed, or I have a sleep problem and usually eat to sleep.
6	Age-related weight gain	I’ve gained weight as I’ve gotten older. I put on weight when I started menopause.
7	Snacks/refreshments	I am not really hungry, but I need something to nibble on.

Typically, behavior change techniques for weight management include many complex and diverse components. Behavior change taxonomies have been used to identify the basic components that modify behavior within theoretical frameworks.10 The incorporation of evidence-based behavior change techniques in an app reportedly facilitates increased engagement and effectiveness.11 Thus, we incorporated several evidence-based behavioral techniques while developing our app, including commitment and goal setting (e.g., a participant could select the option “I will attempt to reach the target weight by a predefined date” when a participant initiated the app). We also added self-monitoring of corresponding behaviors for seven domains (monitoring “critical food” [a concept defined specifically for this study and explained later in this section], sleep, regular meal times, nighttime eating, sleep, stress level, and alcohol intake) and monitored feedback for each behavior (weekly feedback on the above-mentioned seven domains based on the logged information). Suggestions were provided accordingly for behavior improvements or encouraging positive behavior changes.10 We also encouraged participants to monitor their weight and provided feedback based on the weight changes.

The app was designed to collect patients’ behavioral information through dietary habit questionnaires12 and sleep time and stress evaluation using the Korean-translated Brief Encounter Psychosocial Instrument.13 The aim was to categorize participants into the obesity subtypes shown in Table 1. Dietary habit questionnaires were used to identify the dietary patterns of the patients and assess whether they have preferences for foods such as fried foods, instant foods such as ramen or fast foods such as pizza. We used the Korean-translated Brief Encounter Psychosocial Instrument13 to construct the questions, given that stress and sleep affect eating habits12 and that experiencing psychological problems can reduce the motivation to lose weight, leading to a high dropout rate in the weight management program.

We selected behavioral monitoring indicators based on previous studies and reports.8,14 Categories of behavioral monitoring indicators include regular meal intake, regular sleep time, frequency of alcohol drinking, frequency of critical foods intake, body weight, exercise frequency, and stress levels according to several obesity subtypes. Therefore, the app was designed in a “to-do list” format that included a life-log, critical foods, and other behavioral monitoring indicator information.

“Critical foods” is a concept we defined, consisting of foods that the patient is recommended to avoid. Unlike conventional food diaries, the critical food list to be avoided is set based on participants’ choices to monitor their “risk” foods. The list of critical foods included simple carbohydrates, such as white bread, cakes, cookies, and sugar-sweetened beverages, and red meats with high levels of saturated fat, such as pork belly or ribs. Participants can choose up to three items for their critical food lists and log their frequency of critical food intake so that they can effectively monitor this list instead of keeping a food diary. For example, if one participant eats too many cookies, they can add cookies to their critical food list and would be required to monitor how frequently they ate cookies and how many cookies they ate in a day.

The app had an algorithm and message pool configuration to improve user motivation. According to their previous adherence to weekly behavior monitoring, participants were categorized as nonusers (participants whose logging adherence was less than 30%), moderate users (participants whose logging adherence was 30%–50%), and highly active users (participants who logged at least 50% of their suggested behavior monitoring indicators). Motivation messages were randomly selected from the targeted pool and sent to personalize users’ adherence to the app. At the end of 1 month, participants were evaluated for their achievements in bodyweight reduction, and new goals or behavior monitoring indicators were modified to adapt to their new goal or behavior target.

The app also had a reporting section, which showed how participants had been monitoring their behaviors as well as body weight according to their performance and adherence levels. Management plans consisted of guided exercise programs as well as low-calorie healthy recipes, which were suggested to participants as shown in Figs. 1 and 2.

Figure 1

Screenshots of health behavior monitoring in the Dr. Youth application.

Figure 2

Screenshots of the life log and body weight report.

Participants

Patients at a university hospital outpatient weight management clinic were invited to participate in the study from November 1 to 30, 2018. The eligible population was patients with a body mass index (BMI) of 25 kg/m2 or more who were seeking weight control, had their own smartphones, and were able to download the Dr. Youth app (YSBio Co., Ltd., Seoul, Korea). Patients were excluded if they had a drug addiction or alcohol use problem or uncontrolled psychotic disorders. Patients who could not use the app or who were unable to answer the questionnaire were also excluded. This study was approved by the Ethics Committee of Seoul National University Bundang Hospital (IRB No. B-1810-499-306).

Typically, usability testing conducted with five participants can identify at least 85% of usability problems.15 Since our app was developed to provide personalized care linked with an outpatient weight management clinic, the number of problems found in the app was considered to be an important factor.16 Hence, we performed our usability study with at least 50 participants, a study size at which 98%–100% of the problems would be detectable. Table 2 presents the basic characteristics of the 50 participants (14 men and 36 women, aged between 20 and 59 years) The participants’ mean age was 42.6 years (standard deviation [SD], 10.8 years); their mean BMI was 29.6 kg/m2 (SD, 5.7 kg/m2).

Table 2

General characteristics of the study participants

Variable	No. (%)
Total number	50
Sex
Male	14 (28)
Female	36 (72)
Age (yr)
20–29	8 (16)
30–39	9 (18)
40–49	17 (34)
50–59	16 (32)
BMI (kg/m2)
≥25.0 and <30	34 (68)
≥30.0	16 (32)

BMI, body mass index.

Procedure: assessment of usability

Participants who met the eligibility criteria were recruited from a university weight management clinic. If patients agreed to participate in the study, they provided informed consent, and then were asked to download the app from the iPhone App Store or Google Play store. Participants were asked to perform several tasks and complete the 10-item System Usability Scale (SUS) questionnaire.17

Usability was defined as the extent to which the app could be used by a clinical cohort of obese adults to achieve specified tasks with technical effectiveness, efficiency, and satisfaction.18 This definition is in line with the standard ISO9241-11.19 Usability testing refers to the evaluation of information systems that involve participants who are representative of the target user population.20 Effectiveness refers to whether people can complete their tasks and achieve their goals. User efficiency is the extent to which users expend resources toward achieving their goals. User satisfaction is the level of comfort users experience in achieving those goals.

Participants were given eight tasks to complete in order to test the app’s technical effectiveness, user efficiency, and user satisfaction. Each task required, or requested, that the participant obtain or enter specific data. The tasks required the user to enter personal settings, use the goal-setting feature, and enter details regarding their lifestyle.

Technical effectiveness

The participants carried out eight tasks to test the app’s technical effectiveness. A task was completed when the task goal was obtained successfully without a critical error. A critical error was defined as an error resulting in an incorrect or incomplete outcome. A noncritical error was an error that would not affect the final output of the task, but caused it to be completed less efficiently. We analyzed the success and failure rates of participants’ task completion. A success rate of 80% or higher was considered to lower the importance of technical factors in usability assessment.

User efficiency

The user efficiency of the app was tested, with users being timed as they undertook tasks in order to examine whether the app was easy to navigate. User efficiency was measured by recording the mean time (in seconds) a user took to complete a task.21 The Single Ease Question (SEQ) was asked of participants at the end of testing as a quantitative evaluation of difficulty.22 The SEQ is a seven-point rating scale to assess how difficult users find a task, where a rating of 1 indicates that the task was very difficult and a rating of 7 indicates that it was very easy. The SEQ was administered immediately after a user completed a task.

User satisfaction

After the task was complete, the overall usability score was measured using the SUS questionnaires.17 SUS is a brief quantitative assessment of usability, which can be carried out using the questionnaire, and the results are valid and reliable. SUS is a scale ranging from 0 to 100, with scores over 80 and below 70 suggesting high usability and low usability, respectively.23 The ten SUS items were ranked according to a 5-point Likert scale.

In addition, after using the app for a week, the participants were asked to submit online feedback through a Google questionnaire system. Descriptive analysis was performed where the median values and 25%–75% intervals were displayed in continuous variables and frequencies with the percentages as categorical variables. When analyzing data according to sex, age group, or BMI group, we performed the Wilcoxon rank-sum test between two independent groups and the Kruskal-Wallis test among more than two groups. All analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA).

RESULTS

Technical effectiveness

All tasks were completed successfully, except for one task for one participant with a type 2 error. The total success rate of all tasks was about 99%.

User efficiency

The results of user efficiency are presented in Table 3. The longest time taken to complete a task was 120.5 seconds for task 1, and the shortest time was 9.5 seconds for task 6. Regarding the difficulty evaluation, the SEQ ratings indicated that the most difficult case was task 6, and the easiest one was task 1.

Table 3

Types of tasks and user efficiencies

Task	Description	Time (sec)			Difficulty (mean SEQ)
		Median	Q1	Q3
1	Sign up for membership and sign in.	120.5	99	155	6.16
2	Fill out and submit the consent form.	12	9	20	6.3
3	Go to the “Obesity subtype” section and select one. Then check the prescribed monitoring index.	14.5	10	25	6.36
4	Record your meal times and whether you eat snacks or eat late at night.	26	16	32	6.24
5	Go to the “Sleep time records” section and enter your daily sleep record.	18	12	31	6.34
6	Record the amount of alcohol you drink.	9.5	5	12	6.46
7	Record your stress level.	12	9	14	6.37
8	Set your own physical exercise goal and enter performance status.	14.5	10	25	6.4

Q, quartile; SEQ, Single Ease Question.

User satisfaction and usability

The SUS scores corresponding to the ten items are presented in Fig. 3. Odd-numbered items indicate a positive response, while even-numbered items indicate a negative response. Higher numbers indicate increasing degrees of participant agreement.

Figure 3

Mean System Usability Scale scores corresponding to the 10 items. app, application.

The results of the SUS are presented in Table 4. The overall mean SUS score was 76.65 (SD, 15.43). The SUS scores of 37 participants (74%) were 70 or higher, which indicated that the app was an acceptable system.24 There were no significant differences in SUS scores among different sex, age, or BMI groups.

Table 4

SUS scores according to sex, age, and BMI

Variable	SUS			P
	Median	Q1	Q3
Sex				0.442*
Male	81.25	72.50	87.50
Female	75.00	65.00	87.50
Age (yr)				0.798†
20–29	78.75	76.25	86.25
30–39	72.50	65.00	87.50
40–49	80.00	65.00	97.50
≥50	73.75	67.50	86.25
BMI (kg/m2)				0.700*
≥25.0 and <30	77.50	65.00	87.50
≥30	76.25	71.25	92.50

Wilcoxon rank-sum test;

Kruskal Wallis test.

SUS, System Usability Scale; BMI, body mass index; Q, quartile.

Thirty-eight of the 50 participants submitted responses to our Google questionnaires after using the app for a week (a 76% response rate). The most frequent comments made by participants throughout testing of the app to each question were as follows. When asked, “What are the good points when using the app?,” the most frequent response was “It was simple and good.” When asked, “What did you dislike when using the app?,” the most frequent response was “I felt sorry that I couldn’t enter the details.” When asked, “Are there any improvements or suggestions to this app?,” the most common response was “I hope the programs and recording section of the app will be more specific.”

DISCUSSION

Overall, our weight management app was rated as easy to use, and each of the test tasks was successfully completed without critical errors. Evaluation of the usability of smartphone apps is critical for their success, allowing developers to adapt and improve them against the rapidly changing backdrop of mobile technology. App functions that are difficult to use or to understand might result in a significant decrease in usability. These factors are common problems and shortcomings of weight-control mobile apps that are currently available.

Very few studies have reported on the development and use of mobile phone apps for individual behavioral change or weight monitoring.25 Hebden et al.26 developed and tested four mobile phone apps to improve nutrition and physical activity behaviors during young adulthood. Qualitative feedback provided few suggestions for content changes in these apps, with the major concerns being their slow running speed and login requirements. Dennison et al.27 reported that healthy young adults displayed some interest in using health care apps that support behavior change. Legitimacy, effort required, and immediate effects on mood emerged as important influences relating to app usage, as users of behavior-change apps may lose motivation over time and engage in only intermittent use. Whether this view is applicable to obese patients and to long-term users remains to be seen. O’Malley et al.6 tested the usability of a mobile app used in adolescent obesity management. They described the importance of usability testing in refining the development of an app for use in the treatment of obesity.

Considering these earlier studies, usability testing is clearly paramount for the optimal design and development of mobile apps used in clinical settings. The objective of SUS, therefore, was to provide us with a measure of people’s subjective perceptions of system usability in the very short time available during an evaluation session. Tullis and Stetson’s research showed that using SUS provides a measure of the perceived usability of a system with a small sample and allows us to be fairly confident in obtaining a good assessment of how people see the system.28

Overall, the test results were promising, and participants rated the Dr. Youth app as an acceptable, easy-to-use system. Each test task was completed successfully, with only one critical error, indicating the app’s technical effectiveness. Users were satisfied with the app and suggested several ways to improve it further. The mean SUS score of 76.65 and the 74% rating for acceptable usability were promising. Given the current popularity of mobile apps, we anticipated that a mobile app would be a useful tool for obesity treatment, and this study’s results support our view.

Some limitations of this study must be acknowledged. Usability has an initial assessment process that is followed by a feedback process when the app is constantly used and verified. This study comprises the initial assessment process. Thus, further research will be needed to establish clinical evidence with continuous feedback on the app’s usability. The development and the usability testing of the app in the study were two different processes, which makes it difficult to define the distribution of obesity subtypes or Korean-translated Brief Encounter Psychosocial Instrument (BEPSI-K) scores of the participants recruited for the usability testing of the app. Additionally, we did not collect participants’ educational or socioeconomic statuses, which might be a potential limitation of the study. Furthermore, obesity subtyping in this study was not based on domestic data. Considering differences in the demographic characteristics of obese patients, it may be meaningful to define obesity subtypes using domestic data unique to Korea in the future, followed by usability testing of an app for obese patients. In addition, the app should be tested in a cohort of obese adults who are not attending a clinic for weight management, as we do not know whether the users’ level of motivation for lifestyle change affects their perceptions of technical usability. Finally, we assessed the users’ satisfaction with using the app as a whole, rather than their satisfaction with completing each task. Future work to explore individual components of the app may also be warranted. Despite these limitations, our study demonstrated the good usability performance of the Dr. Youth app for weight management.

Overall, the Dr. Youth app performed well in usability testing, with results providing support for its usability by end users. The results of this study guided the final development cycle of the app prior to its use. Usability testing of mobile apps designed to address clinical problems is vital, as it enables developers to account for users’ needs and to better optimize the mobile app’s acceptability and utility.