Development and Usability Evaluation of a Healthy Lifestyle Coaching Chatbot Using a Habit Formation Model.

OBJECTIVES: It is crucial to find ways to fit regular exercise into the daily lives of office workers. Non-exercise activity thermogenesis has been introduced as an effective form of daily exercise. This study aimed to develop a healthy lifestyle coaching program for office workers, to be delivered using a messenger application.
METHODS: The interface was developed using KakaoTalk and Plus Friend. Performance feedback was developed using the IBM Watson conversation program. Twenty office workers used the application for three weeks. Afterward, a survey was conducted to assess the usability of and participants' satisfaction with the application.
RESULTS: The application delivered customized push alarms, provided information related to habit formation, allowed for one-on-one chats, and delivered rewards. The satisfaction measurement results for the application showed that extrinsic reward factors contributed the most to the performance of the activity, followed by reminders and intrinsic rewards. Regarding the usability test, the perceived usefulness of the Healthy Lifestyle Coaching Chatbot was highest, followed by the usage intent and the perceived ease of use.
CONCLUSIONS: This study found that coaching programs using chatbots can improve the effectiveness of performing simple, repetitive exercises.

I. Introduction

Lack of time to exercise due to a busy working life is a common problem experienced by many office workers. Therefore, it is crucial to find effective ways for workers to fit regular exercise into their daily lives. Non-exercise activity thermogenesis (NEAT), which refers to the action of expending energy through lifestyle physical activities such as walking and stair climbing rather than through intentional, planned, and structured activity, has been introduced as an effective and easy form of daily exercise [1]. This lifestyle intervention, along with physical activity, has been advocated in primary care as an effective instrument in reducing the problems related to being overweight or obese [2]. Maintaining lifestyle habits that increase NEAT can prevent further weight gain, even if effective weight loss is not expected. Moreover, this type of physical activity can become habitual because it is unconsciously repeated daily.

Habits are defined as “behavioral dispositions to repeat well-practiced actions given recurring circumstances” [3]. Previous experimental studies have proposed a habit formation model in which a habit is formed by inducing repeated behavior through cue-behavior-reward links [4]. Lally and Gardner [5] suggested that four stages are required to achieve automaticity. First, a decision to act must be made. Second, the intention to act has to be translated into behavior. Third, the behavior must be repeated in the presence of the same contextual cues to form a habit. Finally, the new action must be repeated in a fashion that is conducive to the development of automaticity, which includes creating salient cues for the activity, placing an emphasis on consistency, reducing behavioral complexity, and avoiding extrinsic rewards that have the potential to hinder the habit-formation process.

In 2020, 70% of the world’s population is projected to own a smartphone. At least 50% of those smartphone users are expected to use health-related applications [6]. Therefore, effective interventions can be conducted using mobile applications, which are already widely used to manage long-term health behavior. However, the functions of most health-related applications include providing information and tracking and recording activity status, which makes continuous use difficult. Furthermore, the effectiveness of these applications tends to be low, with many participants not using them after downloading them and commonly deleting them after one month [7]. The biggest limit to potential applications is the lack of proper regulation to ensure accuracy, quality, and performance [8]. As such, it is difficult for users to have continuous motivation. Additionally, habit formation must be motivated continuously by building an environment in which behaviors may form relationships with each other [9].

In Korea, the messenger application with the most domestic participants is KakaoTalk, with 48.2 million active participants in the first quarter of 2015 [10]. KakaoTalk uses the smartphone network to deliver real-time communication in one-on-one or group chats and can be linked to services such as Plus Friend [11]. Although—compared to previous health-related applications—messenger applications are limited in their functional aspects, they supplement the limitations of health-related applications through habituation, convenience, interaction, social presence among network members, and active emotional exchange. Therefore, they can effectively provide sustained health behavior interventions. A Korean study has found that a smart program using KakaoTalk for patients who underwent percutaneous coronary intervention was easy to use, cost-effective, and improved knowledge of the disease and the performance of health behaviors [12]. KakaoTalk has the advantage of allowing users to check whether messages are received or not, and it allows more personalized intervention than a telephone consultation [12]. Although most smartphones have default message apps (e.g., iMessage or Galaxy message app) and function similarly to KakaoTalk (e.g., sending emoticons and free massaging), few avenues provide smart chatting. However, KakaoTalk has a smart chatting function that allows developers to create projects directly using a chatbot hosting service. They can subsequently connect to the KakaoTalk channel.

Previous research has developed intervention methods by adapting habit formation models [13]. However, there have been few instances wherein mobile applications were used. Specifically, there has been a lack of research that uses smartphone messenger applications—including chatbots—to enable a continued relationship with, give motivation to, and provide professional advice to research participants. A chatbot is a conversational agent that uses natural language sentences to interact with users in certain domains or on certain topics [14]. Therefore, it is necessary to develop a mobile intervention using chatbot delivery that maintains a user’s interest in health and manages health behaviors from a long-term perspective. In this study, we developed a healthy lifestyle coaching chatbot for office workers using a habit formation model to be delivered using the KakaoTalk messenger application.

II. Methods

The dissemination of the Healthy Lifestyle Coaching Chatbot (HLCC) was guided by the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) instructional system design model. The first three steps were based on a study by Srikesavan et al. [15] (Figure 1).

Figure 1

Study flow diagram.

1. Step 1 (Needs Assessment: Practical Application Context)

Given how habit formation requires context-specific repetition, this study considered the participants’ available time, possible suggestions for healthy behaviors easily performed during everyday routines, and the optimal time to send cues to participants. Before developing the application and coaching program, we conducted an online survey to assess the daily routines of office workers. The initial online questionnaire was posted on an advertising sharing platform event page. Office workers could get information regarding the survey, which was held in the second week of December 2017. The advertising sharing platform event page also advertised the HLCC intervention. A total of 98 office workers responded to the standardized questionnaire. Based on their responses, we determined that their daily routines were divided into six periods: going to work (07:00–09:00), morning work (09:00–12:00), lunchtime (12:00–14:00), afternoon work (14:00–18:00), after work (18:00–21:00), and before sleep (21:00–01:00). The activities that were considered measurable and easy to execute in daily life were stair-climbing, walking, and standing while taking public transportation. Moreover, the average realistic goals for each respective activity at any one time were 6.52 ± 2.90 floors, 23.05 ± 10.12 min, and 42.00 ± 18.89 min (Table 1). Participants were also asked to set a behavioral goal termed the Daily Mission Challenge, which represented physical activity regularly performed in the same context to increase habit-related automaticity.

Table 1

Characteristics of daily routines and behavior for office workers (n = 98)

	Stair climb (60 kcal/5 stairs)	Walking (60 kcal/15 min)	Public-transportation standing (100 kcal/40 min)
Time
07:00–09:00	15 (34.1)	15 (36.6)	10 (87.5)
09:00–12:00	12 (27.3)	9 (22)	3 (12.5)
12:00–14:00	8 (18.2)	11 (26.8)	-
14:00–18:00	5 (11.4)	2 (4.9)	-
18:00–21:00	4 (9.1)	4 (9.8)	-
Goal per time (min)	6.52 ± 2.90	23.05 ± 10.12	42.00 ± 18.89

Values are presented as number (%) or mean ± standard deviation.

2. Step 2 (Design: System Design Requirements)

The theoretical basis for the system design was the habit formation model [16]. The specific context is summarized in Table 2, and actual examples of messages and a dialog are shown in Figure 2.

Table 2

Specifications of the system design used in this study

Stages of habit formation	Subcategories	Functions	Details	Interval
Translation of intention into action	Remembering intended action	Action and coping plan	Setting the behavioral goal	Before beginningAfter the first day trial
		Cues for the enactment of a plan	Reminder (push alarm)	Once a day
Repetition	Satisfaction regarding the experience	Positive experience of a new behavior	Extrinsic reward (coupon)	After completing the first dayEvery Sunday
		Attaining anticipated outcomes	Uploading picture evidence	Once a day
	Enhancing intrinsic motivation	Positive feedback	Intrinsic reward (compliment message)	Once a day
		Connection with others	Social reward (share picture evidence)	Once a day
	Self-regulatory strategies	Self-monitoring	Performance feedback with chatbot	Once a day
Automaticity	Consistency	Unchangeable rhythm of weekly exercises	Weekly graph of self-reported habit index	Every Sunday
	Reducing behavioral complexity	Weekly content	Providing healthy habit knowledge/tip	Four times per week

Figure 2

Screenshots of (A) push alarm reminders, (B) uploaded compliment message, (C) picture evidence sharing on the application homepage, and (D) performance feedback.

1) Translating intention into action

Habit formation requires one to remember to translate intention into action [5]; thus, reminders were an essential feature of our system design. The system automatically sent daily messages to participants, 10 minutes before the planned activity time.

Action and coping plans can also be particularly meaningful for habit formation [16]. In this study, we implemented this by enabling the participant to set realistic behavioral goals based on their daily routines and to evaluate the perceived differences between their current activities and desired behaviors. When the behavioral goal was impractical (for example, greater than two standard deviations from the realistic goal of each daily activity), the system would provide feedback encouraging the participant to modify their goal. The system also sent a message asking participants if they wished to modify their behavioral goals after the first day. Participants were allowed one chance to change the behavioral goal if it did not appear consistent with their daily routine.

2) Promotion of repetition

According to the habit formation model, repetition often requires continued rewards: intrinsic and extrinsic rewards. During the early stages of habit formation, positive experiences related to a new behavior can increase participant effort [17,18]. In this study, positive emotions were encouraged by sending extrinsic rewards—such as coupons—if the participants completed the first day of goal-directed action. If they repeated the accomplishment more than three times a week, a coupon was sent every Sunday.

3) Supporting the development of automaticity

To support consistency, the Self-Reported Habit Index (SRHI), which measures the self-reported perceptions of habit strength, was sent every Sunday. Depending on the results of the response, a weekly graph of SRHI was sent to the participants encouraging them to check the weekly changes.

3. Step 3 (Development: System Development)

1) Chatbot interface implementation

The interface of the HLCC is shown in Figure 3. In this study, we used KakaoTalk and Plus Friend, an advertising platform built into KakaoTalk with dedicated homepages and custom design tools aimed at e-commerce and business-targeted user engagement features. When participants clicked the “Add Friends” icon in KakaoTalk, they were automatically added as members, and their registered username was saved on the administrator page. Most contents and functions were executable via the Plus Friend administrator page. Performance feedback was independently developed using the IBM Watson conversation program and then linked to the administrator page to allow automatic response communication with registered participants in chatbot form. The HLCC was developed in Korean.

Figure 3

Interface of the Healthy Lifestyle Coaching Chatbot.

After entering the Plus Friend Administrator Center, an administrator account was created. The searchable username was set to HLCC so that the participants could easily find it.

2) IBM Watson conversation service implementation

First, the chat scenario was designed and implemented through the Watson conversation launch tool to enable the automatic chat function. By linking it to the Plus Friend Smart Chat application program interface, participants could chat with the chatbot through KakaoTalk.

Intent referred to the purpose of the participants’ statements. Therefore, all types of requests covered by the service were defined as intent. In our study, an example intent was “#Mission Start”. Various synonyms were set, including “Start” and “Hello”. If any of the synonyms were recognized, the chatbot would accept the request and automatically start the chat. If the entered request did not include a defined intent, the phrase “Hello, I am a Coaching Bot~ Please enter ‘Talk Talk’ or ‘Mission’” was presented.

An entity was a word or subject related to the intent that provided detailed context about the intent. In our study, an example entity was “@whether (Yes, No)”. If the entered content did not include any of the defined synonyms, the phrase “I only understand Yes or No” was presented.

Dialog referred to the flow of communication and determined the response after the application recognized the defined intent and entity. The dialog response flow chart of the chatbot was designed based on the scenario shown in Figure 4. To make the process more entertaining and to encourage a greater response from the participants, the system was designed to randomly print similar responses in different representation formats for certain questions. The dialog messages were developed following each component of the habit formation model. The participants interacted with the chatbot by conversing about their performance. If the planned behavior was completed, participants were provided with points per day. Participants were also asked to take pictures after they finished the activities and upload them to the chat room as evidence of completion. Afterward, the chatbot would automatically send a compliment message providing positive feedback.

Figure 4

Feedback algorithm of the Chatbot (R = response; Q = query).

4. Step 3 (Development: Usability Test of HLCC)

A usability test of the HLCC was performed before system implementation. All respondents were adult males and females who understood the purpose of the research and wanted to participate voluntarily; they gave written consent. The research plan was reviewed and approved by the Institutional Review Board of Seoul National University (No. 1706/003-026), and the clinical trial was registered with the Clinical Research Information Service (No. KCT0004009).

1) Participants and recruitment

The inclusion criteria were those aged 24 years and older, who could understand and respond to the survey content, and had experience using the KakaoTalk app. The selection criteria for survey respondents were based on convenience sampling. The recruitment details were posted on an online bulletin board of a certain IT company during the first and second weeks of January 2018. Twenty individuals were recruited. Once participants showed interest and agreed to join the study, they could connect to the KakaoTalk Plus Friend ID HLCC through a QR code and click the “Add Friends” icon to be automatically added as a member. An online consent form and survey link were then sent.

2) Usability test methods and procedures

The participants tested the HLCC for three weeks. The major usability issues were identified, and subsequent revisions were made. We checked whether the smart chatting system was running automatically based on the designed scenario. If the terms the participants answered were not in the synonyms saved by the system or if unintended answers appeared, then an error would occur. As for the usability test, we used the technology acceptance model [19], which comprise three factors, namely, perceived usefulness (4 items), perceived ease-of-use (5 items), and usage intention (4 items), which are rated using a 5-point Likert scale. Additionally, we investigated the extent to which the program content helped actual behavior performance using a 10-point scale and nine items. A higher score indicated greater satisfaction with a specific function.

3) Statistical analysis

The results of the usability test were analyzed using descriptive statistics, including frequencies and percentages for categorical data and means and standard deviations for continuous data. SPSS statistical software version 25.0 (IBM SPSS, Armonk, NY, USA) was used for statistical analysis.

III. Results

1. Designing and Developing the Chatbot and User Interface

The participants could enter the program through Kakao-Talk. In the administrator messaging system provided by Plus Friend, alarm times could be individually set. Thus, push alarms were automatically delivered to the participants at periods conducive to their daily routines (Figure 2A). The behavior plan was composed of five basic conditions: who, when, where, what, and how much. The push alarms were only sent on working days—Monday to Friday.

The pictures uploaded by participants as evidence of activity completion were posted on the homepage of the HLCC on KakaoTalk Plus Friend every day, allowing participants to share their performance with others and receive positive reinforcement (Figure 2B, 2C).

To encourage self-monitoring, the system was designed to provide a pop-up message at 21:00 asking the participants to tell the chatbot whether they had performed the planned action; afterward, the chatbot would provide feedback related to their answers (Figure 2D). If the mission was completed, participants were awarded 50 points per day. Based on the cumulative points, a coupon was then provided once a week.

At 16:00, Monday through Thursday, information on healthy habits was automatically sent (Figure 5). If personal questions or chatting was required, the text could be entered and sent after clicking “Chat with Researchers” in the one-on-one chat room screen. The researchers would then check the message and send feedback.

Figure 5

Examples of healthy habit knowledge sent to a participant at 16:00 Monday to Thursday.

2. Usability Testing of the HLCC

The preliminary version of the HLCC was produced, and usability testing was conducted to assess program modifications and revisions. The demographic characteristics of the participants who took part in the usability testing are presented in Table 3.

Table 3

Characteristics of participants taking part in the usability test

Characteristic	Value
Sex
Male	4 (20)
Female	16 (80)
Age (yr)
20–29	2 (10)
30–39	14 (70)
40–49	3 (15)
50–59	1 (5)
Weight control experience
Yes	12 (60)
No	8 (40)
Weight (kg)	61.7 ± 10.98
Sitting hours weekly	8.3 ± 2.23

Values are presented as number (%) or mean ± standard deviation.

The purpose of usability testing was to identify major usability issues and subsequent revisions. The test also checked whether the smart chatting system was running smoothly based on the designed scenario. When we first launched the prior version, the system did not work properly, and errors kept appearing. When an unintended answer appeared, an error would occur, and the dialog flow would be terminated. During the 3-week usability test, we summarized the expected answers to the questions to modify the dialog scenario. We also organized the responding words and grouped them into entity synonyms to reduce the rate of system errors. One of the examples collected during the usability test is summarized in Table 4.

Table 4

Synonyms of the entity

Entity	Synonyms
Yes	네, 응, 예스, yes, 그래, 웅, 어, 에, 좋아, 조아, 알았어, 오냐, 웅웅, oo, 예, 네네, 그려
No	아니, 노, no, 아뇨, 아닝, 노우, 노노, 싫어, 시러, 별로, ss

Additionally, we identified other usability features based on the satisfaction measurement results for the HLCC, wherein the extrinsic reward factor contributed the most to the performance of the activity, followed by reminders and intrinsic rewards (Table 5).

Table 5

Function and content satisfaction (n = 20)

Subcategory	Details	Score
Remembering intended action	Setting the behavioral goal	6.68 ± 2.53
	Reminder (push alarm)	8.42 ± 1.88
Satisfaction regarding the experience	Extrinsic reward (coupon)	9.25 ± 1.30
	Uploading picture evidence	7.77 ± 2.54
Enhancing intrinsic motivation	Intrinsic reward (compliment message)	8.32 ± 2.05
	Social reward (sharing picture evidence)	7.02 ± 2.77
Self-regulatory strategies	Performance feedback with chatbot	7.93 ± 2.61
Consistency	Weekly graph of SRHI	8.02 ± 2.13
Reducing behavioral complexity	Providing healthy habit knowledge/tip	7.71 ± 2.39

Values are presented as mean ± standard deviation using a 10-point scale.

SRHI, Self-Reported Habit Index.

We duly addressed this usability issue by revising the HLCC in the final version. The Likert scale scores indicated that all users found the program useful (3.98 ± 0.77) and easy to use (3.79 ± 0.84). The perceived usefulness of the HLCC was highest, followed by the usage intent and the perceived ease of use (Table 6).

Table 6

Characteristics of program usability indicators (n = 20)

Characteristic	Score
Perceived ease of use	3.79 ± 0.84
1. Managing my health via the coaching bot is economical.	3.82 ± 0.86
2. The coaching bot is easy to use and master.	3.88 ± 0.96
3. Using the coaching bot enabled me to respond appropriately to various health management circumstances.	3.71 ± 0.91
4. Using the coaching bot improved my ability to manage my own health.	3.69 ± 0.98
5. The health information I gained from the coaching bot affected my healthy lifestyle choices (diet, exercise, etc.).	3.85 ± 0.99
Perceived usefulness	3.98 ± 0.77
1. Health information gained from the coaching bot is useful and interesting.	4.11 ± 0.66
2. Information gained through the coaching bot is very effective for me.	3.86 ± 0.98
3. Information gained through the coaching bot improves my health management ability.	3.98 ± 0.87
Intention to use	3.86 ± 0.89
1. If the coaching bot continues to be available in the future, I will use the coaching bot while managing my health.	3.85 ± 0.92
2. If the coaching bot continues to be available in the future, I intend to use the coaching bot while managing my health when possible.	3.88 ± 0.89
3. If the coaching bot continues to be available in the future, I will speak positively of the coaching bot to those around me.	3.89 ± 0.94
4. If the coaching bot continues to be available in the future, I will recommend the use of the coaching bot to those around me.	3.83 ± 0.94

Values are presented as mean ± standard deviation using a 5-point Likert scale.

IV. Discussion

It is not easy for individuals to exercise while at work, and there are financial difficulties associated with companies providing exercise facilities and equipment for employees. The HLCC improved the effectiveness of habitually performing simple, repetitive exercises as part of daily life. For example, stair-climbing has a maximal impact over a short period, solving the problem of a lack of time for employees and a lack of space in the workplace. They can be performed without equipment and are very safe and economic. Walking-friendly transport methods, such as taking public transportation and walking to and from work, result in the same exercise effects as daily, regular, and consistent pedestrian activity. The proposed system enabled the habituation of these behaviors; therefore, it helped motivate people to habitually perform behaviors that can be easily accomplished in daily life.

With advances in digital technology, new mobile applications that help exercise management are being released, and wearable devices are becoming increasingly diverse. However, it is not easy to continuously use these tools. Continuous motivation is key for aspects that are difficult to manage by oneself [5,9]; thus, social network service (SNS) platforms, which enable a continuous relationship with participants and provide motivation and professional advice, have been garnering substantial interest. In our study, we designed a method of intervention delivery using an SNS platform. This delivery method can minimize individuals’ aversion to exercise and provide sustained motivation.

KakaoTalk boasts high familiarity, convenience, and interaction, with a strong social presence among network members and active emotional exchange [11]. The Plus Friend service provided by KakaoTalk is a pool-type text advertisement model. Many hospitals currently use Plus Friend for hospital marketing by communicating hospital news and various consultation services. Lee reported that a smart program using KakaoTalk for patients improved their knowledge of disease and the performance of health behaviors [12]; however, the researcher must respond to patients’ reactions individually, requiring personnel to operate the program. We used the Watson conversation launch tool to enable the automatic chat function, which not only reduced operating costs and efforts but also allowed real-time reactions to the participants.

The functions provided by the Plus Friend service proposed in this study included the delivery of custom push alarms, the provision of information relating to habit formation, one-on-one chats with researchers, the delivery of extrinsic and intrinsic rewards, and photo evidence uploading and sharing with other participants. However, a more vital motivating factor in increasing the frequency of research subject participation in habit formation was social rewards. Examples of social rewards included recognition or compliments and having others check and react to participants’ posts by sending texts or tweets or by uploading new pictures [20]. Although social rewards were omitted in our study because the Plus Friend service does not allow the delivery of social rewards, it is thought that incorporating social rewards into the research design may increase research subject participation and encourage more active use of the service in the future.

Recent technological advances have led to tremendous growth in the development and use of chatbots. “Chatbot” is a portmanteau of “chatting” and “robot”, and refers to a conversational messenger in which artificial intelligence talks to a human and answers questions using natural language based on big data analysis, similar to the conversation that would be produced if the human was chatting on an SNS [21]. The advantage of chatbots is that information can be checked and processed on the SNS platform screen without running a separate application. Furthermore, it also has the advantage of being applied to process repetitive jobs without being limited by time and space.

When intervention is delivered via Plus Friend, a single administrator cannot easily have a relationship with multiple participants and continue to provide motivation. To overcome this limitation, a chatbot is used to effectively answer various questions repeatedly asked by participants [22]. Although chatbots can minimize the intervention of the administrator, they cannot provide individualized feedback in more specialized circumstances, which is a disadvantage in terms of sustaining participation. Therefore, through this study, the effectiveness of the HLCC can be improved by collecting the content of the one-on-one chats with researchers. The data may then be used to determine the most frequently asked questions, to develop predicted conversation scenarios. Analyzing the responses and language used by participants may also be valuable in future chatbot development.

Sustainability is a key aspect when developing an intervention program to form healthy behaviors. The most important factor in developing a program is the design of rewards [18]. Rewards function as motivation for sustaining a new behavior; therefore, they are necessary in the repeated execution of behavior. When attempting to induce motivation for weight loss in employees, extrinsic rewards are the most effective [23]. However, previous research has suggested that providing simple financial rewards has limited success in sustaining motivational inducement; instead, it is more effective to improve the work environment and create a healthy atmosphere. Although extrinsic rewards elicit motivation at first, they eventually hinder a particular behavior becoming habitual [24].

According to previous research, the most effective types of reward for sustaining a habit is social rewards [23,25]. In other words, interactions with work peers, such as alerting work peers of their goals and receiving positive support messages, are important. Although we attempted to devise a method that could stimulate this interaction, this aspect had to be excluded from the program design due to the limitations of the service provided by the SNS platform used in this study. We suggest that future designs should recognize the limitations of extrinsic rewards and instead offer a variety of rewards, including intrinsic and social rewards that can serve as positive factors to promote the continuation of behaviors and increase the effectiveness of the program.

In this study, we developed the HLCC and assessed assess its usability. Further studies are needed to evaluate the effectiveness and long-term persistence of the program. Additionally, it is necessary to evaluate the appropriateness of the content and the usability of the program for various participants and age groups.