A look at engagement profiles and behavior change: A profile analysis examining engagement with the Aim2Be lifestyle behavior modification app for teens and their families.

Mobile-Health is increasingly used to deliver lifestyle modification interventions; however, little is known about how users engage with these apps. This study aims to profile how teens engage with Aim2Be- a lifestyles behavior modification app), characterize engagement profiles, and examine which engagement profiles support changes in behaviors (diet, physical activity, screen time and sleep) and changes in the mediators targeted by the app. Data were collected from 301 teens (14.8 years, 49% boys, 68% Caucasian) living in Canada, from March to October 2018, who utilized the Aim2Be app for 4.5 months. App-analytics tracked teen engagement with the app features (selecting aims, completing tasks and quick wins, using the knowledge center and social wall, and accessing the virtual coach). Factor mixture modeling identified the following engagement profiles: Uninvolved (32%) did not use most app features; Dabblers (25%) minimally used the app features; Engaged (24%) had moderate-to-high use of app features; and Keeners (19%) had the highest use of all app features. Regression models showed that teens were more engaged with Aim2Be if their parents were involved and if they participated with their mothers and/or an educated parent. Finally, Keeners significantly improved on most mediators of behavior change and increased their fruit and vegetable intake. The findings suggest that parental engagement supported teen engagement of the Aim2Be app and high engagement was needed to support behavior change among teens. Gaining a greater understanding of the features that appeal to teens is necessary to support behavior change.

Introduction

Background

In Canada, about one third of children aged 5–17 years are either overweight (19.8%) or obese (11.7%) (Roberts et al., 2012). As obesity tracks into adulthood (Ward et al., 2017) and children with obesity are more likely to develop noncommunicable diseases such as diabetes and cardiovascular diseases at a younger age (Reilly and Kelly, 2011); it is crucial to treat and prevent childhood obesity. Recent reviews reiterate that lifestyle behavior modification interventions that involve the family and incorporate dietary and exercise components along with behavioral therapy are effective in treating childhood obesity and improving the cardio-metabolic outcomes (Ho et al., 2012) in children of all ages (Ells et al., 2018).

While face-to-face interventions have been predominantly used for the prevention and treatment of childhood obesity amongst children (Mead et al., 2017, Al-Khudairy et al., 2017); mobile health (mHealth) interventions offer the opportunity to engage both children and their parents in these interventions. MHealth technologies offer a number of features for supporting behavior change including, among others, to being able to intervene during normal daily activities, to seamlessly monitor health behaviors, and to integrate self-regulatory strategies known to support behavior change (Kumar et al., 2013). As compared to traditional in-person interventions, MHealth interventions offers unique advantages as it is more easily accessible and reduce the cost associated with delivering and attending an in-person intervention and in addition the intervention can add synchronous data collection, user feedback, and self-monitoring of weight-related behaviors (Tate et al., 2013). In particular, mHealth provides an attractive way to support teens as they are frequent users of digital devices (53% of the Canadian children have a smartphone) (Brisson-Boivin, 2018).

Despite its potential low-cost reach to a majority of Canadian youth and its appealing nature, there exist several research gaps for MHealth research, including: 1) addressing and improving engagement with mHealth interventions – an issues that exists across all weight-related behavior domains (Kohl et al., 2013); 2) assessing user engagement of specific behavior change techniques which has been linked with the efficacy of mHealth interventions (Perski et al., 2017, Yardley et al., 2016). By nature, mHealth is ideally suited for understanding which components of the intervention the participants are using, as it is possible to track this information through app-analytic to capture how users engage with an app; and 3) elucidating which features incorporated in an app supports behavior change. Methodology to process the digital footprint or the app-analytics to identify profile of users exist (Serrano et al., 2017, Rabbi et al., 2018). For example, Serrano et al. (Serrano et al., 2017) used classification and regression tree analyses to explore which app features were related to engagement with a weight loss app. In a sample of adults, the study found that customization of diet and exercise regimens as well as tracking of weight resulted in greater engagement and suggested that tailoring was key for sustaining interest and engagement (Serrano et al., 2017). Currently, few mHealth studies have comprehensively used the app-analytic data to identify which features supports engagement and little is known about the features that support engagement of teens. As a result, little is known about which app features support behavior change as few studies examine how users engage with an app (Donkin et al., 2011).

This paper aimed to address these aforementioned research gaps, by evaluating version (v1) of the Aim2Be app – an mHealth gamified lifestyle behavior modification app for teens and their parents with the goal of improving children’s dietary quality (i.e. increasing fruit and vegetable intake, reducing sugar-sweetened beverage intake), increasing physical activity, reducing screen time, and meeting sleep recommendations (Mâsse et al., 2020). Specifically, this paper: 1) examined teens engagement with the main features of Aim2Be; 2) investigated whether teens’ profiles of engagement differed by socio-demographic characteristics and parental engagement; and 3) assessed whether engagement profiles were associated with change in the targeted mediators (knowledge, self-efficacy, and intrinsic motivation) and/or health behaviors primarily targeted by the app.

Methods

Study design

This study is a secondary data analysis of the pre and post data collected as part of the formative evaluation of Aim2Be version 1. The evaluation protocol was approved by the Children’s and Women’s Research Ethics Board at the University of British Columbia (H16-03090-A028) and the data was collected from March to October 2018.

Study participants

Participant recruitment was conducted using a pre-established web panel from Insights West (a Canadian market research firm), which recruits panelists through web advertisements supplemented by random digit dialing sampling. All potential panelists were asked to provide an e-mail address and fill out a consent form to be included in the web panel. After consenting, panelists completed a short socio-demographic questionnaire that was used to identify eligible participants for this study. Both teens and parents were compensated when they completed surveys. The web panel participants had previously agreed to be contacted for research studies and to allow their teens to participate in research. Parents were eligible if they were: the primary caregiver of a 13–17 year-old teen, literate in English, and had a mobile phone or computer with internet access at home. Teens were eligible if they were: between 13 and 17 years old, literate in English, could read at the grade five level or above, and had a mobile phone or computer with internet access at home. Families were ineligible if the teen had a diagnosis of type I diabetes, any physical or mental health condition that restricted the amount or type of activity they could do or the types and quantity of food they could eat, or any limitation that precluded them from being able to spend 20 to 30 min using a computer program written at a fifth grade reading level.

A total of 1644 parents were approached, 1418 parents were screened via self-reporting the aforementioned inclusion criteria, 873 were eligible and 632 parents expressed an interest to learn about this study. Of these, 426 parents consented and assented for their teens to be contacted for this study. In total, 301 teens enrolled and used the Aim2Be app for 4.5 months (demographic shown in Table 1) and participants completed questionnaires at baseline and at 4.5 months.

Table 1

Socio-demographic characteristics of the study sample (N = 301).

	%	Mean (SD)	Range
Teen characteristics
Age		14.83 (1.43)	13–17
Sex1
Male	49.2
Female	50.8
Ethnicity2
White/European	67.8
Others	32.2
Parent characteristics
Age3		46.61 (6.30)	31–66
Sex
Male	35.2
Female	64.8
Marital status
Married/common-law	84.4
Others	15.6
Education
College certificate or lower	48.8
Bachelor’s degree or higher	51.2
Family characteristics
Household income
<$100,000 Cdn	50.2
≥$100,000 Cdn	39.9
Prefer not to answer	10.0
Teen app engagement (minutes)		81.65 (155.15)	0–1335.85
Low (0–30 min)	46.84
Moderate (30–90 min)	30.90
High (>90 min)	22.26
Parental app engagement (minutes)		75.88 (106.46)	0–1252.30
Low (0–30 min)	35.3
Moderate (30–90 min)	32.2
High (>90 min)	32.6

SD: Standard deviation.

: Parent-reported sex on the child’s birth certificate.

: The “Others” category included: 15.9% multiple ethnicities, 11.6% Asians, 3.7% not specified, 1% others.

: Two parents did not provide age information missing age information.

Aim2Be app

Teens and parents downloaded their respective Aim2Be app on their phone and used the app for 4.5-month. After completing the baseline assessments, teens and parents were sent instructions on how to download the app on their phone. While Aim2Be can also be accessed via a home computer >95% of the participants downloaded the app on their phones. Briefly, the Aim2Be app is a gamified health behavior modification program that supports adopting healthy behaviors such as improving dietary quality, increasing physical activity, reducing screen time, and improving sleep (see theoretical description eslewhere (Mâsse et al., 2020). In addition, Aim2Be integrates a living green focus (i.e., addresses how health behaviors impact the environment) as well as emphasizes healthy body image and self-esteem. The behavior change techniques incorporated in Aim2Be are rooted into Social Cognitive Theory (Bandura, 2001) and Self-Determination Theory (Ryan and Deci, 2000). Specifically, Aim2Be has a strong focus on supporting change at both the individual and familial levels, emphasizes the development of self-regulatory skills, and supports both self-efficacy and intrinsic motivation. It integrates gamification elements as it recognizes the need for supporting enjoyment and motivation while supporting self-regulatory processes. The content of the app aligns with clinical guidelines, the curriculum of Canadian programs for the management of childhood obesity, and Canadian health recommendations for the behaviors targeted by the app (Tremblay et al., 2016).

The Aim2Be app is a self-guided exploration which is supported by a number of behavior change techniques (BCT) (Michie et al., 2013). After their interactive onboarding process, teens begin their journey by selecting the aims they wish to focus on from a list of 15 topics/aims (drop the pop, be a mindful eater, be a veggie fan, be sugar smart, be well rested, break your addiction, be social, be body positive, be brainy, step it up, use less plastic, cook and enjoy, stand up for others, be outdoorsy, dine out right, and power up your day). Teens can work on up to three aims at a given time and are provided with tasks to make progress on their selected aim. This process served to support teen’s self-regulatory skills by helping them monitor their behaviors, set incremental goals, and evaluate their progress. Teens progressed through the app by completing tasks and activities such as quick wins, quizzes, check-ins, or daily bonuses. Their journey is supported by a discovery knowledge center which they can access at any time or are directed to specific resources along their journey. Along the way, teens accumulate currencies used to unlock app features (i.e., collections for their avatar and access to interactive stories). The app included a moderated social wall where teens can interact with other teens or with a Live Coach. The parent app is a companion app to the teen and included aspect of the teen app such as the discovery knowledge center and the moderated social wall. The parent app aimed to support changes at the familial level and it includes content that was of specific interest to the parents (e.g., modeling healthy behaviors, creating a supporting environment, involving the family into healthy behaviors) (for further description see (Mâsse et al., 2020). The parent companion was developed as we know that parents play a key role in supporting teens’ behavior change through modeling, restructuring, and supporting the teen in their journey to change their health behaviors (Ells et al., 2018).

Study measures

App-analytic data (engagement with app features) tracked all the features teens used and engaged with and the analyses focused on the main features of the app: 1) selecting aims (BCT = goal setting); 2) completing tasks (BCT = action planning, behavioral practice, and habit formation); 3) completing quick wins (BCT = behavioral practice); 4) discovering the knowledge center, i.e., reading an article (BCT = information about social and environmental consequences and prompting action); 5) using social wall including viewing, posting, or commenting (BCT = social support and demonstration of behaviors); and 6) connecting/talking with virtual coach (BCT = social support). The raw app-analytic data was processed to capture the frequency (i.e., number of times) with which the teens used each individual app feature, e.g., the number of times teens selected an aim or the number of conversation sessions with the live coach. As use of these app features were not normally distributed, the data was re-categorized into three groups: 1 = no use, 2 = low use, and 3 = high use where the median for each feature was used to split the data into low versus high users.

Sociodemographic characteristics. Baseline questionnaire was used to collect teens’ and parents’ demographic characteristics, including age, sex, ethnicity, marital status, education and household income. We also include parents’ app engagement, measured as time spent using the app, as a covariate in the analyses.

Mediators. At its core, Aim2Be assumed that engagement with the app would increase teens’ knowledge of Canadian Health Recommendations (a pre-requisite for behavior change), self-efficacy to change their behaviors, and support autonomous motivation to activate their self-regulatory skills. Health knowledge was assessed with 13 multiple choice and true–false questions: three questions assessed physical activity knowledge (i.e., amount of moderate to vigorous physical activities, amount of strengthening activities, and steps per day), eight questions assessed knowledge of food recommendations and general nutrition knowledge (asking about servings of fruits and vegetables, healthiest drinks and meal options, added sugar, healthy fats), one question asked about screen time recommendations, and one question asked about sleep recommendation. All knowledge questions were coded as 0/1 (incorrect/correct) and were averaged to compute a total score. Self-efficacy was assessed with four 5-point Likert scale items modeled from the Perceived Competence Scales (Harter, 1982); one question assessed teens’ confidence in meeting the 60 min of physical activity per day, two questions assessed confidence in consuming recommended amount of fruits and vegetables every day and in eliminating sugar-sweetened beverages, and one question measured confidence in limiting screen time. Autonomous motivation was measured using eight 5-point Likert scale items from the Family Life, Activity, Sun, Health, and Eating (FLASHE) study (Nebeling et al., 2017). These items were modelled after the Self-Regulation Questionnaires (Center for Self-Determination Theory, 2020, Levesque et al., 2007) and two items measured integrated regulation and intrinsic motivation for each behavior (physical activity, fruit and vegetable intake, consumption of sugar-sweetened beverages, and screen time). These mediators were measured at both baseline and at 4.5 months.

Health behaviors. Physical activity was measured using the list of activities from the Physical Activity Questionnaire for Older Children (PAQ-A − 0.75–0.82 test retest and 0.52 correlation with oxygen uptake) (Kowalski et al., 2004) and modifying the instrument to administer it online and asking minutes spent active on each day of the previous week. Teens recalled the amount of physical activity they engaged in over the last seven days, with responses between “None” to “>2 h”, then were asked to indicate which activities they completed from a pre-determined list. Fruit and vegetable and sugar-sweetened beverage intake was measured with the National Youth Physical Activity and Nutrition Study (NYPANS) items. The sugar-sweetened beverage intake items excluded intake of 100% fruit and/or vegetable juices and fried potatoes (previous validation against 24-hr dietary recall range between 0.26 and 0.49 for sugar-sweetened beverage intake, but fruit and vegetable intake estimation tends to be higher) (O'Malley Olsen et al., 2014, Eaton et al., 2013). Screen time was assessed with Rosenberg et al. (Rosenberg et al., 2010) Sedentary Behavior Questionnaire –shown to be sensitive to change among children (Wang and Brownell, 2011). The eight items assessing screen behaviors on most recent weekday and weekend day: watching television; playing computer or video games; using a computer, tablet, or mobile device outside of schoolwork; and talking or texting on a cell phone. Sleep was assessed using self-reported bedtime and wake-up time and sleep duration was coded as meeting (yes = 1) or not meeting (no = 0) the sleep recommendation (9–11 h for 5–13 years old and 8–10 h for 14–17 years old) (Tremblay et al., 2016). These health behaviors were measured at both baseline and at 4.5 months.

Data analysis

A factor mixture analysis served to identify profiles of engagement as it considers the interdependency among the app features. This analysis starts with a conventional exploratory factor analysis (EFA) and the number of factors are then accounted in the subsequent latent classes analysis (Muthén, 2008). The number of profiles retained was informed by evaluating the: 1) Bayesian Information Criterion (BIC), where a lower and large drop in BIC indicates a better fit; 2) Lo-Mendell-Rubin adjusted Likelihood Ratio Test (LRT), where a non-significant test is indicative of a good fit; 3) Bootstrapped LRT (BLRT), where a significant test suggests that adding a class is meaningful; and 4) smallest class which accounts for at least 10% of the total sample. These analyses were conducted in Mplus version 8 (Muthén, xxxx).

Chi-square tests were used to determine whether teens’ engagement profiles differed by the socio-demographic characteristics. To assess whether engagement profiles predicted change in the mediators and health behaviors targeted by the app, multivariate linear and logistic regressions were used, controlling for baseline values and meaningful socio-demographic characteristics. All analyses were conducted in Stata 15 (StataCorp, 2017). Significance was set to p < 0.05.

Results

Teens’ engagement profiles

Table 2 summarizes the EFA and the factor mixture analysis. Based on EFA results in Step 1, a two-factor model was selected as the best model because it had the lowest BIC and the chi-square test of differences among a 1-, 2-, and 3-factor model suggested that a 2-factor EFA explained the correlations in the data. This information was accounted for in the factor mixture model and the results suggested that a 4-class solution best explained the variance in the data – while the LMR test remained significant it was close to being non-significant and BIC minimally decreased between the 4- and 5-class models, suggesting that the 4-class model was a better fit.

Table 2

Summary of the factor-mixture analysis to identify the number of classes that explained teen’s engagement with Aim2Be app features.

Model	Log-likelihood	Parameters	BIC	LMR (p-value)	BLR (p-value)
STEP 1: Exploratory Factor Analysis
One-factor	−1356.98	18	2816.70
Two-factors (Best model)a	−1336.49	23	2804.24
Three-factor	−1327.42	27	2808.92
STEP 2: Factor Mixture Analysis
2-class, 2-factor	−1483.72	19	3075.88	0.00	0.00
3-class, 2-factor	−1356.83	22	2839.22	0.00	0.00
4-class, 2-factor (Best model)b	−1344.63	25	2831.94	0.02	0.00
5-class, 2-factor	−1334.68	28	2829.17	0.53	0.00

BIC: Bayesian Information Criteria.

LMR: Lo-Mendell-Rubin adjusted LRT test.

BLRT: Bootstrapped Likelihood Ratio Test.

The chi-square (χ2) test of difference between the two-factor and one-factor solution was significant χ2 (5) = 20.49, p < 0.05 whereas the difference between the three-factor and two-factor solution was not significant χ2 (4) = 9.07, p > 0.05. This suggests that a two-factor solution best explained the correlations; where Factor 1 included behavioral features (aims, tasks, quick wins, and knowledge center) and Factor 2 included social support features (social wall and virtual coach). Factors 1 and 2 were highly correlated (0.79).

Although the BLR is significant between 5-class and 4-class solution, we chose the 4-class solution as the best solution because: (1) the 5-class solution results in a group <10% of the sample, and that (2) the decrease of BIC from 4-class to 5-class solution is very minimal.

Fig. 1 shows the 4-class solution depicting engagement with the app features and Fig. 2 shows overall engagement for each profile. Class 1 (31.9%) labelled as the Uninvolved regroups teens who do not use most app features. Class 2 (25.3%) labelled as the Dabblers regroups teens who minimally used all app features, with the exception of the discovering knowledge center feature which they did not use. Class 3 (23.9%) labelled as the Engaged regroups teens who had high use of the app features. Finally, class 4 (18.9%) labelled as the Keeners is similar to the Engaged profile but teens in this class had much higher level of engagement with all app features.

Fig. 1

Teen four profiles of use of the app features.

Fig. 2

Teens app engagement by week of intervention.

Characterizing engagement profiles

Table 3 summarizes the socio-demographic characteristics comparisons among the four app engagement profiles. There was no statistical difference amongst engagement profiles in terms of children’s age, sex, ethnicity, parents’ marital status, and family household income. However, parents’ sex, education level, and engagement profiles were notably different. The percentage of mothers was significantly higher amongst the Keeners (82.5%) as compared to the Uninvolved (56.3%) (χ2 = 11.855, p = 0.008). The Engaged profile included significantly more university-educated parents than the Dabblers (χ2 = 7.886, p = 0.048). Finally, the Keeners included significantly more parents who were highly engaged with the app than the Dabblers (χ2 = 20.405, p = 0.002).

Table 3

Socio-demographic profiles for each teen profile / class of use.

	Class 1: UninvolvedN = 96	Class 2: DabblersN = 76	Class 3: EngagedN = 72	Class 4: KeenersN = 57	TotalN = 301	Chi-square test
						χ2	p
Child characteristics
Age
13–14	46.9%	40.8%	43.1%	42.1%	43.5%	0.723	0.868
15–17	53.1%	59.2%	56.9%	57.9%	56.5%
Sex
Male	58.3%	46.1%	47.2%	40.4%	49.2%	5.404	0.145
Female	41.7%	54.0%	52.8%	59.7%	50.8%
Ethnicity
White/European	69.8%	60.5%	72.2%	68.4%	67.8%	2.670	0.445
Others	30.2%	39.5%	27.8%	31.6%	32.2%
Parent characteristics
Age
31–46	44.8%	56.6%	50.0%	47.4%	49.5%	2.486	0.478
47–66	55.2%	43.4%	50.0%	52.6%	50.5%
Sex
Male	43.8%	32.9%	40.3%	17.5%	35.2%	11.855	0.008
Female	56.3%	67.1%	59.7%	82.5%	64.8%
Marital status
Married/common-law	82.3%	86.8%	87.5%	80.7%	84.4%	1.785	0.618
Others	17.7%	13.2%	12.5%	19.3%	15.6%
Education
College certificate or lower	44.8%	61.8%	40.3%	49.1%	48.8%	7.886	0.048
Bachelor’s degree or higher	55.2%	38.2%	59.7%	50.9%	51.2%
Family characteristics
Household income
<$100,000 Cdn	53.1%	48.7%	47.2%	50.9%	50.2%	3.697	0.718
≥$100,000 Cdn	38.5%	44.7%	40.3%	35.1%	39.9%
Prefer not to answer	8.3%	6.6%	12.5%	14.0%	10.0%
Parental app engagement for the duration of the intervention
Low (0–30 min)	40.6%	44.7%	25.0%	26.3%	35.2%	20.405	0.002
Moderate (30–90 min)	28.1%	31.6%	45.8%	22.8%	32.2%
High (>90 min)	31.3%	23.7%	29.2%	50.9%	32.6%

Table 4 summarizes the analyses that tested whether change in the mediators and behaviors targeted by the app differed by engagement profiles at follow-up, while controlling for the baseline values of the mediators or behaviors and significant parental socio-demographics. Of note, at baseline, only health knowledge of the sleep recommendations differed by teen’s engagement profile (p = 0.046) – the Dabblers were significantly less knowledgeable at baseline than the Uninvolved (β = −0.167, p = 0.019) about the sleep recommendations. Overall, significant differences were observed between the Keeners as compared to the Uninvolved. Specifically, in comparison to the Uninvolved, the Keeners had positive changes in nutrition health knowledge (β = 0.181, p = 0.006), intrinsic motivation in heathy eating (β = 0.194, p = 0.004), and physical activity (β = 0.174, p = 0.008), as well as self-efficacy in healthy eating (β = 0.175, p = 0.005), unhealthy eating (β = 0.131, p = 0.031), and sedentary behaviors (β = 0.133, p = 0.031). In terms of behavior change, intake of fruits and vegetables (reported yesterday) significantly increase at the 4.5-month follow-up for the Keeners (β = 0.111, p = 0.046), Engaged (β = 0.182, p = 0.001), and the Dabblers (β = 0.115, p = 0.040) in comparison to the Uninvolved teens.

Table 4

Changes in health knowledge, motivation, self-efficacy and behaviors at 4.5 months follow-up.

Reference group = Class 1: Uninvolved	N	Class 2:Dabblers		Class 3:Engaged		Class 4:Keeners		p-value	Adjusted Incremental R2
		β	p	β	p	β	p
Health knowledge
Physical activity	269	0.009	0.882	−0.003	0.962	−0.015	0.807	0.987	–
Nutrition	255	0.118	0.072	0.010	0.128	0.181	0.006	0.039	0.012
Screen time	269	0.011	0.848	0.011	0.855	0.133	0.028	0.108	–
Sleep	270	−0.032	0.612	−0.008	0.893	0.017	0.788	0.904	–
Intrinsic motivation
Healthy eating	270	0.019	0.776	0.057	0.401	0.194	0.004	0.024	0.021
Unhealthy eating	270	0.105	0.115	−0.017	0.803	0.114	0.088	0.107	–
Physical activity	270	0.066	0.306	0.007	0.910	0.174	0.008	0.033	0.017
Sedentary behaviors	270	−0.018	0.782	0.013	0.842	0.099	0.131	0.335	–
Self-efficacy
Healthy eating	269	0.017	0.788	0.067	0.284	0.175	0.005	0.029	0.016
Unhealthy eating	270	−0.057	0.346	0.004	0.950	0.131	0.031	0.028	0.016
Physical activity	269	0.037	0.584	0.013	0.841	0.013	0.845	0.958	–
Sedentary behaviors	268	−0.012	0.838	0.040	0.518	0.133	0.031	0.099	–
Health behaviors
Fruit and vegetable intake (yesterday)	270	0.115	0.040	0.182	0.001	0.111	0.046	0.009	0.019
Fruit and vegetable intake (last week)	264	−0.027	0.691	0.029	0.672	0.108	0.113	0.261	–
100% fruit juice (last week)	264	−0.078	0.182	−0.096	0.099	−0.066	0.263	0.336	–
Sugar-sweetened beverages (last week)	270	0.010	0.884	0.008	0.901	0.056	0.403	0.846	–
Moderate-vigorous PA (last week)	245	−0.070	0.283	−0.031	0.632	−0.001	0.991	0.682	–
Screen time (last week)	273	−0.012	0.838	−0.051	0.390	−0.044	0.466	0.791	–
Meeting sleep guidelines (last week)*
Not meeting to meeting guidelines	273	1.658	0.436	1.199	0.786	2.004	0.330	0.748	–
Meeting to not meeting guidelines	273	1.346	0.510	1.654	0.247	0.863	0.771

Note: all βs were standardized and all models were controlled for baseline mediators/behaviors, parent’s gender, education and app engagement.

*Relative risk ratios (RRR) were presented for the sleep outcomes

– Incremental R2 shown only for significant models.

Discussion

The factor mixture analysis identified four engagement profiles that teens belong to: namely the Uninvolved, Dabblers, Engaged, and Keeners. As expected, the Uninvolved, who made little use of active app components had little change in the outcomes at 4.5 months. Amongst all engagement profiles, the Keeners had the most changes as they changed one of their health-related outcomes (increased fruit and vegetable intake) and had significant improvements in the targeted mediators (health knowledge, self-efficacy, and intrinsic motivation). The Keeners had the highest engagement, improved the most at 4.5 months, and had parents that were also engaged with the app. These results highlight that even in the mHealth context, it is important to integrate parents as teens have limited abilities to make change at the familial level. This is not surprising as integration of the family is recommended to support behavior change at the child level (Lau et al., 2007). This finding suggests that it may be worthwhile to investigate whether making participation a team effort would better support engagement of teens in the mHealth context.

A key strength of this study is the significant inclusion of fathers. However, in terms of the role that fathers played, results indicated that the Keeners who had a higher engagement of mothers had more overall changes at 4.5 months than the Uninvolved. In a recent scoping review, fathers are found to be involved in food parenting practices but not to the same extent as mothers and less so with day-to-day responsibilities (Davison et al., 2020). The initial version of the Aim2Be app had a strong focus on supporting change in nutritional behavior, and it may be that this content is more relatable to mothers. In addition, as mothers typically assume the food parenting responsibilities (Davison et al., 2020); mothers may have more familiarity in how to support their teens, which could explain why the Keeners, who were more supported by mothers, were the most engaged.

The Dabblers were an interesting group as they showed an overall interest in the app, but unlike the Engaged or the Keeners their engagement never progressed to high use. The Dabblers had the least engaged parents and the lowest proportion of highly educated parents. Dabblers showed a relatively low involvement in the app, which could be explained by the lack of parental support especially considering that the Dabblers never discovered the knowledge center feature. While it is tempting to think that parental lack of awareness or skills might explain why the Dabblers received less support from their parents, research has shown that education or lack of education leads to different economic and familial realities (less job stability and shift work which compete with familial roles), which may explain why parents were less involved (Zajacova and Lawrence, 2018). From a health disparity perspective, the Dabblers are likely a priority group to target in future studies as they showed an initial interest in the app but their parents were not able to support them. Future studies can perhaps identify teens who are at risk of being a Dabbler by collecting demographic information (i.e., educational information) as well as collecting app-analytic to identify parents who, early on, are low users of the app.

Interestingly, the Dabblers, Engaged, and Keeners all significantly improved their fruit and vegetable intake as compared to the Uninvolved. Importantly, the Keeners were the only ones who saw significant improvements in the targeted mediators (i.e., increase in knowledge, self-efficacy, and intrinsic motivation), highlighting that the app likely worked for a subgroup of users (i.e., the Keeners) for some the health behaviors targeted by the app. Previous studies have found that apps can be effective at changing health behaviors (Ho et al., 2018) but there is a limited number of studies among teens. In Aim2Be, the aims, tasks, and knowledge center features were hypothesized to support self-regulatory processes in teens (Mâsse et al., 2020); however, as the impact on teen’s health behaviors was limited it might be important to examine whether these components are working as hypothesized. Furtermore, it is unclear why improvement in fruit and vegetable intake was also observed amongst the Dabblers and Engaged without improvements in the targeted mediators. It is possible that such observation is driven by other mediators not considered or measured in the current study. The findings amongst the Dabblers and Engaged also provide insights for future studies to understand what features promote “effective engagement,” rather than simply assuming that more engagement is better (Yardley et al., 2016).

This study is not without its limitations. First, the factor mixture modeling approach categorized use of the app features (no, low and high use) based on the distribution of use which may not necessary equate to high involvement or minutes spent using the app. Second, parent engagement, which was assessed as time spent using the app, may not capture the other ways in which parents can engaged with and support their teens (e.g., inquiring about their use of the app, discussing what they learned in the Aim2Be app, or participating in healthy behaviors as a family). Therefore, profiling how parents support their teens’ journey outside of the app may shed more light on these results. Third, app-analytics only captured the total frequency that participants used the app and as such it may not capture the time that participants worked on the outcomes of interests outside of the app. Fourth, as participants were recruited from a web-based panel, the results may not generalize to disadvantaged and/or marginalized population who do not have access to technologies. Finally, the sample for this study while relatively small was representative of the Canadian population, and as such the results may not generalize to other countries or populations.

Conclusion

Using factor mixture modelling approach, this study provided insights as to how teens engaged with the six main features incorporated into the Aim2Be app and as such provide insightful information to develop just-in-time adaptive interventions which aims to provide the features that support behavior change (Spruijt-Metz et al., 2015). This study found that broad and high engagement with the main features of the app resulted in more changes in the outcomes targeted by Aim2Be. Noteworthy was the role of parents to support the engagement in the Aim2Be app. Ensuring parental support in the mHealth context is key to support teens’ continued engagement leading to their changes in health behaviors. Future research is warranted to understand how mHealth interventions can be more appealing to certain profiles of users to elucidate which specific intervention components are most effective and for whom.

Disclosure statement

YL has no competing interest to declare. LCM received funding to conduct the evaluation of the Aim2Be which is developed by the Childhood Obesity Foundation with funds as described above. LCM does not have any ownership in the Aim2Be app.

Ethical statement

The Children’s and Women Research Ethics Board at the University of British Columbia approved the protocol and consent form for this study (#H16-03090) which align with institutional and national policies (i.e., bound to Tri-Council Policy Statement: Ethical Conduct for Research Involving humans in accordance with the recent Helsinki’s Declaration). All participants consented to be part of this study and have agreed to have the results published in aggregate form.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: The Childhood Obesity Foundation received funds from the Public Health Agency of Canada (Project number 1617-HQ-000046, contact number: 604-251-2229) and matched financial and in-kind funds from Ayogo Health Inc., Merck Canada Inc., Heart and Stroke, Obesity Canada, Diabetes Canada, Craving Change, Canadian Society for Exercise Physiology, David Suzuki Foundation and Pacific Blue Cross Foundation. YL has no competing interest to declare. LCM received funding to conduct the evaluation of the Aim2Be. LCM does not have any ownership in the Aim2Be app and own the rights to publish study results.