Sustaining eHealth engagement in a multi-year weight gain prevention intervention.

INTRODUCTION: Digital tools are widely used and effective in weight management interventions; however, usage declines over time. Strategies to promote continued engagement should be explored. We examined the effects of offering additional modes of weight reporting as well as periodic online campaigns to promote engagement, assessed by frequency of weight reporting, in a weight gain prevention study for young adults.
METHODS: Using an observational design, self-reported weights obtained through digital tools were pooled across participants assigned to two interventions (n = 312). Analysis examined the effects before during and after introduction of an additional reporting modality (email) and for three time-limited refresher campaigns over 2 years.
RESULTS: Adding a new modality to the three existing modes (SMS, web, and mobile web) increased weight reporting as well as the number of modalities participants used to report weights. The use of several modes of reporting was associated with more weights submitted (p < 0.01). Refresher campaigns did not increase the proportion of participants reporting; however, the number of weights submitted during the 4-week campaigns increased compared with the 4 weeks before the campaign (p's ≥ 0.45, <0.001, respectively).
CONCLUSION: Using multiple digital modalities and periodic campaigns shows promise for sustaining engagement with weight reporting in a young adult population, and incorporating such strategies may mitigate typical declines in eHealth and mHealth interventions.

Introduction

eHealth and mHealth interventions are increasingly convenient and popular for delivery of weight management interventions. The use of these interventions can have positive effects on weight 1, 2, 3, and studies have shown that the usage of the interventions as measured by logins or submission of self‐monitoring information is associated with more positive outcomes 4, 5. Across studies, the usage of, or engagement with, eHealth tools tends to decline over time, both in the amount of average usage (e.g. mean logins per week) and in the proportion of users who log in during a given interval 6, 7, 8, 9.

Increasingly, digital interventions use multiple platforms to engage users (website, apps, social media, SMS, etc.) not only across studies but also within studies to offer users a choice in how to engage with the tools 10, 11, 12. For example, in eHealth behavioral weight loss interventions where participants are typically asked to self‐report weight and/or behavior at regular intervals, participants may be offered a variety of ways to electronically submit self‐monitoring data (e.g. web or SMS, web or IVR). However, few studies have reported on methods to promote adherence to self‐monitoring in digital interventions or studied their effects. A recent study found that adding two phone calls during the first month of a web‐based weight loss program increased the likelihood that participants' completed online activities compared with web only 13. Another study offering choice of modality for self‐monitoring found that more participants chose to monitor with interactive voice response than via web; however, by 24 months, fewer than 40% of participants were monitoring at expected levels with either modality 11. Because of the dearth of literature on engagement, scholars have called for the development of a science of engagement with studies considering different ways to promote engagement over time 14.

Study of Novel Approaches to Prevention (SNAP), a randomized controlled trial for weight gain prevention in young adults, allows for examination of engagement in a long‐term behavioral intervention that used a variety of eHealth tools for weight reporting. SNAP (n = 599) compared two technology supported self‐regulation interventions to a self‐guided control group over an average follow‐up of 3 years among young adults ages 18–35 years 10, 15. Both of the SNAP interventions used a core self‐regulation approach grounded in daily self‐weighing, instructed participants to compare weight with a goal and provided corrective feedback. The Small Changes approach (n = 200) recommended daily small changes to diet or activity to produce a 100–200 calorie deficit each day. The Large Changes approach (n = 197) encouraged a short‐term, 500–1,000 calorie deficit each day, to produce an initial 5–10 lb weight loss to buffer against future gains. Both interventions were better than the control for preventing weight gain, with Large Changes appearing more efficacious than Small Changes 15.

SNAP provides a unique opportunity to examine long‐term engagement with technology tools for self‐monitoring. When the study began, participants were offered three modalities to submit their weight data (web, mobile web or text message reporting); later, participants were offered the opportunity to submit weight data via an auto‐link within a reminder email. Thus, the study allows for examination of the following questions: does offering participants additional methods to report weight increase the proportion of the sample reporting weight and/or the frequency with which they report their weights? Also, which modalities are used most frequently for weight reporting?

SNAP also allows for examination of the effect of offering periodic ‘refresher campaigns’ on program engagement. Refresher campaigns, which are time‐limited (4–8 weeks) programs, are being used increasingly in long‐term behavioral weight loss programs, most notably in the Diabetes Prevention Program and Look Ahead 16, 17, to help re‐engage participants in the behavioral intervention. In SNAP, two 4‐week campaigns were offered per year, thus allowing us to examine whether the introduction of a campaign increases key engagement metrics (i.e. the proportion of participants who submit their weight and the average number of weights reported by participants during the campaign) compared with the months before and after. The current study offers insight for digital health interventions and for long‐term behavioral weight loss program that might use periodic campaigns to boost program participation. To our knowledge, there have been no studies to evaluate whether these campaigns are effective in boosting program engagement.

Materials and methods

Interventions

The interventions were delivered with 10 face‐to‐face group sessions over 4 months, followed by monthly contact delivered primarily via web, email or text message (SMS) throughout the rest of the study. After the initial 4‐month period, participants were encouraged to weigh regularly, preferably daily, and to use their weight for self‐regulation. The study encouraged participants to continue weighing frequently as an important part of long‐term weight gain prevention. However, given that the intervention was 3 years in duration, the intervention was designed to balance reporting needs with recommendations for the behaviour. In order for a monthly feedback message that was matched to the participants' weight status to be sent via email, a minimum of monthly reporting was expected. In addition to the monthly email feedback, token reinforcers were sent monthly via postal mail if a participant's weight was below the baseline at month's end. Throughout the extended intervention period, the monthly email offered optional limited telephone or email counselling if a participant had gained above their baseline weight. See Figure 1 for more information on intervention activities.

Figure 1

Intervention activities.

During the initial part of the trial (from November 2010 to April 2012), participants were offered three methods for reporting their weight data: web, mobile web or via SMS. In response to participant requests, a fourth option for weight submission was offered beginning in April 2012, allowing participants to report weight via direct link and log in from monthly reminder emails. Clicking this link logged participants into the study website directly using a saved password. All methods fed the weight into the database, and method of reporting was recorded.

Because SNAP was designed to be a long‐term intervention, after the initial 4‐month intervention, optional biannual (2× per year) refresher campaigns, each lasting 4 weeks, were used as opportunities for additional intervention and re‐engagement. Refresher campaigns were conducted via electronic communication channels (e.g. email, interactive Qualtrics surveys and text messages). Each focused on themed content grounded in self‐regulation theory concepts with targeted messages and approaches consistent with the randomized weight gain prevention approach (e.g. Small or Large Changes). Refresher campaigns often involved small monetary incentives (e.g. $5) or weight‐related merchandise (e.g. exercise DVD). See Table 1 for more details on the refresher campaigns used during the timeframe of this analysis.

Table 1

SNAP refresher campaign description

Refresher number	Time period	Format(s)	Message frequency (per week)	Topic and details
3	Fall 2012	Email	2	Topic: Weekly behavior change challenges Participants who decided to enrol in the refresher received weekly emails encouraging them to practice one intervention specific goal per week. Goals included daily self‐monitoring, choosing healthier (Small Changes) or lower calorie (Large Changes) snacks and food items and increasing daily steps (Small Changes) or exercise intensity (Large Changes). Participants in the Yellow or Red Zones were encouraged to complete goals on a daily basis, whereas participants in the Green Zone were encouraged to meet goals 3–4 d per week. At the end of each week, participants were sent an email request for reporting weight and compliance with weekly goals. Participants could earn $5 each week for submitting a challenge report.
4	Spring 2013	Email	2	Topic: Spring transformation Participants who enrolled in Small Changes were encouraged to make one healthy habit and break one healthy habit for the refresher series. Participants were given a list of habits from which they selected the most applicable habit change. Examples of healthy habits to make included drinking a glass of water before each meal, practicing portion control at lunch or dinner and eating slower. Examples of unhealthy habits to break included sitting for prolonged periods of time, eating dessert every night and unplanned snacking. Participants were provided tangible cues (e.g. re‐usable tumbler for increasing drinking water; standing balance disk for reducing sitting time), valued at $10–15 each, to support habit changes. Participants who enrolled in the Large Changes group were given a choice of three exercise DVD's (light, moderate and intense valued at $10). Participants were encouraged to use this DVD to add to their current exercise routine and to meet their exercise goal (250 min per week). Participants were also encouraged to self‐monitor their daily calorie intake. At the end of each week, participants were sent an email request for reporting weight and compliance with weekly goals.
5	Fall 2013	Email	2	Topic: Diet change challenges Participants who enrolled were given four different diet changes to try each week. The weekly email also included research on the topic. The weekly challenges included: changes to breakfast, hunger and appetite awareness (hunger scale), Mediterranean diet and weekend versus weekday eating. The challenges were set to be appropriate for Small versus Large behavior changes using the same challenge theme each week. Participants were informed that submitting a report on the challenge each week could earn anywhere from $1 to $15, and the amount would be randomly selected each week.

Measures

Data for this analysis include participants in the two active treatment arms of the SNAP study (N = 397). See Table 2 for the demographic information on the SNAP sample overall and for the two intervention groups examined in this analysis. Participants in this subsample were healthy young adults, who were normal or overweight and were able to read and speak English. Participants in the two intervention groups were on average 27.7 years (standard deviation [SD] = 4.5) old with average body mass index of 25.4 (2.5) kg m−2 with 45.3% normal weight and 54.7% overweight at baseline. All participants signed consent forms that had been approved by the institutional review boards at the University of North Carolina at Chapel Hill, The Miriam Hospital and Wake Forest School of Medicine.

Table 2

Baseline demographic characteristics: overall and by arm (N[%])

		All randomized	Sample for this analysis
		N = 599	Self‐guided N = 202	Small Changes N = 200	Large Changes N = 197
Age	18–24	169 (28.2)	53 (26.2)	60 (30.0)	56 (28.4)
	25–35	430 (71.8)	149 (73.8)	140 (70.0)	141 (71.6)
Sex	Male	130 (21.7)	44 (21.8)	43 (21.5)	43 (21.8)
	Female	469 (78.3)	158 (78.2)	157 (78.5)	154 (78.2)
Race	African American	66 (11.0)	19 (9.4)	25 (12.5)	22 (11.2)
	White (non‐Hispanic)	438 (73.1)	148 (73.3)	146 (73.0)	144 (73.1)
	Other	95 (15.9)	35 (17.3)	29 (14.5)	31 (15.7)
Education	Not college graduate	120 (20.0)	39 (19.3)	46 (23.0)	35 (17.8)
	College graduate	479 (80.0)	163 (80.7)	154 (77.0)	162 (82.2)
Body mass index (kg m−2)	<25	277 (46.2)	97 (48.0)	84 (42.0)	96 (48.7)
	≥25	322 (53.8)	105 (52.0)	116 (58.0)	101 (51.3)

The primary outcome for this study are the online self‐reported weights submitted by participants over a 2‐year period between March 2012 and March 2014. The date and mode used to report the weight were recorded. Because SNAP was conducted at two sites with concurrent cohort recruitment and randomization, the final two cohorts of the study (n = 85), randomized in February 2012, are excluded from the analysis of the modalities and the first refresher campaign because they were not yet eligible to participate in these study activities. During the intervention, participants were suspended from intervention involvement until 6 months post‐partum if they reported a pregnancy (n = 48). These participants also are excluded from the analyses and specific Ns are noted on tables and figures.

Data analysis

To examine the effect of direct email reporting, the 4 weeks prior to the introduction of email reporting were designated as the ‘PRE’ period and the 4 weeks after were classified as the ‘POST’ period. To examine the effects of refresher campaigns, data were separated for each campaign period: the 4 weeks prior to the campaign (PRE) were compared with the four active weeks of the campaign (CAMPAIGN) and to the 4 weeks after the campaign ended (POST). All eligible participants were classified as reporting one or more weights during each of the examined periods or no weights reported. Changes in percentage of participants reporting and number of total number of weights reported during the period were analysed using PROC GENMOD with repeated measures for a binomial or Poisson distribution and a logit link function. All analyses were conducted in sas 9.4 and controlled for clinic site.

Results

Overall engagement was examined over the time period of March 2012 to March 2014, representing an average of 2.8 years (SD = 0.5; range 2.1–3.4 years) from randomization. Figure 2 shows the percentage of the participants who reported their weight at least once per month during each month of this period. On average, 66.9% (SD = 10.3) of eligible participants reported at least one weight per month via any mode of reporting.

Figure 2

Percentage of reporting weight at least once a month. * indicates that month email reporting was introduced. Striped bars and † indicate months with active campaigns. Note that campaigns did not overlap entirely with calendar months; therefore, values in the figure are not expected to replicate values in the table. N's varied over months as cohorts were recruited, and participants were suspended due to pregnancy. N's possible per month are noted below the axis.

Methods of reporting

To assess whether the introduction of direct reporting to the study website from email and email reminders increased the proportion of participants reporting weights, the 4 weeks prior to email introduction (PRE) on 27 April 2012 were compared with the 4 weeks after (POST). The percentage of eligible participants (N = 312) who reported their weight increased from 57.1% in the month prior to this introduction to 75.0% in the month following (p < 0.001). The number of methods participants used to report their weights also increased from an average of 1.08 methods (SD = 0.30) PRE email introduction to 1.38 (SD = 0.55; p < 0.001) POST email. Whereas 92.6% of participants used only one method to report their weights and 7.4% used two methods prior to the introduction of email reporting (see Figure 3 for methods used), following its introduction, only 64.8% used only one method, 32.2% used two methods and 3.0% used three methods. The number of weights reported by participants also increased after email reporting was introduced from 4.5 (±7.6) weights per month to 5.2 (±7.4) weights per month (p < 0.001). Among those reporting, there were significant relationships between the number of modalities used to report weights and the number of weights reported (PRE: Spearman's r = 0.20, p = 0.007; POST: r = 0.43, p < 0.001).

Figure 3

Modes of reporting pre and post email introduction.

The number of methods used for data submission and the types chosen also were associated with the number of weights submitted. Prior to the email/reminders introduction, those using both the website and mobile options reported more weights (14.54 ± 10.66) than those using only one option (website (7.58 ± 8.69) or mobile (6.50 ± 6.89); p values <0.01. The number of weights submitted also was associated with the mode or number of modes used post‐email and reminder introduction (p < 0.001) such that those using two or more modes reported the most weights each month. Those who reported only via email reported fewer weights (2.16 ± 1.0) than those only using the web (9.47 ± 9.13), mobile only (6.67 ± 6.67) or two or more modes to report their weights (10.02 ± 8.66; p values <0.01).

Refresher campaigns

There were no significant changes in the proportion of participants engaging with the intervention from the PRE to the CAMPAIGN period for the any of the three campaigns (see Table 3; p = 0.62; p = 0.45; p = 0.75, respectively, for three campaigns). However, participants reported significantly more weights per month on average during the CAMPAIGN than during the PRE periods (all p's < 0.001). The change in percentage of participants reporting between PRE and the POST periods were significant in each comparison with compliance decreasing in the months after the campaigns (respectively: p = 0.02; p < 0.001; p = 0.04). Following the fall refresher campaigns, the number of weights submitted returned to similar levels as PRE campaign in two campaigns (Fall 2012, p = 0.89; Fall 2013, p = 0.06) and was lower compared with the PRE period in the Spring 2013 campaign (p < 0.001).

Table 3

Participant engagement and number of weights reported before, during and after refresher campaigns

	n a	Preb	Campaign	Post	Pre to campaign difference (p value)	Pre to post difference (p value)
Campaign Fall 2012 (1–29 October 2012)	296
Proportion of participants reporting		68.9%	69.9%	62.8%	0.62	0.02
Number of weights reported/participant		3.5 ± 5.5	6.7 ± 8.6	3.5 ± 5.5	<0.001	0.89
Campaign Spring 2013 (22 April to 19 May 2013)	377
Proportion of participants reporting		71.9%	70.8%	63.9%	0.45	<0.001
Number of weights reported/participant		3.3 ± 5.0	3.9 ± 4.9	1.9 ± 3.0	<0.001	<0.001
Campaign Fall 2013 (21 October to 17 November 2013)	369
Proportion of participants reporting		62.0%	62.5%	58.2%	0.75	0.04
Number of weights reported/participant		2.6 ± 4.3	3.3 ± 4.7	2.4 ± 4.2	<0.001	0.06

Number of participants who were eligible for monthly weight reporting (i.e. had not been suspended from intervention (e.g. due to pregnancy) and had not terminated participation in the study). Indicate again that some cohorts had not reached this point and so were not eligible.

Values are observed as percentages or means and standard deviations.

Discussion

This study capitalized on observational data from SNAP, a study of long‐term interventions using multiple eHealth methods for weight reporting, to examine the effects of various efforts to sustain engagement over time. Over two‐thirds of the sample of young adults remained engaged with eHealth tools over the 2 years of this analysis. Offering more modalities for participants to report weight increased the proportion of participants who engaged with this behavior and those who used multiple modes to report weight reported more weights per month on average. Time‐limited periodic refresher campaigns did not increase the proportion of participants who reported their weight but increased the frequency that participants reported their weights during the month of the campaign. Taken together, these findings suggest that using multiple eHealth modalities and periodic campaigns may be effective in sustaining engagement with weight reporting in a young adult population over time and could be examined for other behaviors and populations.

Historically, young adults have been more likely to drop out of weight control interventions and have poor adherence 18, 19. At the end of the Spring 2013 refresher campaign, participants in this study were an average of 2.8 years post‐randomization and almost 60% were reporting weight at least monthly. By comparison, in CHOICES, another long‐term weight gain prevention intervention for young adults, 27% of participants were still reporting weight at least monthly at 2 years post‐randomization 20, 21. Differences in the two studies that might be related to engagement are informed by this analysis. In SNAP, participants had multiple modalities available to report weight and periodic campaigns designed to boost engagement. CHOICES, in contrast, used one modality (the web) for reporting and ongoing incentives for logging in and reporting weight and behavior. In future research, randomizing participants to receive interventions that differ on features or methods designed to boost engagement would permit more rigorous tests of these hypotheses.

Although introduction of an email reminder with auto‐login increased the number of participants reporting weight, participants who relied only on receiving the email reminder to prompt weight reporting submitted fewer weights than those who used the other modes or combination of modes. Interestingly, although SMS could include reminders, in the present study, web and SMS did not include reminders to report. Thus, the finding that more frequent weights were received from those using these other modalities may be a reflection of users who proactively reported their weight or had established a routine of reporting their weight versus those who reported in response to being reminded. The frequency of weight reporting by approach supports this hypothesis; those who reported via email averaged only two weights per month (reminders were biweekly) and those reporting via other methods used alone or in combination reported 7 to 10 weights per month. Therefore, this finding may reflect motivation or other aspects of the reminder system rather than characteristics of the modality, per se. As technology has evolved since this study's inception, future platforms could include automated syncing of weight or other behaviors from a smart scale, activity tracker or app that would make separate reporting unnecessary. However, periodic messages to encourage the use of technologies likely will still be required to mitigate lapses in weighing or other behaviors in long‐term interventions. Furthermore, at present, resources do not always permit the use of such integrated technologies in many populations and settings; thus, it is likely that some checking‐in or promoting behaviors and reporting over time will still be necessary.

Offering periodic campaigns was effective in increasing the number of weights being reported but did not significantly increase the proportion of participants who reported compared with the previous month. Given the consistent prior finding that engagement declines over time 5, 8, 22, 23, 24 and the results presented here, it is clear that the trend is for fewer participants to report over time. Thus, while the campaigns did not boost the percentage of participation, it perhaps sustained participation during a month when participation otherwise would have declined by a few percentage points. Unfortunately, immediately after a campaign, the proportion of participants reporting significantly declined compared with pre‐campaign levels by about 5–7%. It is unclear why this occurred, but the pattern was consistent across campaigns. Future studies might consider increased frequency or shorter duration campaigns to potentially mitigate these declines. It is also important to note that the refresher campaigns included some form of incentive (either small monetary incentive or weight related gifts) linked to reporting on campaign activities that likely encouraged additional weight reporting during this month for those who participated. SNAP used biannual campaigns; however, the frequency, timing and effects of campaigns in general would benefit from additional study.

This study has several limitations. The design is observational and non‐randomized. The use of control periods before and after the campaigns, the temporal pattern and the consistency of the findings lend strength to the potential causal effect of these interventions on engagement, although causal inferences are better drawn from a randomized controlled trial. There are potential carry‐over effects from one campaign to another, which diminish the ability to determine the full effect of subsequent campaigns. This analysis also considers only one form of engagement, weight reporting using various technologies, and results may vary with other forms of eHealth engagement. Finally, the SNAP study sample was recruited for weight gain prevention, included no participants with obesity at baseline and had limited diversity. Thus, these findings may not generalize to other populations. Despite these limitations, the study also has several important strengths. SNAP provides long‐term data with discrete intervention periods to examine patterns of engagement using an objectively measured outcome of weight submissions. The study contributes to the literature on building a science of engagement and offers an example of a natural experiment embedded within an ongoing randomized controlled trial, which can be conducted without competing or distracting from the overall study outcome. Finally, this study provides some of the first evidence of the effect of periodic campaigns delivered as part of long‐term behavioral weight control programs on markers of program engagement.

As the use of eHealth interventions continues to increase, understanding aspects of long‐term, perhaps episodic, engagement with these interventions will be vital for creating meaningful health effects. This study provides an important first step by evaluating the effects of two types of engagement efforts, offering multiple approaches to reporting self‐monitoring data and refresher campaigns, as part of an effective weight gain prevention intervention, with good levels of engagement over 2–3 years. Continuing to focus on how, and what, promotes participant engagement with eHealth interventions will be important to develop a science of eHealth engagement to guide future efforts.

Conflict of Interest Statement

Dr. Tate is a member of the Scientific Advisory Board for WW.

Funding

This study was supported by the National Institutes of Health/National Heart, Lung, and Blood Institute grants U01HL090864 and U01HL090875.