Adherence in Internet-based interventions.

BACKGROUND: The Internet is a promising channel for delivering health-promoting interventions. A common problem for Internet-based interventions is low adherence. The current paper reports adherence rates from three different Internet-based trials with potential covariates.
METHODS: Data on adherence and baseline characteristics of users were collected from three different Internet-based trials: one supporting diabetes self-management, one supporting smoking cessation, and one offering an online personal health record. Logging of web use was used as the measure of adherence in two of the trials, while logging of authentication SMS messages was used in the third.
RESULTS: In all three trials, users dropped out at a high rate early in the intervention. The baseline variables that were related to use were self-efficacy, having smoking friends, age, gender, and education. Tailored emails increased use for up to five months into a one-year intervention.
CONCLUSION: Dropout from Internet-based trials is substantial, and attrition curves can be a valuable tool for more accurate pretrial estimates of sample size power. Automated follow-up of users via email seems likely to increase adherence and should be included in Internet-based interventions. Tailoring on baseline covariates to adherence such as self-efficacy could make them even more effective.

Background

The Internet is a promising channel for strengthening self-management skills. Web-based education and self-care programs improve patient knowledge and have shown to be effective in managing several chronic conditions and lifestyle changes. Favorable outcomes have, for instance, been found in nutrition (Brug et al 1999), physical activity (Vandelanotte et al 2007), and smoking cessation (Strecher 1999; Walters et al 2006).

Reviews of Internet-based interventions can, however, present a rather confusing picture as the only common ground is the delivery medium. The interventions may range from posting pamphlets online, through dynamic combinations of text-based information and communicative features, such as forums or “ask an expert” or multimedia, to individually computer tailored content. The latter, personalizing the intervention based on user characteristics, is the component that has most consistently been shown to be effective (Brug et al 1999; Kreuter and Skinner 2000; Oenema et al 2001; Strecher 1999; Strecher et al 2006; Walters et al 2006; Johnson et al 2007; Vandelanotte et al 2007).

What all Internet-based interventions have in common is great potential for reaching people. In Norway, where the trials reported in the current study were conducted, 85% of the population has used the Internet during the last three months while 58% has used it for health purposes (Andreassen et al 2006; Statistics Norway 2007).

Another common feature for these Internet interventions is what other studies have termed low adherence. Despite the fact that most participants in Internet-based trials volunteer, and thus must have some intention to use the program, a substantial proportion of them never use the program, and many use it only once. Discontinued use has for instance been seen in studies of Internet-based interventions for panic disorder (Farvolden et al 2005), depression (Christensen et al 2004), weight loss (Vandelanotte et al 2007; Verheijden et al 2007), diabetes self-management (Glasgow et al 2003; Wangberg 2008), and smoking cessation (Feil et al 2003; Strecher et al 2005; Swartz et al 2006).

Since a dose-response relationship for the use of Internet-based interventions has been found (Eysenbach 2002; Cobb et al 2005; An et al 2006; Japuntich et al 2006; Graham et al 2007; Steele et al 2007), low adherence could underestimate the efficacy of such interventions. In this respect it is important to differentiate the concept of retention of recruited users with which we are concerned from initial reach, a relevant factor to take into account when considering the effectiveness of these interventions (Green and Glasgow 2006).

While characteristics of those lost to follow-up assessment are routinely reported, fewer studies have looked directly at covariates of use. Knowledge about aspects that influence use may be a valuable aid in designing studies. Previous research has found that older people (Japuntich et al 2006; Verheijden et al 2007), users with higher self-efficacy for target health behavior (Steele et al 2007), and women (Feil et al 2003; Glasgow et al 2007) have higher adherence to Internet-based interventions. A recent more general meta-analysis of patients’ adherence to prevention and treatment of a variety of acute and chronic conditions found that patients’ belief in the severity of their disease predicts adherence (DiMatteo et al 2007). Thus, we could also expect to see correlations between needs such as long-term illness, subjective health status, or poor self-efficacy for self-management to be related to the use of Internet-based programs.

Eysenbach (2005) has called for more research on attrition from Internet-based interventions. He argues for the need to develop models for discontinuation of eHealth applications and dropouts in Internet-based trials. Christensen and Mackinnon (2006) found that there has been little debate about this issue despite the fact that attrition is a major problem in the use and evaluation of Internet sites. In this paper, we will contribute to this debate by modeling attrition and assessing factors related to adherence. The main objectives are to report and model adherence curves from three different Internet-based trials, to investigate whether covariates to use exist and to analyze the need for tailoring. We hypothesize a substantial dropout early on in the interventions, and a slower decline in use thereafter. Further, we expect health status, age, gender, and self-efficacy to be related to use.

Method

We used data from three Internet-based interventions: Study 1: an Internet-based diabetes self-care intervention; Study 2: an Internet-based smoking cessation intervention; and Study 3: an Internet-based personal health record. All study protocols were submitted to review by a regional ethics committee before any contact was made with participants.

Study 1: Internet-based diabetes self-care intervention

Via an advertisement on the Norwegian Diabetes Association’s website, 90 persons signed up to participate in the trial. Inclusion criteria were 1) diagnosis of type 1 or type 2 diabetes, and 2) having access to the Internet. The baseline questionnaire had to be completed before access was given to the website directly thereafter. The intervention was tailored to the level of self-efficacy, so that participants were randomized to receive intervention exclusively on that aspect of self-care (blood glucose monitoring; diet care; physical activity) with a) the lowest self-rated self-efficacy or b) the highest self-rated self-efficacy. The interventions consisted of several components derived mainly from social cognitive theory (Bandura 2004), which were intended to increase the performance of self-care such as peer modeling through videos, information, and tools to monitor own target behavior. The users received no particular instructions as to how often they should log on to the site.

Adherence was assessed as the number of logins and time spent at the intervention website gathered through web logging per anonymous and unique user ID. The following assumptions were made in order to estimate the number of accesses and time spent there: if no activity was detected in a 15-minute period, the next onset of activity was coded as a new visit.

The perceived competence scales (PCS) were used for assessing self-efficacy for glucose monitoring, diet, and physical activity, respectively. The perceived competence for glucose monitoring scale consists of four items representing the degree to which a person feels he or she can manage daily aspects of glucose monitoring. Responses are given on a scale from 0 (not at all) to 6 (to a great extent). Previous research has shown good psychometric properties of the scales (Williams et al 2004). Perceived external barriers were measured in a similar fashion with three items such as “external barriers such as lack of time or money often make it hard for me to be physically active”. The internal consistency for this scale was acceptable with a Cronbach’s alpha of 0.75. The perceived external barriers scale was also found to have a good concurrent validity by being negatively related to self-efficacy, r = −0.47, p < 0.001. For more details on this sub-study, please see Wangberg (2008).

Study 2: Internet-based smoking cessation intervention

Participants were mainly recruited through media coverage of the project and snowball sampling, but the intervention's URL was also spread in various other ways. The participants were randomized into two groups: 1) A tailored group who had access to tailored content on the website and received tailored emails. The emails were received daily around their preset quit date, and thereafter more seldom throughout the year. 2) A control group who had access to exactly the same website, but without the tailored content and emails. Both groups received information about various aspects of smoking cessation and had access to the online community features such as the discussion forum and guest books. Because of continuous recruitment to the program, different samples were used for different analyses. By September 23, 2006 a total of 618 users had been registered in the Internet-based smoking cessation program, and could thus by the time of analysis potentially have used the intervention for a year. While the participants in the tailored group received emails requesting them to visit the site and tell us how they were doing at 3 weeks, 6 weeks, 3 months, 6 months, 9 months, and 12 months after their quit date, the control group received no instructions as to how often they should log on to the site.

A survival analysis was performed with the web-log data from the 618 first user IDs in the data set, since we were unable to link the logins to registration date per user. If no activity was registered for a user during 40 minutes, they were automatically logged out. Data on self-efficacy were gathered at baseline and after one month’s use of the site using the Smoking Self-Efficacy Questionnaire (SEQ; Etter et al 2000).

Study 3: Internet-based personal health record

Participants were recruited by mailing a four-page information pamphlet to all members (approximately 6500) of the Norwegian patient organization for people with thyroid disorder. Information was also published in the organization's magazine and on their website. Volunteer participants were randomized during online registration and given individual usernames and passwords that would enable participants to log in on the project site to answer the baseline and later the one-month follow-up questionnaires. Emails distributed after registration contained information about login procedure at a separate web address (intervention group, n = 218), or contained a PDF file that could be printed and used as a paper-based health record (control group, n = 192). Login required an authenticating code sent by short message service (SMS) to the participant’s mobile phone, in addition to entering a username and a password at the intervention web-site.

The number of these SMS messages sent out per user was used as the main measure of adherence. In addition, the users were asked to self-report their number of logins.

The intervention was an Internet-based, nonspecific symptom monitoring tool that enabled users to register symptoms, rate these for “severity”, and monitor how they developed over time. Symptom development could be monitored through graphic illustrations, and users could choose to print a “report” of selected symptoms, for instance to use in consultations with health professionals. The users received no particular instructions as to how often they should log on.

Analyses

In Study 1 and 2, logging of web use, number of logins, and time spent at the site were used as a measure of adherence, while in the third study logging of authentication SMS messages was used. Groups were compared using ANOVA. A survival analysis where survival (adherence) was defined as at least one login every month was performed with data from the 618 persons who had been registered for more than a year in Study 2. Functions for use were estimated through curve fitting exponential and logarithmic functions to the data. These functions were chosen on the basis of the general shape of the attrition curves from previous research. A linear multiple regression was performed with baseline characteristics as predictors and number of logins to an Internet-based smoking cessation intervention as the dependent variable to assess which variables was related to adherence. Pearson’s correlation analysis was performed to assess relations between two continuous variables. All reported confidence intervals (CI) are 95%, and all analyses were performed with SPSS 15.0 (SPSS, Inc., Chicago, IL, USA).

Results

Of the 90 people who signed up for the study, 64 (71%) completed the baseline questionnaire. We know that at least one person had signed up more than once, and that at least one of the email addresses provided was invalid. Of the 64 who completed the baseline questionnaire, and through that gained access to the intervention, 18 (28%) never accessed the site again; 13 (20%) logged in twice, while the remaining 33 (34%) logged in from 3 to 25 times during the six weeks. Login time per week can be seen in Figure 1. It appears that in general the participants spend time on the interventions when first given access, and then do not return before one month post-assessment. Neither a logarithmic function (R2 = 0.62, CI = <0.0001–0.81) nor an exponential function fits the data significantly (R2 = 0.33, CI = <0.0001–0.67). The correlation between number of logins and minutes spent at the site was r = 0.60 (0.41–0.73). A plot of proportion of users per week (not shown) also corresponded well with number of logins per week.

Figure 1

Mean number of logins at site per week of an Internet-based intervention for diabetes self-management (N = 64).

Those who had used the intervention site only once did not differ significantly from those who had used the site more often on any baseline variables. This included which version of the intervention they were randomized to, type of diabetes, duration of diabetes, whether they used insulin or not, blood glucose variability (HbA1C), body mass index, self-management, self-efficacy, value conflicts, intention, perceived social norms, gender, or age. The two factors that differed the most between the groups were perceived external barriers to diabetes self-management and educational attainment. Perceived external barriers were on average higher among those who had logged in only once (M = 8.6, CI = 6.5–10.8) than for those who had logged in more than once (M = 5.8, CI = 4.3–7.2) while education was on average lower among those who had logged in only once (M = 2.1, CI = 1.6–2.7) than for those who had logged in more than once (M = 2.8, CI = 2.5–3.1).

For the 2884 registered users of the Internet-based smoking cessation program from August 15, 2006 until October 23, 2007, the mean number of logins was 15 (12.7–17.2). Of these, 26 (0.9%) people dropped out of the intervention during the registration process, and were therefore registered with zero logins. The correlation between the number of logins and minutes spent at the site was r = 0.89 (0.89–0.90). Recruitment was continuous, so some of these users had used the site for a year, while others only for hours. The mean number of logins per month for the tailored group and the control group can be seen in Figure 2. The tailored group had more logins during the first months (significantly more in month 2, 3, 4, and 5) than the control group, who received no emails and no tailored content. But this difference disappeared by month 10, when participants in both groups had logged in 0.13 times on average (0.06–0.21).

Figure 2

Mean number of logins per month for the tailored (N = 1469) and the control group (N = 1415) in an Internet-based smoking cessation intervention.

A survival analysis including the 618 persons who had been registered for more than a year showed that 19% (61) of the users were still adherent in the second month of the intervention, and 5 people (0.8%) made it to the end of the one-year intervention. Mean survival time for the tailored group was 1.7 (1.5–1.9) months compared with 1.3 (1.2–1.4) months for the control group. Plotting the reduction in the number of logins for this group did not show significant differences in fit between an exponential (R2 = 0.91, CI = 0.63–0.95) and a logarithmic (R2 = 0.89, CI = 0.58–0.94) function (Figure 3).

Figure 3

Mean logins per month for one-year users of the Internet-based smoking cessation program Opptur (N = 618).

Correlations between baseline characteristics (age, gender, education, living alone, cigarettes per day, proportion of friends smoking, and self-efficacy for smoking cessation) and use were assessed for 1895 users who had registered before February 27, 2007 (Table 1). The average effect sizes were small for all variables, but being older, being female, having a higher education, having few smoking friends, and a higher self-efficacy for cessation were associated with more logins.

Table 1

Regression with baseline characteristics as predictors and number of logins to an Internet-based smoking cessation intervention as the dependent variable (N = 1895)

Baseline characteristics	B (95 CI)	β
Age	0.5 (0.1–0.8)	0.08
Education	−3.9 (−7.2–0.6)	−0.06
Being female	13.8 (5.8–21.9)	0.09
Living alone	1.8 (−7.0–10.6)	0.01
Cigarettes per day	0.5 (−0.3–1.0)	0.05
Proportion of friends smoking	−7.0 (−13.7–0.4)	−0.06
Self-efficacy for cessation	0.6 (0.3–0.8)	0.11

At the start of the trial, 410 people with thyroid disorders had signed up and completed the baseline questionnaire. Of these, 210 (51%) completed the one-month follow-up questionnaire. Only one significant difference in baseline characteristics was found between responders and nonresponders. Nonresponders had a slightly higher mean expected number of uses of the Internet-based personal health record during the next year, 1.6 (1.5–1.8) compared with the responders, 1.3 (1.2–1.4). They were equally distributed across the intervention and control arm of the trial, across gender, education, age, income, self-efficacy, illness perceptions, and subjective health.

Of the 218 participants with access to the Internet-based personal health record, 168 (77%) did not answer a question about how much they had used it during the last month. Thirty-two responders (15%) reported that they had not used it, while of those remaining, 14 (6%) responders reported having used the program only once, which leaves 4 (2%) people who reported having used the intervention more than once. In total 62 logins during the last month of the intervention were reported. System logging had registered 121 logins over approximately 3 months up to the day before the last follow-up questionnaire was submitted, equaling a mean of 40 logins per month. In Figure 4 the distribution of logins over the trial can be seen. Neither a logarithmic function (R2 = 0.06, CI = <0.0001–0.37) nor an exponential function fitted the data significantly (R2 = 0.22, CI = <0.0001–0.53).

Figure 4

Mean number of logins per month to an Internet-based personal health record (N = 218).

Before the intervention, these persons were asked how much they thought they would be using the Internet-based personal health record. What they said they would do did not correlate with what they reported that they actually did, r = 0.048 (−0.24–0.33). Three (1%) participants did not answer. Ten (5%) said they would use it daily, 109 (50%) said they would use it every week, 79 (36%) monthly, while 17 (8%) reported they would use it more seldom.

Discussion

In all our three trials we found that many users never used the intervention, substantial proportions of the participants used the interventions only once, and few used the programs until the end of the interventions. This corresponds to previous research (Feil et al 2003; Glasgow et al 2003; Christensen et al 2004; Eysenbach 2005; Farvolden et al 2005; Strecher et al 2005; Swartz et al 2006; Vandelanotte et al 2007; Verheijden et al 2007). In a recent review of Internet-based interventions for physical activity, attrition rates at short-term follow-up (<3 months) ranged from 10% to 50% (Vandelanotte et al 2007). In our diabetes self-management trial, about half the participants used the intervention more than once, and in the smoking cessation trial, 19% were still using the program by the second month, and only 0.8% used the program at least monthly throughout the one-year intervention.

High correlations were found between number of logins and time spent at the site in both the diabetes self-management and the smoking cessation intervention. The number of logins was also proportional to the number of users in the intervention at any given time. This suggests that these different indicators of adherence are relatively comparable. Since the number of logins was available for all three trials, we chose this variable for plotting attrition curves. Only the smoking cessation trial had power to estimate a significant function. Both a logarithmic and an exponential function fit the data well, essentially showing that use is reduced by 25% every month of the intervention. Although not significant, the attrition appeared to be about twice as fast in the other two interventions.

One might have expected to find differences between adherence to interventions for self-managing chronic disease, which is a lifetime venture, versus quitting smoking, which might be considered a more time-limited project, albeit one that for the majority of quitters is started more than once. On the other hand, it may be argued that all health behaviors are in essence the same and that their maintenance may require conscious effort and support for a long time. Newsom and colleagues (2005) found in a large North American survey that health behaviors were largely independent of each other. Others would argue that even though the behaviors in question may differ, the process of changing them, and thus interventions to support them, have more similarities than differences (Strecher et al 2002; Jones et al 2003; Johnson et al 2007). Reviews of interventions based on common stages of change principles have, however, found limited overall evidence for effects (Bridle et al 2005; West 2005). Nevertheless, in this study, and in the previous Internet-based interventions reviewed, attrition curves seem similar. We therefore saw it as an interesting research question to pursue whether there were similarities in factors related to attrition over three different interventions too.

Surprisingly, we found no correlations between needs such as severity of illness, subjective health, illness perceptions, current self-care, or number of cigarettes per day, and use of the interventions. This is in contrast to adherence research in other fields (DiMatteo et al 2007).

In the smoking cessation trial we found that higher self-efficacy for quitting predicted higher use. Steele and colleagues (2007) found that higher self-efficacy for physical activity (but not for using the Internet) was related to greater program exposure. Higher self-efficacy for changing the target health behavior was not found to be significantly related to use in the Internet-based diabetes self-management trial, but was found to be related to better intervention effects in a previous study (Wangberg 2008). Previous research has found that self-efficacy is an important factor for engaging in and maintaining a host of health behaviors (AbuSabha and Achterberg 1997; Ockene et al 2000; Bandura 2004). Taken together this suggests that self-efficacy is an important variable to take into account when tailoring an intervention.

The use of tailored emails and website content increased adherence up to five months into the smoking cessation intervention, and emailed post-assessment reminders increased the use in the two other trials. This is in contrast with Tate and colleagues (2006), who found that the participants who received automated tailored messages had the lowest number of logins, while they found no difference between the access to an email counseling condition and a basic intervention condition. Still, we believe that in most cases follow-up is likely to increase use by reminding the participant to visit the site. Mobile text messaging (SMS) could be an even more effective way of achieving this through its even greater ability to reach users regardless of time and location and without login procedures (Wangberg et al 2006).

There is reason to believe that tailoring these messages may have an independent effect. “Tailoring” is a wide-ranging concept, however. Dijkstra (2005) suggests that tailoring approaches can be classified into three basic groups: 1) personalization, eg, greeting the user by first name; 2) feedback, eg, giving the user information about how her level of physical activity compares with a norm group; and 3) adoption of content in relation to theory. This latter group in turn contains a variety of approaches, such as receiving feedback on current health behavior (Verheijden et al 2007), focusing on a specific behavior according to self-efficacy (Wangberg 2008), or presenting information relevant to the participant’s stage of change (Johnson et al 2007). It is not always clear from reports what a specific tailored intervention actually does and which mechanisms are most effective. There is also some doubt about whether it is the actual tailoring or the perception of such that accounts for the observed effects (Webb et al 2005).

Future research should assess whether there is an additional effect of tailoring emails on adherence, and if so which kind of tailoring is most effective. It could also be useful to separate the adherence to interventions from adherence to the objective of the intervention, eg, smoking cessation. Tomson and colleagues (2005) found that nonresponders to a quitline trial were even more likely to have quit smoking than the responders, thus clearly showing that quitting an intervention does not necessarily mean having given up on behavior change. In fact, attrition could in some cases even be an indicator of success, suggesting that the intervention is no longer needed.

Limitations

As seen from the wide confidence intervals, a key limitation of the data from our two smallest trials, as well as for several of the studies reviewed, is that they do not have adequate statistical power to enable assessment of subgroup differences between users and nonusers, or relations between use and other variables.

Variance was increased in the study of the Internet-based personal health record by using a fixed answering format (daily, weekly, monthly) for predicted use, while an open response format (number of times during last month) was used for actual use.

Conclusions

In Internet-based trials one should plan for the worst case scenario of losing half of the participants during the first month of the intervention. Education and self-efficacy seem to be factors that should be taken into account in the design of interventions in order to increase adherence. Furthermore, emails are likely to lead to more logins, at least during the first months of an Internet-based intervention. Whether tailoring these emails has an additional effect warrants further research.