Predictors of acceptability and engagement in a self-guided online program for depression and anxiety.

BACKGROUND: Low engagement with self-guided online programs limits the potential of these programs to provide effective and low-cost treatment of mild to moderate depression and anxiety at scale. Identifying factors that increase uptake and adherence in self-guided online programs may facilitate the development of targeted implementation strategies to increase engagement with these programs in the community. Using data from a randomized controlled trial of a self-guided online program for depression and anxiety, the aim of this study was to identify predictors of the acceptability of internet-based psychological programs, and engagement (uptake and adherence) with the online program tested in the trial.
METHODS: A total of 556 community members with elevated symptoms of depression or anxiety were recruited via social media into the two active conditions of a three-arm randomized controlled trial. This trial tested the effectiveness of a 7-week self-guided online program for depression and anxiety called myCompass 2, delivered with or without an Engagement-Facilitation Intervention. Predictors of uptake (accessing at least one therapeutic module of the program), adherence (modules completed), and acceptability of internet-based psychological programs (Unified Theory of Acceptance and Use of Technology, UTAUT scale) were examined, including demographics, mental health status, help-seeking attitudes, stigma, acceptability of internet programs, and personality factors.
RESULTS: Logistic regression demonstrated that higher levels of conscientiousness (OR = 1.06, p = .026, 95% CI =1.01-1.12), and acceptability of internet-based psychological programs (OR = 1.09, p = .005, 95% CI =1.03-1.16) predicted greater uptake, and that failing to complete a module was predicted by lower levels of acceptability (OR = 0.88, p = .027, 95% CI =0.78-0.99). Linear regression showed that higher levels of agreeableness (t = 4.66, p < .001), lower levels of stigma (t = -2.28, p = .023) and more positive help-seeking attitudes (t = 2.05, p = .041) predicted higher acceptability attitudes. DISCUSSION: Acceptability of internet-based psychological programs was identified as a factor that increased both uptake and adherence to the myCompass 2 program. Efforts to increase the acceptability of these programs may improve engagement with these programs in the community. It may also be useful to consider personality traits and clinical profiles when considering the appropriate audience for self-guided internet interventions.

Introduction

Depression and anxiety are high prevalence disorders that contribute considerably to the global burden of disease (Kessler and Bromet, 2013; Lim et al., 2018; Baxter et al., 2012). However, many people with these conditions do not seek professional help (Burgess et al., 2009). Broad provision of evidence-based, low intensity, self-guided online mental health programs in the community is proposed as a potential method of improving access to treatment in people who may not otherwise seek it (Karyotaki et al., 2017; Batterham et al., 2015). These programs have been shown to prevent or treat depression and anxiety using similar therapy to that provided through face-to-face care (Karyotaki et al., 2017, Batterham et al., 2015). Critically, the role of online programs may be even more significant during times of public health crises, such as pandemics (Moreno et al., 2020), as they provide an evidence-based alternative when face-to-face care may be inaccessible. However, despite the effectiveness of these programs and their availability in the community, many users do not adhere sufficiently to them to obtain a therapeutic benefit (Christensen et al., 2009). Research trials of self-guided e-mental health programs also frequently suffer from poor adherence. For example, a systematic review of cognitive behavior therapy (CBT) delivered by computer found only half (56%) of users completed the full program, compared with 65% of those using a guided version, and 85% for face-to-face (Waller and Gilbody, 2009). Studies investigating rates of uptake (i.e. initially accessing or starting a program) are lacking; however, one study by Healey et al. (2017) reported rates of registration at 49% (74,233 registrations) for 152,933 unique visitors to a CBT web program for depression during a 6-month period in 2015.

Reasons for poor uptake and adherence to self-guided programs are complex. A systematic review by Batterham et al. (2008) examining self-guided online interventions for a range of mental health conditions found several factors were associated with adherence including female gender, higher treatment expectancy, having sufficient time, the intervention content being desirable, and feeling as though there was a person behind the intervention (e.g., including a photo of the creators). For self-guided online programs that target depression and anxiety specifically, previous studies have reported that lower symptoms at baseline (Christensen et al., 2009; Farrer et al., 2014), younger age, poorer knowledge of psychological treatments (Christensen et al., 2009), male gender, lower education, and comorbid anxiety symptoms (Karyotaki et al., 2015) predict greater adherence among adults. However, when measured in naturalistic settings outside the context of a clinical trial, different factors have emerged as being predictive of adherence, such as higher symptom severity for both depression and anxiety, higher dysfunctional thinking, younger age, higher education, female gender, and mental health professional referral to the program (Batterham et al., 2008). It is not currently clear if or why these discrepancies may exist between research and naturalistic settings. Some possible explanations include differences in methodology (e.g., differing definitions and measures of adherence or uptake), differences between interventions, or differences in target populations and delivery settings, which may also influence expectations around treatment outcomes (Batterham et al., 2008).

Reasons for non-adherence may be considered as negative (e.g., did not gain benefit), neutral (e.g., did not perceive a need) or positive (e.g., gained sufficient benefit without completing the full intervention) (Davis and Addis, 1999; Batterham et al., 2008; Christensen et al., 2009; Donkin et al., 2011; Donkin and Glozier, 2012). There is further complexity when considering community-based users who are not referred or supported by a clinician to use these programs. Such individuals may encounter an intervention through diverse pathways, such as perceiving a need for support and using a search engine or directory of programs, responding to an advertisement, receiving a recommendation from family or friends, or simply trying an intervention out of curiosity (e.g., (Bennett and Glasgow, 2009, Coulson et al., 2016, Donkin and Glozier, 2012)). Low adherence is likely to have different implications depending on the needs and expectations of the user, and the reasons for non-adherence. Producing a clinically significant change in a small proportion of motivated users may have important public health impact (Batterham et al., 2021). Nevertheless, early intervention dropout is likely to be associated with reduced clinical benefit, so it is important to characterize users who are least likely to adhere to facilitate better targeting and design of such interventions.

Predictors of uptake have been comparatively less researched, particularly in general community-based settings. One study by Cillessen et al. (2020) that examined an online mindfulness intervention with therapist support for psychological distress in people with cancer found that both uptake and adherence were predicted by the personality trait of conscientiousness. A similar study (Beatty et al., 2017) examining a CBT program for people with cancer showed that younger age may also be associated with greater uptake. A review of smart phone applications (apps) indicated that there were 26 different factors that influenced uptake and engagement with apps; those associated with uptake included personal recommendations, awareness of the app, and availability at low-cost (Szinay et al., 2020).

Acceptability of online mental health programs is also thought to be a factor in promoting engagement with these programs. Acceptance-Facilitation Interventions (AFIs) are brief information packages that are designed specifically to increase acceptability and thus subsequent engagement in an online program (Ebert et al., 2015). Previous randomized controlled trials of AFIs have demonstrated an increase in acceptance attitudes for people with chronic pain (Baumeister et al., 2015), diabetes (Baumeister et al., 2014) and for depression (Ebert et al., 2015). The next generation of these, called Engagement-Facilitation Interventions (EFIs), target the improvement of actual engagement (uptake and adherence) behavior in addition to acceptability attitudes. Their effects on engagement have not yet been demonstrated (Karyotaki et al., 2015).

The current study involves an examination of the factors influencing engagement, which we define as uptake and adherence, and acceptability of internet-based psychological programs from the two active conditions of the previously conducted Enhancing Engagement with Psychosocial Interventions (EEPI) Randomized Controlled Trial (Batterham et al., Submitted). The EEPI trial compared the delivery of an EFI and a self-guided online program with the delivery of an online program alone, and not able to demonstrate an effect for the EFI in increasing uptake or adherence to the online program for participants with mild to moderate symptoms of depression and/or anxiety (Batterham et al., Submitted). Participants in both active conditions of the trial had low engagement with the intervention, with only 37.8% accessing at least one module (uptake) and 22.3% completing at least one module (adherence). However, identifying the characteristics of those who find online programs more acceptable, and are more likely to use them, can lead to the development of strategies to improve engagement with these programs.

Aim

This study aimed to determine which factors were associated with intervention uptake and adherence in the context of a trial of a self-guided online program for depression and anxiety, and the acceptability of internet-based psychological programs among trial participants. Although the study was largely exploratory, we expected that lower engagement would be associated with by younger age, male gender and less severe symptoms.

Method

Trial design

The study was a randomized controlled trial with three conditions to test the independent efficacy of both an EFI and the self-guided online program myCompass 2. The conditions were (1) myCompass 2 + EFI, (2) myCompass 2 (alone), and (3) an attention control condition. The method is described below, with additional details provided previously in the main outcomes paper for the trial (Batterham et al., Submitted). Ethical approval for this research was provided by The Australian National University Human Research Ethics Committee (ANU HREC protocol number 2018/257) and the trial was prospectively registered (Australian New Zealand Clinical Trials Registry (ANZCTR) ACTRN12618001565235).

Procedure

Recruitment

We recruited participants from the general community through an advertisement on the social networks Facebook and Instagram during January to March 2019. Online programs are frequently marketed directly to the consumer, so we used social media to maintain the ecological validity of the trial and target a broad cross-section of the community. The paid advertisements included nature imagery (e.g., trees, waterfalls) and a tagline that asked: ““Want to learn more about your mental health? Complete a survey and 7 week online program now”. Community members were eligible for the trial if they 1) had not previously used the myCompass online program, 2) were not currently receiving psychological therapy, 3) had not made a suicide plan in the past month, 4) had not been diagnosed with psychosis or bipolar disorder, 5) were aged 18 years or over, 6) were currently living in Australia, and 7) reported current symptoms in the mild to moderate range (score 5–14) for either depression (PHQ-9) or anxiety (GAD-7). All measures are described below.

As outlined in the study protocol (Batterham et al., 2019), we required at least 231 participants per condition to meet power requirements for our primary hypotheses. We continued our recruitment beyond this goal to increase the lower numbers of male participants, which is common in online mental health trial research (Thornton et al., 2016), by using advertisements targeted to males. Participants who clicked on the advertisement were taken to an information page, which they read and then indicated their consent to participate if eligible. Participants who completed the screening measures and met the eligibility criteria above, were invited to provide an email address and password to participate in the trial. Those who were screened ineligible were directed to a page that thanked them for their interest and provided a list of relevant mental health resources. Any potential participants who endorsed the suicide screening item of the PHQ-9 were provided with a prompt that encouraged them to telephone Lifeline, the Suicide Call Back Service, or emergency services if at immediate risk of harm.

Blinding and randomization

Participants were randomized to one of three conditions by a computerized randomization algorithm embedded into the trial portal using the digital infrastructure of the Black Dog Institute, Sydney, Australia. The trial was double-blinded; participants were not told which condition was expected to be effective. Participants received information that they would be randomized to receive one of three online programs: (1) strategies for challenging unhelpful thoughts and behaviors (myCompass 2), (2) education about online interventions plus program (EFI + myCompass 2), or (3) general health and lifestyle information (attention control). Assessments were also blinded as they were self-report; the analysis was also blinded as the statistician performing the analyses was not aware of the condition identities when conducting the analyses.

Randomization was stratified on general psychological distress symptom severity (as measured by the DQ5 (Batterham et al., 2016) at pre-test score 5–13 vs. 14–25), age (18–45 vs. 46+ years), and gender (female/male) (permuted block randomization, block size of 6 within each stratum) to ensure balance across conditions. Prefer not to answer and ‘other’ were categorized with the group expected to be smaller (i.e., male gender, and age 56+ years) for stratification purposes. The portal also enabled us to collect online usage data automatically for all conditions allowing us to collect objective data on participants' uptake and usage (adherence) of their assigned program. Participants assigned to the myCompass 2 conditions (Conditions 1 and 2), used the same username and password to log into the myCompass 2 program when automatically redirected to it.

Data collection

Using the email and password they selected, participants were able to access their assigned program at any time for as long as they wished over a 7-week period. Automatic weekly reminder e-mails encouraged participants to engage with their assigned intervention (myCompass 2/attention control website). They were also sent emails to complete the post-test (7-week) and follow-up (6-month) surveys. There were no incentives for completing the baseline survey that forms the data for the current study. However, participants in all conditions received small incentives (online (e-) gift cards) for completing the post ($15AUD) and follow-up ($25AUD) surveys only. The receipt of an incentive was not contingent on engagement with the interventions being trialed.

Participants

Participants for the current study were those who were randomized to the two intervention conditions (N = 565), which were 1) EFI + myCompass 2 program, or the 2) myCompass 2 program alone and had complete data on the variables we included for analysis (N = 556). Participants who received the control condition (n = 287) were excluded from the current analyses.

Interventions

The engagement-facilitation intervention (EFI)

The EFI was a brief series of click-through linear webpages that participants in the EFI + myCompass 2 condition viewed prior to commencing the myCompass 2 program. It comprised approximately 5 min of written and audio-visual content including tailored feedback about the participant's symptom levels for depression and anxiety (based on their PHQ-9 and GAD-7 scores at baseline), a written description of the benefits of participating in online programs, information about the efficacy of the online mental health program, its content, and the time commitment involved, and its data security, and finally two testimonials presented in a 1 min video outlining the benefits of online mental health programs to provide information and normalize online self-guided therapy interventions. The content was informed by theory that explores the role of social norms in the acceptability of online mental health programs (Ajzen, 1991; Ebert et al., 2015; Venkatesh et al., 2003). A paper describing the creation of the EFI using participatory design (Gulliver et al., 2020) and the main outcomes paper (Batterham et al., Submitted) provide further information on the EFI and its development.

2.3.2. myCompass 2.

The myCompass 2 program is a free, private, accessible on a variety of platforms (mobiles, tablets, computers), fully automated and interactive self-guided (without therapist assistance) program for people with mild to moderate symptoms of stress, anxiety and/or depression. The original myCompass self-guided program has been shown to reduce symptoms of depression and anxiety in previous community studies (n = 89; n = 720) (Clarke et al., 2016; Proudfoot et al., 2013). The program we used for the current study was an updated version of myCompass, we named myCompass 2 (see https://www.mycompass.org.au/).

Unexpectedly, in our trial that formed the basis of the current study (Batterham et al., Submitted), we found both low uptake (37.8% accessed at least one module) and adherence (22.3% completed at least one module) to the myCompass 2 program. We proposed several reasons for this (Batterham et al., Submitted), including a practical issue that meant participants could not be automatically logged in to the program, which likely led to lower uptake – over 80% of active condition participants clicked a link to the program but more than half of these failed to log in manually. It is also possible that adherence may have been negatively impacted by a poor fit between the program and the needs of participants, for example, the characteristics of the sample (mild-moderate symptoms), and/or the fully online nature of the trial (with a lack of direct human support). We also did not find efficacy for improving depression or anxiety symptoms for the myCompass 2 program. The reason for this null finding was unclear, as the program is highly similar to the original myCompass program (Proudfoot et al., 2013), although it is possible that the low adherence could have impacted on our ability to find evidence for the efficacy of the program in this study (Batterham et al., Submitted). The myCompass 2 program was designed to be tailored, by screening the user and then directing them to the content that is most appropriate. The clinical approach of the original and the updated program is identical; however, the myCompass 2 program was upgraded from the original myCompass in the following key areas: 1) faster and easier sign-up for users, 2) more comprehensive user dashboard with more options for personalization, 3) clearly displayed and easier to navigate modules, and finally, 4) more comprehensive symptom trackers that are easier for the user to personalize.

The myCompass 2 program contains 14 modules (30–45 min each) delivered over 7 weeks. The module content is based on CBT, problem solving therapy, interpersonal psychotherapy, and positive psychology. The modules provide both core transdiagnostic CBT (7 modules), and content targeting specific concerns related to mental health (7 modules, e.g., sleep). Interactive quizzes provide real-time self-monitoring of thoughts, feelings and behaviors, and the program also provides reminders, feedback, facts, and mental health-care tips or motivational statements delivered via SMS or email depending on user preference.

Measures

Demographic information

The following demographic characteristics were assessed and included in the current study: gender (male/female/other), age (18–35/36–55/56+ years), level of education (high school or less, certificate/diploma, bachelor's degree, postgraduate degree/diploma). Other demographic information was assessed (e.g., employment status, region) see (Batterham et al., Submitted)), but are not included here as they were not expected to impact on the outcome variables in the current study analysis.

Diagnosis and treatment

Two items measured if the participant had ever in their life 1) had a mental disorder diagnosis, and 2) had psychological treatment. The first question was “Have you ever in your life been diagnosed with any of the following disorders” (Please select yes for all that apply) – followed by a list of disorders (e.g., major depression, generalized anxiety disorder) (yes/no/prefer not to answer) and an open-ended box for any other disorders not listed. The second question was “Have you ever in your life participated in psychological treatment for any psychological problem (e.g., with a psychologist, psychiatrist, counsellor, mental health nurse, general practitioner etc.)?” (yes/no/prefer not to answer).

Suicidal ideation

A single item from the Psychiatric Symptom Frequency Scale (PSF (Lindelow et al., 1997)) was used to measure suicidal ideation. The item asked if the respondent had “thought about taking your own life?” during the past 6 months (yes/no). The full PSF displays good internal reliability and validity (Lindelow et al., 1997).

Depression symptoms

We used the PHQ-9 (Spitzer et al., 1999) to screen for inclusion in to the trial and to assess depression symptoms. The PHQ-9 comprises nine items measuring the frequency of DSM-IV symptoms of Major Depression in the previous 2 weeks. Respondents rate items on a 4-point scale (0 = not at all to 3 = nearly every day). Scores on each item are summed, producing an overall severity score (potential range = 0–27), with higher scores indicating higher depression symptom severity. The PHQ-9 demonstrates good sensitivity and specificity for detecting major depression in clinical and general population samples (Kroenke et al., 2010). Internal reliability using Cronbach's alpha was 0.42 (n = 556) at baseline. However, in the current study only participants with symptom scores of 5–14 for both PHQ-9 and GAD-7 were included and alpha can be unreliable when participants' scores fall within a restricted range (Fife et al., 2012). The alpha for the PHQ-9 at post-test was 0.86 (n = 158).

Anxiety symptoms

The GAD-7 was used to measure anxiety symptoms. It consists of seven items corresponding to DSM-IV criteria for generalized anxiety disorder (GAD) (Spitzer et al., 2006). Similar to the PHQ-9, respondents rate items on a 4-point scale (0 = not at all to 3 = nearly every day); scores for each item are summed to produce a total score (range 0–21), with higher scores indicating greater symptom severity. The GAD-7 has good psychometric properties in general population samples and is highly accurate compared with clinical diagnosis (Kroenke et al., 2010; Lowe et al., 2008). At baseline, the current study alpha was 0.76 (n = 556), although as above, this may not be a reliable estimate. Alpha was 0.89 at post-test (n = 158).

General psychological distress

The Distress Questionnaire-5 [DQ5, (Batterham et al., 2016)] was used to assess general psychological distress, and to stratify participants at randomization using a case-finding cut-point of ≥14 (lower distress = 5–13 vs higher distress = 14–25) (Batterham et al., 2016). This measure comprises five items, that assess the frequency of a range of distressing situations, thoughts, and feelings over the past 30 days on a 5-point Likert-type scale (1 = Never, 2 = Rarely, 3 = Sometimes, 4 = Often, 5 = Always). Item scores are summed, total scores range from 5 to 25, with higher scores indicating higher severity of general psychological distress. The DQ5 has shown high internal consistency and external validity in previous studies (Batterham et al., 2016; Batterham et al., 2018). Despite the restricted range on depression and anxiety symptoms, the DQ5 alpha was 0.73 in the current study sample at baseline.

Acceptability of internet-based psychological programs

Participant's views on the acceptability of internet-based programs was measured using items from Ebert et al. (2015), based on the Unified Theory of Acceptance and Use of Technology (UTAUT). Each item is rated on a 5-point scale ranging from totally disagree (1) to totally agree (5) and then summed to produce a total score (range 4–20); higher scores indicate higher acceptance of internet-based programs. Participants were provided with a description - “An internet-based intervention involves self-help psychological therapy to treat or prevent mental health symptoms delivered through the internet or using an app” and were then asked to respond to the items e.g., “If I was suffering from psychological strain such as enduring lowered mood, loss of interest and lowered energy, sleeping problems, rumination, loss of joy in life…I could imagine trying out an Internet-based intervention for mental health problems”. Total scores range from 4 to 20, and the scale has acceptable internal consistency (Ebert et al., 2015). The current study alpha was 0.81 at baseline.

Days out of role

Days out of role (full and partial) were measured by two items; the first assessing number of days out of role: “In the last 30 days, how many days were you totally unable to work, study, or manage your day-to-day activities because of emotional problems (such as feeling depressed or anxious)?” and the second assessing partial days out of role “Aside from those days, in the last 30 days, how many days were you able to work, study, or manage your day-to-day activities but had to cut back on what you did or did not get as much done as usual because of emotional problems?”

Depression stigma

The Depression Stigma Scale [DSS (Griffiths et al., 2006)] was used to assess stigmatizing attitudes towards depression. Respondents are asked to indicate their level of agreement with each of nine statements about depression using a 5-point Likert scale (strongly disagree = 0, strongly agree = 4). Scores are summed, with total scale scores ranging from 0 to 36, and higher scores indicate higher levels of personal stigma about depression. Example items include “People with depression could snap out of it if they wanted” and “Depression is not a real medical illness”. The DSS has demonstrated good internal consistency and validity (Griffiths et al., 2006). The current study alpha for the DSS at baseline was 0.77.

Help-seeking attitudes

Attitudes to seeking professional psychological treatment were assessed using five items with updated language and wording (Calear et al., 2014) from the short form of the Attitudes Towards Seeking Professional Psychological Help scale [ATSPPH-SF (Fischer and Farina, 1995)]. Reverse-scored items on this scale did not load very well on a single factor; thus, these items from the original scale were removed, with the remaining five items retained (items 1, 3, 5, 6, 7). Each of the five items asks respondents to rate their view on statements about psychological treatment using a 4-point Likert scale (Disagree = 0, Agree = 3). Scores on the 5-item abbreviated scale range from 0 to 15, with higher scores indicating more positive attitudes towards seeking professional help. An example item includes “If I was having personal or emotional problems, the first thing I would do is seek professional help”. The original 10-item scale has previously shown sound psychometric properties (Fischer and Farina, 1995; Elhai et al., 2008; Calear et al., 2014). The current study alpha for the 5-item scale was 0.72 at baseline.

Personality

Personality characteristics were measured using the Mini-IPIP, which is a short (20-item) version of the larger International Personality Item Pool [IPIP, (Donnellan et al., 2006)]. The Mini-IPIP scale comprises 4 items each assessing one of the Big Five personality traits: Extraversion (example item: “I am the life of the party”), Agreeableness (e.g., “I sympathize with others' feelings”), Conscientiousness (e.g., “I get chores done right away”), Neuroticism (e.g., “I have frequent mood swings”), and Intellect/Imagination/Openness (e.g., “I have a vivid imagination”). Respondents are asked to rate how well each item describes them personally on a 5-point scale ranging from very inaccurate (1) to very accurate (5). Scores for each item are summed to produce a total score ranging from 0 to 20 on each sub-scale. The Mini-IPIP has demonstrated good reliability and validity (Donnellan et al., 2006). Acceptable alpha for the Mini-IPIP is lower than for the full scale at approximately 0.60 or above (Donnellan et al., 2006). In the current study, the alpha scores were as follows: Extraversion (0.79), Agreeableness (0.74), Conscientiousness (0.66), Neuroticism (0.59), and Intellect/Imagination/Openness (0.75).

Analyses

Whilst there are different ways of measuring the concepts of uptake, adherence, and acceptability of internet interventions, in this study acceptability of internet-based psychological programs was measured using the UTAUT, uptake was defined as accessing at least one therapeutic module of the program (0 modules/1–14 modules), and adherence was assessed by using the number of modules completed of myCompass 2 during the 7-week intervention period (0 modules/1 module/2–14 modules). We selected module completion as it captured the dosage of therapeutic content received.

Three separate analyses were conducted to identify significant predictors of uptake (accessing at least one therapeutic module of the program), adherence (modules completed), and acceptability of internet-based psychological programs (UTAUT). Data were analyzed using SPSS 26.0 for Windows (IBM Corp). Logistic regression using adjusted models were used to identify significant predictors of uptake. Multinomial logistic regression was used to assess significant predictors of adherence. Finally, linear regression assessed the predictors of acceptability of internet-based psychological programs.

Results

Participants

Table 1 presents the demographic and baseline characteristics (independent variables), and the uptake and adherence data (dependent variables) of the current study sample. Acceptability of internet-based psychological programs was both an independent and dependent variable. For the full trial a total of 858 participants were eligible, randomized to one of three conditions, and completed the baseline measures: EFI + myCompass 2 (n = 284), myCompass 2 alone (n = 287), and Attention Control (n = 287). Six participants in the intervention conditions withdrew from the study (EFI + myCompass 2 = 4, myCompass 2 alone = 2), and a further nine participants (EFI + myCompass 2 = 4, myCompass 2 alone = 5) were excluded from the analysis as they did not have complete data on the included variables. After excluding control participants, a final sample of 556 participants was analyzed for the current study.

Table 1

Baseline characteristics of participants included in the study (N = 556).⁎

	EFI + myCompass 2	myCompass 2 alone	Total
Independent variables	(n = 276)	(n = 280)	(N = 556)
Age category (years), n (%)
18–35	84 (30.4)	72 (25.7)	156 (28.1)
36–55	149 (54.0)	163 (58.2)	312 (56.1)
56+	43 (15.6)	45 (16.1)	88 (15.8)
Gender, n (%)
Male	56 (20.3)	61 (21.8)	117 (21.0)
Female	216 (78.3)	217 (77.5)	433 (77.9)
Other	4 (1.4)	2 (0.7)	6 (1.1)
Highest level of education, n (%)
High school or less	52 (18.8)	49 (17.5)	101 (18.2)
Certificate/diploma	108 (39.1)	92 (32.9)	200 (36.0)
Bachelor's degree	66 (23.9)	67 (23.9)	133 (23.9)
Postgraduate degree/diploma	50 (18.1)	72 (25.7)	122 (21.9)
Diagnosis and treatment n (%)
Any diagnosed mental disorder	208 (75.4)	213 (76.1)	421 (75.7)
Any psychological treatment	226 (81.9)	221 (78.9)	447 (80.4)
Suicidal ideation n (%)
Had thought about taking own life in past 6 months	65 (23.6)	51 (18.2)	116 (20.9)
Symptom measures, M (SD)
Anxiety (GAD-7)	7.61 (3.41)	7.30 (3.25)	7.45 (3.33)
Depression (PHQ-9)	9.78 (2.76)	9.63 (2.70)	9.70 (2.73)
General psychological distress (DQ5)	14.49 (2.98)	14.42 (3.04)	14.45 (3.01)
Days out of role	3.03 (4.51)*	4.19 (5.96)*	3.62 (5.32)
Partial days out of role	9.09 (8.35)	9.36 (8.51)	9.22 (8.43)
Attitudes, M (SD)
Depression stigma (DSS)	10.79 (8.12)	10.67 (7.73)	10.74 (7.92)
Help-seeking (ATSPPH-SF)	10.70 (2.80)	10.93 (2.58)	10.81 (2.69)
Personality, M (SD)
Extraversion (Mini-IPIP)	10.18 (3.88)	9.87 (3.64)	10.02 (3.77)
Agreeableness (Mini-IPIP)	16.36 (3.04)	16.39 (3.03)	16.37 (3.03)
Conscientiousness (Mini-IPIP)	12.27 (3.27)	12.31 (3.46)	12.29 (3.36)
Neuroticism (Mini-IPIP)	14.26 (2.72)	14.24 (2.87)	14.25 (2.79)
Intellect/Imagination/Openness (Mini-IPIP)	15.28 (3.28)	15.08 (3.46)	15.18 (3.37)
Dependent variables
Uptake (modules started), n (%)
0 modules	171 (62.0)	175 (62.5)	346 (62.2)
1–14 modules	105 (38.0)	105 (37.5)	210 (37.8)
Adherence (modules completed), n (%)
0 modules	212 (76.8)	219 (78.2)	431 (77.5)
1 module	24 (8.7)	24 (8.6)	48 (8.6)
2–14 modules	40 (14.5)	37 (13.2)	77 (13.8)
Acceptability of internet programs (UTAUT), M (SD)	14.50 (3.17)	14.41 (3.02)	14.46 (3.09)

= p < .05.

Chi-square and t-tests demonstrated that there were no significant differences between the groups at baseline, except for a single measure of days out of role. Due to several outliers reporting 30/30 days, and the ordinal nature of the days out of role measures, we also assessed these using a Mann Whitney U test, which showed significant differences between groups for days out of role, z = −2.36, p = .02, but not for partial days out of role, z = −35, p = .73.

Predictors of uptake

Table 2 presents the logistic regression analysis, which indicated that uptake (accessing at least one module) was significantly related to conscientiousness and acceptability of internet-based psychological programs, with those with higher levels of conscientiousness, and higher acceptability of internet-based psychological programs demonstrated increased odds of between 6.4 and 9.0% of accessing at least one module of the program. There were no other significant factors associated with intervention uptake in the model.

Table 2

Logistic regression model for predictors of uptake (accessing at least one module (N = 556).

	S.E.	Wald χ2	OR	CI	p
Intercept	1.019	11.55	0.031		0.001
Gender (Female)	0.240	0.524	1.190	(0.743–1.906)	0.469
Age		1.220			0.543
36–55 years	0.206	1.098	0.806	(0.538–1.207)	0.295
56+ years	0.289	0.654	0.791	(0.449–1.395)	0.419
18–35 years
Education		1.184			0.757
Certificate / diploma	0.261	0.030	0.956	(0.574–1.594)	0.864
Bachelor's degree	0.280	0.000	1.006	(0.581–1.741)	0.982
Higher degree	0.294	0.763	0.773	(0.434–1.377)	0.383
High school or less
Extraversion	0.025	0.015	1.003	(0.955–1.053)	0.903
Agreeableness	0.032	0.264	1.017	(0.955–1.083)	0.608
Conscientiousness	0.028	4.959	1.064	(1.007–1.123)	0.026*
Neuroticism	0.035	3.499	1.067	(0.997–1.142)	0.061
Intellect/Imagination/ Openness	0.028	0.213	0.987	(0.935–1.043)	0.645
Acceptability	0.031	7.893	1.090	(1.026–1.158)	0.005*
Condition (myCompass 2 alone)	0.180	0.002	0.992	(0.697–1.411)	0.963

Note: *p < .05; Ref = Reference category.

Predictors of adherence

The multinomial logistic regression presented in Table 3, demonstrated that adherence (modules completed) was significantly related to acceptability of internet-based psychological programs, with those who had higher acceptability scores demonstrating decreased odds (12%) of not completing any modules in comparison to those who completed 1 module. There were no other significant factors associated with intervention adherence in the model, including no factors associated with completing more than one module.

Table 3

Multinomial logistic regression model for predictors of adherence (modules completed) (N = 556).

	S.E.	Wald χ2	OR	CI	p
0 Modules vs. 1 Module
Intercept	2.21	5.383			0.020
Gender (Female)	0.517	1.807	0.499	(0.181–1.375)	0.179
Age
36–55 years	0.339	2.891	1.778	(0.916–3.454)	0.089
56+ years	0.598	2.974	2.807	(0.869–9.070)	0.085
18–35 years	(Ref)
Education
Higher degree	0.558	1.011	1.752	(0.587–5.227)	0.315
Bachelor's degree	0.462	0.383	0.751	(0.304–1.857)	0.536
Certificate / diploma	0.466	0.605	1.437	(0.576–3.580)	0.437
High school or less	(Ref)
Anxiety	0.064	0.096	1.020	(0.900–1.157)	0.757
Depression	0.068	0.249	1.035	(0.905–1.182)	0.618
General psychological distress	0.071	0.447	1.048	(0.913–1.205)	0.504
Help-seeking	0.068	0.019	1.009	(0.883–1.154)	0.891
Stigma	0.021	0.012	0.998	(0.957–1.040)	0.913
Extraversion	0.045	0.395	0.972	(0.891–1.061)	0.530
Agreeableness	0.062	0.466	0.959	(0.849–1.082)	0.495
Conscientiousness	0.049	0.240	1.024	(0.931–1.127)	0.624
Neuroticism	0.069	2.583	0.895	(0.781–1.025)	0.108
Intellect/Imagination/ Openness	0.049	0.019	1.007	(0.915–1.108)	0.891
Acceptability	0.059	4.915	0.877	(0.781–0.985)	0.027*
Condition (myCompass 2 alone)	0.316	0.013	0.965	(0.519–1.792)	0.910
2–14 Modules vs. 1 Module
Intercept	2.651	0.426			0.514
Gender (Female)	0.609	0.030	0.900	(0.273–2.967)	0.862
Age
36–55 years	0.418	2.110	1.836	(0.809–4.166)	0.146
56+ years	0.706	1.518	2.387	(0.598–9.521)	0.218
18–35 years	(Ref)
Education
Higher degree	0.650	0.778	1.773	(0.496–6.336)	0.378
Bachelor's degree	0.570	0.326	0.722	(0.237–2.207)	0.568
Certificate / diploma	0.571	0.006	0.958	(0.313–2.934)	0.940
High school or less	(Ref)
Anxiety	0.076	1.448	1.096	(0.944–1.271)	0.229
Depression	0.081	0.133	1.030	(0.879–1.206)	0.715
General psychological distress	0.086	0.398	0.947	(0.801–1.120)	0.528
Help-seeking	0.081	0.366	0.952	(0.813–1.115)	0.545
Stigma	0.025	1.532	1.032	(0.982–1.084)	0.216
Extraversion	0.055	3.510	0.902	(0.810–1.005)	0.061
Agreeableness	0.073	0.578	0.946	(0.821–1.091)	0.447
Conscientiousness	0.059	0.873	1.057	(0.941–1.187)	0.350
Neuroticism	0.082	1.179	0.915	(0.778–1.075)	0.277
Intellect/Imagination/ Openness	0.060	0.746	1.053	(0.936–1.184)	0.388
Acceptability	0.071	0.040	1.014	(0.882–1.167)	0.841
Condition (myCompass 2 alone)	0.379	0.146	0.865	(0.411–1.820)	0.703

Note: Model reference category = 1 module completed; *p < .05; Ref = Reference category.

Predictors of acceptability

Finally, Table 4 presents the results of the linear regression, which showed that acceptability of internet-based psychological programs was significantly related to help-seeking attitudes and agreeableness. Those with more positive attitudes towards help-seeking, lower levels of stigma, or higher levels of agreeableness had higher levels of acceptability of internet-based psychological programs. Help-seeking attitudes was the strongest predictor, with the model indicating that after controlling for all other factors, a 1 unit increase in help-seeking attitudes will result in an 0.24 increase in acceptability of internet-based programs on each scale.

Table 4

Linear regression models for acceptability of internet-based psychological programs predicted by demographic data (N = 556).

	Estimate	SE	t	p
Intercept	8.219	1.702	4.829	<0.001
Gender (male)	−0.523	0.344	−1.519	0.129
Gender (other)	−0.484	1.253	−0.386	0.700
Age
36–55 years	0.048	0.304	0.157	0.875
56+ years	−0.546	0.418	−1.306	0.192
Education
Certificate / diploma	0.641	0.373	1.720	0.086
Bachelor's degree	0.410	0.406	1.008	0.314
Higher degree	0.272	0.422	0.645	0.519
Anxiety	−0.065	0.050	−1.289	0.198
Depression	0.057	0.056	1.021	0.308
General psychological distress	0.053	0.061	0.868	0.386
Help-seeking	0.238	0.051	4.662	<0.001**
Stigma	−0.039	0.017	−2.277	0.023*
Suicide ideation (past 6 months)	−0.483	0.326	−1.484	0.138
Any diagnosed mental disorder	−0.357	0.362	−0.985	0.325
Any psychological treatment	−0.759	0.387	−1.963	0.050
Days out of role	0.024	0.027	0.896	0.371
Partial days out of role	0.009	0.016	0.547	0.585
Extraversion	−0.017	0.036	−0.482	0.630
Agreeableness	0.093	0.046	2.045	0.041*
Conscientiousness	0.052	0.040	1.305	0.192
Neuroticism	0.057	0.053	1.075	0.283
Intellect/Imagination/ Openness	0.068	0.039	1.711	0.088

Note: * = p < .05, ** = p < .001; Ref = Reference category.

Discussion

The current study sought to investigate factors influencing engagement and acceptability of internet-based psychological programs. We found small but significant relationships for predicting uptake, adherence and acceptability of internet-based psychological programs. Acceptability predicted both uptake and adherence to the myCompass 2 program. The personality trait of conscientiousness also predicted uptake, and positive help-seeking attitudes, lower stigma, and the trait of agreeableness predicted higher acceptability of internet-based psychological programs.

The findings for uptake indicated that greater baseline levels of acceptability of internet-based psychological programs predicted uptake of the myCompass 2 program. This finding supports the importance of the role of acceptability of online programs in increasing their uptake (Ebert et al., 2015). This provides a critical opportunity for the provision of advertising or acceptance-facilitation programs that can educate people on the benefits of internet-based psychological programs and potentially improve attitudes towards them, which may increase uptake in the community. Our randomized controlled trial (Batterham et al., Submitted) was not able to improve the acceptability of online programs with a brief 5-min educational EFI, thus more intensive programs or targeting of social norms (Venkatesh et al., 2003) more broadly may be required for meaningful attitudinal change. The effects of EFIs may also be intervention-dependent, with fit between therapy programs and end users' expectations and preferences mediating the ability of EFIs to promote greater engagement (Kelders et al., 2012). Further studies examining potential mechanisms related to individual change in acceptability are needed.

Uptake was also associated with personality; those with higher levels of conscientiousness were more likely to access at least one module of the program. This finding is broadly consistent with the Cillessen et al. (2020) study of people living with cancer, which found that conscientiousness predicted uptake of an online mindfulness intervention. The personality trait of conscientiousness has also been found to be strongly related to a propensity to engage in many other health-related behaviors (Gerhards et al., 2011), such as healthy eating (Wilson et al., 2016), and cancer screening (Aschwanden et al., 2019).

Acceptability of internet-based psychological programs predicted greater adherence to the myCompass intervention. Those who had higher acceptability towards these programs were less likely to fail to complete a single module in comparison to those who completed 1 module. However, this relationship did not hold for those who had completed any further (2–14) modules. This may indicate that acceptability of internet-based interventions provides an opportunity to get a ‘foot in the door’ as those with higher acceptability seem more likely to attempt the program (i.e., engage in uptake of the intervention). However, unless the program is engaging and useful, people may not adhere and continue the program beyond that initial first step [“early dropout”; (Davis and Addis, 1999)]. Engaging, humanized, and personalized intervention content is thought to be critical for adherence to online programs (Beatty and Binnion, 2016). In contrast to previous studies on adherence in depression and anxiety interventions, we found no evidence for many of the previously identified predictive factors for adherence including baseline symptom levels, gender, age, or education (Christensen et al., 2009; Farrer et al., 2014; Karyotaki et al., 2015; Beatty et al., 2017). Our lack of finding could have been due to differences in how the factors of interest were categorized in this study compared with previous research, and the choice to simultaneously adjust for multiple putative factors to account for possible confounding. As mentioned, differences between the interventions such as some requiring a higher level of knowledge of technology or education to self-complete an intervention (Batterham et al., 2008) may also have contributed.

Positive help-seeking attitudes, lower levels of stigma, and the personality trait of agreeableness also predicted greater acceptability of internet-based psychological programs. Previous research has shown that personality traits such as agreeableness (Park et al., 2018) and conscientiousness (Schomerus et al., 2013) may also influence help-seeking attitudes. Personality traits may need to be considered when developing strategies to promote more positive attitudes towards internet-based psychological interventions, or in tailoring content to the needs and preferences of users. Given that attitudes are modifiable, and higher acceptability of programs is likely to lead to greater uptake, improving help-seeking and reducing stigmatizing attitudes in the community is critical. Nonetheless, targeting only acceptance of internet-based programs may not lead to an improved uptake rate or adherence (Lin et al., 2018). More intensive programs may be required to change actual health behaviors.

Limitations

We included a large range of variables in the current study, which may have increased the chance of a type 1 error, although the study was adequately powered for these comparisons. In addition, the current study comprised a predominantly female sample recruited via a social media platform, which may have biased the sample. Although this issue is common in internet-based research, it limits the generalizability of the study to males and those with limited access to the internet. However, the trial was designed to reflect real world marketing of online programs, so is likely to reflect users outside of a trial setting. Although the sample was aligned with the target population of people with mild-moderate symptoms, it is possible that the limited focus on treatment in our recruitment strategy may have influenced the types of people who engaged in the study and their propensity to engage in the intervention. More comprehensive measures of some of the predictors may have revealed different relationships with engagement outcomes. In addition, there are other engagement outcomes, such as time in the program or number of activities completed, that were not examined due to lack of data or questions about the validity of these indicators. Finally, there is likely to be a range of other factors associated with engagement, including the design of the therapy intervention, along with a range of additional user characteristics such as treatment beliefs and perceived need.

Conclusions

Higher levels of acceptability of internet-based psychological programs were associated with increased uptake and adherence to the myCompass 2 program. The personality traits of conscientiousness and agreeableness also predicted uptake and acceptability of internet-based psychological programs, respectively. Higher levels of help-seeking attitudes also increased the likelihood of uptake.

Practical implications

Improving awareness and acceptability of internet-based psychosocial programs could translate to increased uptake and adherence to these programs in the community. Changing community perceptions about internet interventions may require drawing on diverse delivery pathways, such as direct-to-user marketing, government campaigns, online directories and search optimization, delivery in diverse clinical settings (doctors, psychologists, pharmacists), and delivery through schools and workplaces.