The effectiveness of unguided internet cognitive behavioural therapy for mixed anxiety and depression.

Clinician-guided internet-delivered cognitive behavioral therapy (iCBT) is an effective treatment for depression and anxiety disorders. However, few studies have examined the effectiveness of completely unguided iCBT. The current research investigated adherence to, and the effects of two brief unguided iCBT programs on depression and anxiety symptom severity, and psychological distress. Study 1 evaluated a four-lesson transdiagnostic iCBT program for anxiety and depression (N = 927). Study 2 then evaluated a three-lesson version of the same program (N = 5107) in order to determine whether reducing the duration of treatment would influence adherence and treatment effects. Cross-tabulations and independent t-tests were used to examine the extent to which users adhered and remitted with treatment. Linear mixed models were used to evaluate the effects of treatment in the entire sample, and stratified by gender and completer-type (e.g., users who completed some but not all lessons vs. those who completed all lessons of treatment). Among those who began treatment, 13.83% completed all four lessons in Study 1. Shortening the course to three lessons did not improve adherence (e.g., 13.11% in Study 2). In both studies, users, on average, experienced moderate to large effect size reductions in anxiety and depressive symptom severity, as well as psychological distress. This pattern of results was robust across gender and for those who did and did not complete treatment. Approximately two-thirds of those who completed treatment experienced remission. These data show that unguided iCBT programs, which have the capacity to attract large numbers of individuals with clinically significant symptoms of depression and anxiety, and psychological distress, can produce significant improvements in wellbeing.

Introduction

Depressive and anxiety disorders are highly prevalent, annually affecting between 10 and 20% of adults (Andrews et al., 2001, Kessler et al., 2005, McEvoy et al., 2011). Remarkably, less than half of these individuals seek help from a mental health professional (Andrews et al., 2001, Harris et al., 2015). Although cognitive behavior therapy (CBT) has been shown to be an effective treatment for depression and anxiety (Butler et al., 2006), many people have difficulty accessing face-to-face CBT. This is because of the direct and indirect costs of treatment, limited access to qualified therapists, and concerns about stigma (Anderson et al., 2016, Andrade et al., 2014). These barriers have led researchers to develop Internet-based CBT (iCBT). This treatment modality can be used to provide evidence-based treatments at a low cost to end-users who can then access care in a way that is convenient and private.

iCBT programs can be guided or unguided. Guided programs also referred to as “clinician-guided” or “guided self-help” involve support from a clinician or technician (e.g., the user receives emails or phone calls during the program), whereas unguided programs are delivered entirely by computer and are also referred to as “unguided self-help” or “pure self-help” (Furmark et al., 2009). Fully automated, unguided iCBT programs represent an attractive avenue for optimizing the benefits of iCBT because they are eminently scalable and can be rapidly disseminated (Leykin et al., 2014).

There is considerable evidence that supports the use of guided iCBT for treating depressive and anxiety disorders (Andrews et al., 2010, Othuis et al., 2016, Păsărelu et al., 2017). Indeed, iCBT results in similar outcomes as traditional face-to-face CBT (e.g., Andersson et al., 2014, Cuijpers et al., 2010, Reger and Gahm, 2009), and has been shown to be effective under randomized controlled trial conditions as well as in “real world” clinical practice (Hobbs et al., 2018, Hobbs et al., 2017, Mewton et al., 2012, Newby et al., 2013, Newby et al., 2017, Newby et al., 2014, Williams and Andrews, 2013).

Early meta-analytic studies suggest that guided iCBT programs attain better results than unguided iCBT programs (e.g., Andersson and Cuijpers, 2009, Richards and Richardson, 2012, Spek et al., 2007). However, recent efficacy studies have found that unguided iCBT programs can be just as beneficial as guided iCBT in treating depression and anxiety (e.g., Dear et al., 2014, Furmark et al., 2009, Titov et al., 2013). That said, there are few studies that have investigated the effectiveness of completely unguided iCBT. Many studies that have investigated unguided iCBT have either done so in the context of a randomized clinical trial, offered payment for participation, or included some form of human encouragement such as telephone calls, diagnostic interviews, moderated discussion forums, or personalised automated email reminders (e.g., Berger et al., 2011, Berger et al., 2017, Boettcher et al., 2012, Christensen et al., 2004a, Christensen et al., 2004b, Dear et al., 2014, Donker et al., 2013a, Donker et al., 2013b, Karyotaki et al., 2017, Lintvedt et al., 2013, Twomey et al., 2014). It is largely unclear whether the outcomes of these studies generalize to completely unguided self-help programs that are widely disseminated to the general population in uncontrolled settings.

While hundreds of mental health programs are available on the Internet and via mobile phone applications, the evidence base demonstrating the effectiveness of these programs in naturalistic settings is very limited, as is the literature regarding the effect of treatment length on adherence and effects (Donker et al., 2013a, Donker et al., 2013b). The limited number of studies that have evaluated fully-automated, publically available iCBT programs in naturalistic settings do however, suggest that these programs are feasible and can lead to significant reductions in depression and anxiety symptom severity but adherence is modest (~ 16%) (e.g., Christensen et al., 2006, Christensen et al., 2004a, Christensen et al., 2004b, Farvolden et al., 2005, Leykin et al., 2014). Nevertheless, fully-automated iCBT programs may represent the most scalable and cost-effective way to disseminate such interventions.

Given the scant data regarding the effectiveness of publically available iCBT self-help programs, the current study examined adherence to, and the effects of two freely available self-guided versions of a validated transdiagnostic program for symptoms of depression and anxiety (the Worry and Sadness program [now called the Mixed Depression and Anxiety program]) (Newby et al., 2013, Newby et al., 2014). In order to maximize the ecological validity of the study, no control group or exclusion criteria were utilized and no personal contact with users was provided. Specifically, Study 1 evaluated the effectiveness of a four lesson Worry and Sadness program and Study 2 evaluated a shorter three lesson version of the same program to investigate whether reducing the number of lessons would improve adherence. The two self-help programs were offered sequentially by the ThisWayUp.org.au service, and as such users did not nominate the length of their program. On the basis of the unguided self-help literature and previous evaluations of the guided Worry and Sadness course, we hypothesized that: (1) completion rates would be modest; (2) that the 3 three lesson program would yield greater adherence than the four lesson program; and (3) that users would experience significant effect size reductions in depression and anxiety symptom severity, and psychological distress.

Methods

Participants

927 users who enrolled themselves in the four-lesson Worry and Sadness iCBT program between 1st July 2012 and 8th April 2013 were included in Study 1. 5107 users who enrolled in the three-lesson Worry and Sadness iCBT program between 9th April 2013 and 21st July 2015 were included in Study 2.

Assessments

The assessment measures, procedures and analytical strategies applied in Study 1 were replicated in Study 2 and are therefore presented below in one Methods section.

Demographic information

Consistent with the anonymity of pure self-help iCBT programs, users were only required to report their gender.

Outcome measures

Users completed the Generalized Anxiety Disorder-7 (GAD-7) and the Patient Health Questionnaire (PHQ-9) at the first and last lesson of treatment, and the Kessler Psychological Distress Scale (K-10) at each lesson.

Depression and anxiety symptom severity

The PHQ-9 is a 9-item self-report screener for Major Depressive Disorder (MDD) (Kroenke et al., 2001) and the GAD-7 is a 7-item self-report screener for Generalized Anxiety Disorder (GAD) (Spitzer et al., 2006). On both measures, users reported how often they had experienced symptoms in the past two weeks as either “not at all”, “on several days”, “on more than half the days” or “on nearly every day”. Items were scored from 0 to 3, with a total score of 10 (or more) indicative of a probable MDD and/or GAD diagnosis respectively. The PHQ-9 has excellent reliability and validity (Kroenke et al., 2001). The internal consistency of the PHQ-9 prior to treatment was α = 0.86 in both Study 1 and Study 2. Evidence of the reliability and validity of the GAD-7 has been demonstrated (Löwe et al., 2008, Spitzer et al., 2006). The internal consistency of the GAD-7 prior to treatment was α = 0.87 in Study 1 and 0.86 in Study 2.

Distress

The K-10 is a 10-item self-report measure that indexes non-specific psychological distress experienced in the past two weeks (Kessler et al., 2002). Users reported how frequently they experienced each item as either “none”, “a little”, “some” or “all” of the time. Each item was scored from 1 to 5 with total scores ranging from 10 to 50. The K-10 can be used as a valid predictor of common mental disorders, with scores of 20 or more indicative of clinical levels of distress (Andrews and Slade, 2001, Kessler et al., 2002). The K-10 has strong psychometric properties, including excellent internal reliability and validity (Furukawa et al., 2003, Sunderland et al., 2012). The internal consistency of the K-10 prior to treatment in Study 1 was α = 0.89 and 0.88 in Study 2.

Procedure and intervention

In general, users registered with ThisWayUp (https://thiswayup.org.au) by providing an email address, specifying their gender, creating their own username and password, and confirming that they were an Australian resident. All users provided their electronic informed consent that their pooled de-identified data could be collected, collated, analyzed and published for quality assurance purposes. Users then selected the Worry and Sadness program from a list of available treatments for depressive and anxiety disorders.

The two programs evaluated in this manuscript are shortened versions of the six-lesson Worry and Sadness guided iCBT program described by Newby et al. (2013). During the latter program, users follow an illustrated character that is gaining mastery over her depression and anxiety. The program includes: psychoeducation about depression and anxiety, cognitive restructuring, education about avoidance and safety behaviors, graded exposure and relapse prevention activities. This 6 lesson program has been shown to be effective using intention-to-treat analyses under randomized controlled trial conditions (N = 99; g range = 0.87–1.05; adherence = 89%)(Newby et al., 2013) and in routine clinical care (e.g., N = 175, g range = 0.98–1.09, adherence = 33%: Newby et al., 2013; and N = 707, g range = 0.51–1.20; adherence = 47.3%: Newby et al. (2014)).

The content of the two unguided programs is detailed in Table 1. The three lesson program was highly similar to the four-lesson program, but included additional information about rumination and worry, and omitted information about troubleshooting difficulties with safety behaviors and graded exposure.

Table 1

Lesson content of the unguided Worry and Sadness iCBT programs evaluated in Study 1 and Study 2.

Lesson Number	Content	Homework tasks
Study 1. 4 lesson program
1	Psychoeducation about anxiety and depression, controlled breathing, and physical exercise	Controlled breathing, physical exercise
2	Cognitive model, common thinking distortions, thought monitoring, activity planning, and pleasant event scheduling	Thought monitoring, activity planning, pleasant event scheduling
3	Thought challenging, shifting attention, structured problem solving, and hunt for positives	Thought challenging, hunt for positives, structured problem solving
4	Avoidance and safety behaviors, graded exposure, common pitfalls to exposure, relapse prevention and how to get more help	Facing fears with exposure hierarchies
Study 2. 3 lesson program
1	Psychoeducation about anxiety and depression, controlled breathing, and physical exercise	Controlled breathing, physical exercise
2	Cognitive model, common thinking distortions, rumination and worry, thought monitoring, activity planning, and pleasant event scheduling	Thought monitoring, activity planning, pleasant event scheduling
3	Thought challenging, shifting attention, structured problem solving, hunt for positives, graded exposure, relapse prevention and how to get more help	Thought challenging, hunt for positives, structured problem solving, facing fears with exposure hierarchies.

After registering for their respective program, users were required to complete each lesson before moving onto the subsequent lesson. A minimum of five days was required to elapse before users could access subsequent lessons. At the end of each lesson, a lesson summary that described practical homework exercises was available for download and users were encouraged to complete this homework before their next lesson. A lesson was considered to be complete once users had viewed all lesson slides and downloaded the lesson summary. Automatic emails were sent to users congratulating them once a lesson was completed. Users also received an automated email if their K-10 total score was greater or equal to 30. This email advised the user that their scores were severe and recommended that they seek help from a general practitioner, psychologist, or crisis support telephone service. Users were given 60 days to complete their respective program.

Analyses

Adherence

Users were categorized as either non-starters (e.g., those who enrolled but did not start their respective program), completers (e.g., users in Study 1 who completed 4 lessons and users in Study 2 who completed 3 lessons) or non-completers (e.g., users who completed 0–3 lessons in Study 1 or 0–2 lessons in Study 2). Regression analyses and t-tests were used to examine whether gender, pre-treatment symptom severity (GAD-7 and PHQ-9 total scores) and/or psychological distress (K-10 total scores) were associated with the extent to which users adhered to treatment.

Treatment effects

All analyses were conducted using SPSS version 23. Linear mixed models (MIXED) with a random intercept for users were conducted to examine the effects of treatment on users' depression and anxiety symptom severity, and their psychological distress. Measurement occasion was treated as a categorical variable. These intention-to-treat models account for the unbalanced nature of these data and yield more accurate estimates of effect compared to completer analyses (Salim et al., 2008, West et al., 2014). First, models were estimated separately for each outcome measure and the covariance structure that provided the closest model fit for the residual correlation matrix was identified. Based on model fit indices, the compound symmetry covariance structure was chosen for the GAD-7 and PHQ-9, and a toeplitz covariance structure for the K-10 in Study 1. In contrast, a first order autoregressive structure provided the closest model fit for the K-10 and an unstructured covariance matrix provided the closest fit for the GAD-7 and PHQ-9 total scores in Study 2. Three types of models were then estimated. First, treatment effects were calculated for the entire sample. Second, effects were stratified by gender. Thirdly, the relationship between treatment completion and distress was examined. The K-10 was selected for the latter analysis rather than the GAD-7 or the PHQ-9 because the K-10 was measured at each lesson and the GAD-7 and PHQ-9 measured at the first and last lesson of the respective program, which precluded such analyses. Within–group Hedges' g effect sizes were calculated using the estimated marginal means at pre- and post-treatment assessments correcting for the correlation between repeated measurements. Between-group effect sizes (e.g., male vs. female; completers vs. non-completers) were estimated by calculating the difference between the estimated marginal means of each group divided by the pooled standard deviation of the two groups at their post-treatment assessments. Effect sizes of ≤ 0.49, ≥ 0.50 and ≥ 0.80 were considered to be small, moderate and large respectively (Cohen, 1988).

Clinically meaningful change among completers

Clinically meaningful change was operationalized with respect to remission and reliable change. First, users who began treatment with a probable diagnosis of GAD and/or MDD and completed treatment with total GAD-7 and/or PHQ-9 total scores less than the standard threshold for a probable diagnosis of GAD and/or MDD (< 10) were classified as in remission. Second, reliable change indices (RCI) were calculated for the GAD-7 and PHQ-9 to determine the proportion of patients who evidenced reliable improvements with treatment (Jacobson and Truax, 1991). Test-retest reliability estimates of 0.83 for the GAD-7 (Spitzer et al., 2006) and 0.84 for the PHQ-9 (Kroenke et al., 2001), and the baseline standard deviation of each measure in Studies 1 and 2 were used in these computations (detailed further below in the respective Results section). Clinically meaningful change was limited to completers because the GAD-7 and PHQ-9 were administered before and after treatment.

Results

Study 1: four lesson Worry and Sadness program

Sample characteristics

Of the 927 users who enrolled in the four-lesson Worry and Sadness program, 14.99% did not start the program (i.e., n = 139 were categorized as non-starters). Gender was the only known characteristic of this group, 63.31% of which were females (n = 88). Gender was not associated with enrolling and commencing treatment compared to enrolling and not starting treatment (χ2 (1) = 0.72, p = 0.40; OR(95% CI) = 1.18(0.81–1.71)). Non-starters were excluded from all subsequent analyses.

The pre-treatment characteristics of users in Study 1 are shown in Table 2. Of the 788 who started the Program, females comprised 67.01% of the sample. In general, users enrolled in the four-lesson Worry and Sadness course with clinical levels of anxiety and depressive symptom severity, and psychological distress. Indeed, 81.85% met criteria for a probable diagnosis of GAD and/or for MDD, and 87.56% with clinically significant levels of psychological distress (i.e., K-10 total score ≥ 20).

Table 2

Pre-treatment characteristics and adherence rates of users who started their 4- or 3-lesson unguided Worry and Sadness iCBT program.

	Study 1: 4 lesson program (N = 788)		Study 2: 3 lesson program (4233)
	n	%	n	%
Pre-treatment characteristics
Sex
Female	528	67.01	3087	72.93
Male	260	32.99	1146	27.07
Probable diagnoses
Subthreshold GAD and MDD
Yes	143	18.15	711	16.80
No	645	81.85	3522	83.20
GAD only
Yes	137	17.39	415	9.80
No	651	82.61	3818	90.20
MDD only
Yes	78	9.90	624	14.74
No	710	90.10	3609	85.26
Comorbid GAD/MDD
Yes	430	54.57	2483	58.66
No	358	45.43	1750	41.34
Clinically significant distress
Yes	690	87.56	3793	89.61
No	98	12.44	440	10.39
Adherence
Completed 0 lessons	114	14.47	516	12.19
Completed 1 lesson	389	49.37	2517	59.46
Completed 2 lessons	105	13.32	645	15.24
Completed 3 lessons	71	9.01	555	13.11
Completed 4 lessons	109	13.83	–	–

Predictors of adherence

Of the 788 who started the program, 14.47% did not complete any of the lessons, 49.37% completed only one lesson, 13.32% completed a total of two lessons, 9.01% completed a total of three lessons and 13.83% completed all four lessons (see Table 2). Compared to those users who did not complete the program, completers did not differ with respect to their gender (χ2 (1) = 1.20, p = 0.27; OR(95% CI) = 1.26(0.83–1.93)), anxiety symptom severity (t(df) = 1.61(786), p = 0.12) or psychological distress (t(df) = 1.95 (786), p = 0.05). Completers were however, less depressed prior to the program than those who completed 0–3 lessons (t(df) = 2.15 (786), p < 0.05).

The effects of the four-lesson Worry and Sadness program are displayed in Table 3 for the entire sample and stratified by gender. Treatment produced, on average, large effect size reductions in depressive and anxiety symptom severity, and psychological distress in the entire sample. The effect of treatment on anxiety symptom severity (GAD-7: F(1, 196) = 2.22, p = 0.14) and psychological distress did not vary by gender (K10: F(3, 464) = 1.49, p = 0.22); however, females showed a small but superior effect size reduction in depression symptom severity compared to males (PHQ-9: F(1172) = 6.27, p < 0.05). Completers and non-completers did not vary with respect to reductions in their psychological distress across treatment (F(2, 378) = 0.56, p = 0.57).

Table 3

Treatment effects of the four lesson unguided Worry and Sadness iCBT program (N = 788).

	Pre-treatment		Post-treatment		Pre-post treatment within-group ES					Post between-group ES
	EMM	SD	EMM	SD	t	p-Value	r	g	(95% CI)	g	(95% CI)
GAD-7
Entire sample	11.55	5.28	6.55	4.42	11.90	p < 0.001	0.54	1.10	(0.89–1.30)
Male	11.30	4.56	6.93	4.27	6.61	p < 0.001	0.55	0.98	(0.63–1.32)
Female	11.79	4.63	6.17	4.30	10.86	p < 0.001	0.54	1.27	(1.00–1.53)	0.17	(− 0.21–0.56)
PHQ-9
Entire sample	13.56	6.46	7.98	5.18	11.48	p < 0.001	0.62	0.96	(0.76–1.17)
Male	13.34	5.58	8.97	5.01	5.70	p < 0.001	0.67	0.73	(0.39–1.07)
Female	13.79	5.66	6.99	5.03	11.34	p < 0.001	0.58	1.27	(1.00–1.53)	0.39	(0.00–0.78)
K10
Entire sample	28.70	8.25	21.85	5.85	12.64	p < 0.001	0.67	0.99	(0.78–1.19)
Male	28.44	7.12	22.61	5.64	6.86	p < 0.001	0.82	0.65	(0.31–0.99)
Female	28.97	7.25	21.09	5.71	11.70	p < 0.001	0.59	1.29	(1.03–1.56)	0.26	(− 0.12–0.65)

Note. EMM = estimated marginal means were calculated using intention-to-treat mixed models.

Among the 109 users who completed the program, 67.19% of those who began treatment with a probable diagnosis of GAD (e.g., n = 43/64) and 62.31% who began treatment with a probable MDD diagnosis (e.g., n = 43/69) remitted with treatment. Based on the standard deviations of the estimated marginal means (e.g., 5.28 for the GAD-7 and 6.46 for the PHQ-9), users whose pre-treatment GAD-7 total score varied by at least 6.03 points and/or pre-treatment PHQ-9 total score varied by at least 6.25 with treatment were classified as either experiencing reliable improvement or deterioration with 95% confidence (Jacobson and Truax, 1991). Based on these estimates, 28.44% experienced reliable improvements in their anxiety symptom severity (n = 31/109), 34.86% experienced reliable change in their depression symptom severity (n = 38/109), and 19.27% evidenced reliable change in both their anxiety and depression symptom severity (n = 21/109). One user experienced reliable deterioration in symptom severity (i.e., GAD-7: n = 0; PHQ-9: n = 1). No user experienced reliable deterioration with respect to both their anxiety and depression symptom severity.

Study 2: three-lesson Worry and Sadness program

Of the 5107 users who enrolled in the three-lesson Worry and Sadness program, 17.11% did not start the Program (i.e., n = 874 were classified as non-starters). Gender was the only known characteristic of this group, 70.14% of which were females (n = 613). Gender was not associated with enrolling and commencing treatment compared to enrolling and not starting the Program (χ2 (1) = 2.79, p = 0.10; OR(95%CI) = 1.15(0.98–1.35)). Non-starters were excluded from all subsequent analyses.

The pre-treatment characteristics of users in Study 2 are shown in Table 2. Of the 4233 users who started the Program, females comprised 72.93% of the sample. In general, users enrolled in the three-lesson Worry and Sadness course with clinical levels of anxiety and depressive symptom severity, and psychological distress. Indeed, 83.20% met criteria for a probable diagnosis of GAD and/or MDD, and 89.61% with clinically significant levels of psychological distress (i.e., K-10 total score ≥ 20).

Of those 4233 who started the program, 12.19% did not complete any of the lessons, 59.46% completed only one lesson, 15.24% completed a total of two lessons, and 13.11% completed all three lessons (see Table 2). Compared to those who completed all 3 lessons of the program, non-completers (e.g., completed 0–2 lessons) were more anxious (t(df) = 3.82(4231), p < 0.001), depressed (t(df) = 6.24(4231), p < 0.001), and distressed (t(df) = 7.00(4231), p < 0.001) prior to treatment. Completers were also more likely to be male compared to non-completers (χ2(1) = 4.85, p < 0.05;OR(95% CI) = 1.24(1.03–1.51)).

The effects of the three-lesson Worry and Sadness course are displayed in Table 4 for the entire sample and stratified by gender. Treatment was associated with, on average, large effect size reductions for all outcome measures in the entire sample. Gender differences in treatment effects were however identified, with females experiencing significantly larger reductions in anxiety symptom severity (F(1, 609) = 6.28, p < 0.05) and psychological distress (F(1, 808) = 6.44, p < 0.01) compared to males. There was also a trend towards females experiencing greater effect size reductions in depression symptom severity (F(1, 595) = 3.77, p = 0.05). Although gender was associated with some statistical differences, these effects are unlikely to be clinically meaningful, given the small magnitude of the between group differences (Table 4). Completers did not differ from non-completers with respect to their reductions in psychological distress (F(1, 3850) = 0.15, p = 0.70).

Table 4

Effects of the three lesson unguided Worry and Sadness iCBT Program (N = 4233).

	Pre-treatment		Post-treatment		Pre-post treatment within-group ES					Post-treatment between-group ES
	EMM	SD	EMM	SD	t	p-Value	r	g	(95% CI)	g	(95% CI)
GAD-7
Entire sample	11.96	5.47	7.55	4.45	23.19	p < 0.001	0.49	0.99	(0.90–1.08)
Male	11.55	4.84	7.62	4.12	12.44	p < 0.001	0.49	0.96	(0.80–1.13)
Female	12.37	4.83	7.49	4.11	23.14	p < 0.001	0.50	1.19	(1.08–1.30)	0.03	(− 0.14–0.21)
PHQ-9
Entire sample	13.59	6.70	8.63	4.88	24.69	p < 0.001	0.62	0.91	(0.82–1.00)
Male	13.10	5.96	8.52	4.51	13.64	p < 0.001	0.67	0.85	(0.69–1.02)
Female	14.08	5.95	8.73	4.46	24.00	p < 0.001	0.60	1.10	(0.99–1.20)	− 0.05	(− 0.23–0.13)
K10
Entire sample	28.94	8.52	23.10	6.08	23.46	p < 0.001	0.62	0.87	(0.78–0.96)
Male	28.29	7.58	23.29	5.64	12.07	p < 0.001	0.62	0.80	(0.64–0.96)
Female	29.60	16.89	22.91	5.60	24.15	p < 0.001	0.63	1.07	(0.96–1.18)	0.07	(− 0.11–0.24)

Note. EMM = estimated marginal means were calculated using intention-to-treat mixed models.

Among users who completed the program, 63.97% of GAD cases (n = 222/347) and 59.94% of MDD cases (n = 217/362) remitted with treatment. Based on the standard deviations of the estimated marginal means (e.g., 5.47 for the GAD-7 and 6.70 for the PHQ-9), users whose pre-treatment GAD-7 total score varied by at least 7.16 points and/or pre-treatment PHQ-9 total score varied by at least 7.43 with treatment were classified as either experiencing reliable improvement or deterioration with 95% confidence (Jacobson and Truax, 1991). As a result, 25.41% and 23.42% experienced reliable change in their anxiety (n = 141/555) and depression (n = 130/555) respectively, and 13.33% experienced reliable reductions in both their anxiety and depression symptom severity. A total of 6 users (1.08%) evidenced reliable deterioration with respect to their anxiety only (n = 1/555), depression only (n = 4/555) or both their anxiety and depression symptom severity (n = 1/555).

Discussion

This study evaluated two fully-automated, self-guided iCBT programs for mixed anxiety and depression in a naturalistic setting. Users experienced significantly milder symptoms of depression, anxiety and psychological distress following their program. We observed moderate to large within group effect size reductions (g range = 0.65–1.29) on all outcome measures in both the four- and three-lesson programs, and when the samples were stratified by gender and completer type. Among those who completed their program, approximately two thirds of probable cases of GAD and MDD had remitted. As predicted, completion rates were modest in both studies (13.83% in Study 1 and 13.11% in Study 2). Reducing the length of the program from four to three lessons thus does not appear to have changed treatment adherence or effects.

Predictors of adherence

iCBT programs tend to have lower completion rates once disseminated in naturalistic vs. controlled settings. This trend is generally exacerbated with unguided programs (Leykin et al., 2014). Consistent with this trend, the adherence rates reported herein were lower than those reported in controlled studies (e.g., Donker et al., 2013a, Donker et al., 2013b, Lintvedt et al., 2013). Yet these rates are commensurate with those found in previous unguided iCBT studies that have been conducted in naturalistic conditions (e.g., Christensen et al., 2006, Christensen et al., 2004a, Christensen et al., 2004b, Farvolden et al., 2005, Leykin et al., 2014).

We note that there were minor differences in the predictors of adherence between Study 1 and 2. First, baseline anxiety symptom severity and distress were not associated with adherence in Study 1. In Study 2 these differences were statistically significant but may not be clinically meaningful, with completers reporting only slightly less distress, anxiety and depression than non-completers (e.g., 1 or 2 points difference on each measure). Second, although gender did not predict adherence in Study 1, males were slightly more likely to complete treatment in Study 2. The magnitude of these differences (2–3 percentage points) is unlikely to be clinically meaningful, instead these discrepancies are more likely to be associated with differential power to detect subtle variations. A consistent trend across studies was however, the relationship between distress and adherence. The rates of change in psychological distress across treatment did not vary between non-completers and completers, whom, on average, received similar benefits from each lesson. It is therefore possible that failure to complete all lessons was not due to completers receiving additional benefits from each lesson compared to non-completers.

Treatment effects

The effects of the three and four-lesson self-guided Worry and Sadness courses are consistent with previous findings. We observed similar reductions in depression and anxiety symptoms severity and psychological distress in the six-lesson clinician-guided Worry and Sadness program (Newby et al., 2013, Newby et al., 2017, Newby et al., 2014). Compared to other fully unguided iCBT studies, our study found somewhat larger effect size reductions in anxiety and depression symptom severity (e.g., Boettcher et al., 2012, Christensen et al., 2006, Christensen et al., 2004a, Christensen et al., 2004b, Farvolden et al., 2005, Leykin et al., 2014). Importantly, baseline levels of psychopathology observed in our samples were not trivial, with approximately 90% of users reporting clinically significant symptoms of psychological distress, and 80% reporting symptoms consistent with a probable diagnosis of MDD and/or GAD. Thus, our findings, coupled with results from previous research, indicate that brief self-guided iCBT programs that are disseminated in “real world” settings have the capacity to attract large numbers of users who are experiencing clinically significant levels of anxiety and depression symptoms, and that these programs can improve users' wellbeing.

With anxiety and depressive disorders affecting between 10 and 20% of adults each year (Andrews et al., 2001, Kessler et al., 2005, McEvoy et al., 2011), effective and scalable treatments are needed. Although face-to-face CBT and clinician-guided iCBT have been shown to be effective, fully automated unguided iCBT programs are eminently scalable and can be rapidly disseminated. Within a stepped-model of care, unguided iCBT may represent an initial phase of treatment for some who subsequently progress to face-to-face CBT or guided iCBT. Others may benefit from using unguided iCBT as an adjunct to other treatments. Unguided iCBT programs also present a feasible option for individuals who do not want clinical assistance and prefer an anonymous and autonomous treatment. Nevertheless, as we have shown herein for some users – two thirds had remitted and a third of whom achieved reliable improvements in their anxiety and/or depression symptom severity – unguided iCBT appears to be a sufficient stand-alone intervention.

Strengths and limitations

The two studies reported herein have several strengths. Both programs were fully-structured and therefore have more fidelity than face-to-face services. The predictors of adherence and treatment effects were examined among a large number of users seeking help for their mental health problems. Moreover, both programs were completely unguided and thus true represent the effectiveness of unguided interventions once disseminated into “real world” settings. These studies however, are not without limitation. First, our use of “real world” data means that we did not employ a control group. It is therefore not possible to determine the programs' relative effectiveness against spontaneous remission and regression to the mean. Although a number of existing RCTs suggest that unguided iCBT is effective, future research may compare self-guided iCBT programs to other self-guided online treatments (such as mindfulness-based interventions) and control conditions. Second, no follow-up data were collected and as a result, we were unable to assess the long-term effects of the unguided iCBT programs. Finally, our results can only be generalized to individuals who are computer-literate and have enough insight into their symptoms to enroll in an iCBT program. Despite the limitations of these two studies, the automated nature of theses interventions renders them scalable and cost-effective, and as such, these type of programs are likely to continue to play a role in the delivery of mental health care.

Conclusions

This evaluation of two fully automated self-guided transdiagnostic iCBT programs for depression and anxiety found that a substantial number of individuals with clinically significant symptoms are attracted to such treatments. Although adherence was relatively low, the extent to which users adhered to treatment was comparable to other unguided iCBT programs that are available. Importantly, these programs significantly reduced users' depression and anxiety symptom severity and psychological distress, and effects were robust across gender and for those users who did and did not complete treatment.

Funding acknowledgment

JN is supported by a NHMRC Early Career Research Fellowship (1037787). AM is supported by an Australian Government Research Training Program Scholarship.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.