Guided Internet-based cognitive behavioral therapy for mild and moderate depression: A benchmarking study.

Major depression is among the most common and debilitating disorders worldwide, associated with large societal and individual costs. Effective treatments exist, but accessibility is scarce. Guided Internet-Based Cognitive Behavioral Therapy (guided iCBT) is a promising approach to reach more people in need of help. In the present pilot study, we investigated the outcome of a guided iCBT program for mild and moderate depression when disseminated from Sweden to Norway. The guided iCBT intervention was implemented within a university-based outpatient clinic by six student therapists under supervision. Twenty-two participants with mild and moderate depression were included in the study. Large treatment effects were found for depressive symptoms, whereas small to medium effects were observed for anxiety symptoms. More than half (55%) of the participants were classified as recovered at post-treatment and more than a third (41%) at follow-up. No participants had a significant deterioration from pre- to post-treatment, but two reported a significant deterioration from post-treatment to 6-month follow-up. Benchmarking the present results against those reported in the four original Swedish studies, we found that the treatment effect in the Norwegian study was slightly higher at post-treatment and slightly lower at 6-month follow-up compared to the outcome in the Swedish studies. The results should be interpreted with caution, as our sample was small and had no control group.

Introduction

Major depression is one of the most common disorders with a lifetime prevalence of 15 to 17% (Kessler et al., 2003) and causes a considerable health problem worldwide (Ferrari et al., 2013). Major depression has detrimental consequences for both the individual and the society. The individual costs include impaired relationships, reduced quality of life, and reduced income (Ebmeier et al., 2006, Pincus and Pettit, 2001). The societal costs include losses in productivity, high rates of sick absence, and social and health care expenses (OECD, 2015).

Both psychological and pharmacological treatments have been found to be effective in treating depression (Ebmeier et al., 2006). Several psychological treatments for depression have been investigated in randomized controlled studies and have been found to be effective (Cuijpers et al., 2008, Wampold et al., 2002). Although effective treatments for depression exist, many people in need of help do not access evidence-based treatment (Collins et al., 2004, Ebmeier et al., 2006, Shafran et al., 2009). Kessler et al. (2003) reported from a US sample (n = 9090) that only 46–57% of the 12-month cases were receiving health care treatment for major depression, and that the treatment was evidence-based in only 18–25% of the cases. Thus, there seems to be a gap between the need for psychological therapy and the accessibility of effective therapy (Shafran et al., 2009).

Internet interventions have been developed in order to accommodate the growing demand of effective psychological treatment of depression (Andersson, 2016). In a systematic review, Hedman et al. (2012) identified 20 randomized controlled trials conducted by six independent research groups reporting on iCBT for depression and/or depressive symptoms. The outcomes reported in these studies when compared to waiting list or treatment as usual control groups have varied from strong (Andersson et al., 2005, Andersson et al., 2013b, Christensen et al., 2004, Kessler et al., 2009, Meyer et al., 2009, Perini et al., 2009, Spek et al., 2007b, Vernmark et al., 2010), to moderate (Christensen et al., 2004) to no effect (Clarke et al., 2002, Clarke et al., 2005, de Graaf et al., 2009, O'Kearney et al., 2006). The heterogeneity of effect across studies can partly be attributed to the varying degree of therapist support and differences in study design (Spek et al., 2007a). In an early meta-analysis of iCBT and other computerized interventions for depression, Andersson and Cuijpers (2009) reported that guided iCBT had stronger between-group effects (d = 0.61) than unguided (d = 0.25) treatments. This is in line with more recent reviews and meta-analyses (Johansson and Andersson, 2012, Richards and Richardson, 2012, Cowpertwait and Clarke, 2013). Moreover, a recent systematic review showed that there were no difference between face-to-face and iCBT for depression (Andersson et al., 2016), even if the number of direct comparisons was small (n = 5).

The research group led by Andersson et al. (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010) developed the guided iCBT for depression piloted in the present study and have conducted four randomized controlled trials on guided iCBT for depression. The within-group effect sizes on primary outcome measures of depression in these four studies have been reported to be large at both post-treatment and follow-up. Furthermore, Hedman et al. (2014) conducted an effectiveness study investigating the effects of guided iCBT for depression in a routine psychiatric care setting (n = 1203). The results showed that the patients had made large improvements from pre- to post-treatment on depressive symptoms (d = 1.27), and that the improvements were sustained at 6-month follow-up.

Guided iCBT for depression has been implemented in the public mental health care systems in several countries, for instance in England (https://www.england.nhs.uk/mentalhealth/adults/iapt/) and Sweden (http://web.internetpsykiatri.se/). In Norway, treatment of depression based on the Internet has not been available in the public mental health clinics. Furthermore, no studies have been conducted on guided iCBT for mild and moderate depression in Norway. This is somewhat surprising as the clinical guideline on mild to moderate depression, published by The Norwegian Directorate of Health in 2009, recommended guided CBT for depression, exemplified with MoodGym (https://moodgym.anu.edu.au/welcome) (Helsedirektoratet, 2009). It is important to document the effects of guided iCBT for depression before implementation in the Norwegian mental health care services. There is a lack of knowledge in regard to dissemination effects across countries, subcultures, research groups, and clinical settings (Andrews and Williams, 2015), which makes this study a useful contribution to the generalization of iCBT effects into new settings.

Thus, we had two aims in the present study: First, to investigate the effects of guided iCBT on depressive and common comorbid symptoms and complaints at post-treatment and 6-month follow-up. Second, to compare the effects of the iCBT program in the present study to the effects reported in the Swedish studies (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010).

Method

The present study was an open pilot with assessments at pre-treatment, post-treatment and at 6-month follow-up. Preliminary results from the study including post-treatment outcome, but not the follow-up results, have been published previously in Norwegian (Nordgreen et al., 2015). In the present study, we present both post-treatment and follow-up outcome.

Translation and adaption

Modules were received from Gerhard Andersson as word.doc documents, all text-based content. As the first step we translated the text from Swedish to Norwegian using psychology students and clinical psychologists. The cultural adaptation was minimal as Norway and Sweden have large cultural overlap. The second step was to implement the text in a web platform. Pictures and illustrations were included in order to make the text more readable. Due to limited funding for this project the text was implemented in a web-platform without the highest security level. The patients and therapist dialogs were therefore conducted via telephone.

Therapists

All clinical contact was conducted by 6 student therapists at the psychological outpatients clinics, University of Bergen, under the supervision of a certified clinical psychologist (TN). The students had completed at least one year of standard clinical internship. They were in addition trained and supervised in conducting diagnostic assessments with Mini-International Neuropsychiatric Interview (MINI; Sheehan et al., 1998), and guided iCBT through seminars and weekly supervision by a certified clinical psychologist (TN).

Recruitment

Participants were recruited via two press announcements in a local newspaper, one in September 2013 and the other in February 2014. The announcements included information about the treatment program and contact information. In total, 57 persons responded and were screened in a telephone interview.

Screening and inclusion

The screening comprised information about the treatment program, and assessment of the fulfillment of the inclusion criteria: 1) aged 18 to 65 years, 2) presenting a major depressive episode according to the criteria in Mini-International Neuropsychiatric Interview (MINI; Sheehan et al., 1998), 3) if usage of anti-depressants, the dosage had to be stabile for 4 weeks, 4) no current or previous treatment with CBT for depression, 5) access to internet, 6) no substance abuse, and 7) not in need of treatment for more acute problems. The latter was based on information from the MINI interview and from the screening interview. After the screening, a total of 19 persons were not further assessed for eligibility due to: no current depressive episode (n = 7), currently receiving or previously received CBT (n = 6), suicidal thoughts (n = 3), use of medication less than 4 weeks (n = 2), and substance abuse (n = 1).

Diagnostic interviews were conducted using MINI in order to assess the diagnostic criteria for DSM-IV: A. Major depressive episode, C. Suicidality, D: Manic episode, E: Panic disorder, F: Agoraphobia, G: Social phobia, H: Obsessive-Compulsive disorder, O: Generalized anxiety disorder, L: Psychotic disorders, and M: Anorexia nervosa. Suicidality, Manic episode and psychotic disorders were exclusion criteria.

The 38 individuals who met the inclusion criteria were invited to a face-to-face interview assessing the exclusion criteria and the additional inclusion criteria: 1) a total score between 15 and 30 on the Montgomery-Åsberg Depression Rating Scale Self-Report (MADRS-S; Svanborg and Ekselius, 2003, Svanborg and Åsberg, 1994), as used in the four original benchmarking studies (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010) and 2) a score of less than 4 on item 9 on MADRS-S (zest for life). The following exclusion criteria were used: 1) severe depression (a score of 31 or higher on MADRS-S), 2) bipolar disorder, 3) suicidal thoughts or plans, 4) psychosis, and 5) substance use assessed by The Alcohol Use Disorders Identification Test (AUDIT; Saunders et al., 1993) and The Drug Use Disorders Identification Test (DUDIT; Berman et al., 2007).

The participants were informed about the purpose of the study, and that they were expected to work actively with the program 4–6 hours each week. In total, 16 of the 38 persons who were invited to the face-to-face interview were not included in the study, due to the following reasons: did not meet the criteria of mild or moderate depression (n = 5), did not attend the interview (n = 4), did not have time to work on the program (n = 4), did not want to participate without any given reason (n = 3). Included participants provided their informed consent, and were informed that they could withdraw from the study without giving any reason. Out of the 22 participants, 7 qualified for mild depression (MADRS-S > 13) and 15 moderate depression (MADRS-S ≤ 20). The patients were informed that they had to contact their general practitioner if in need of acute treatment.

Ethical approval

The study was reviewed and recommended by the Regional Committee for Medical and Health Research Ethics, West.

Treatment

Program

The present study used the guided iCBT program for mild and moderate depression originally developed by Andersson et al. (2005), with minor amendments in subsequent trials. The program is based on Beck's cognitive model (Burns, 1999) and behavioral activation (Lewinsohn, 2010, Martell et al., 2001) for depression. The material in the program comprised text organized into eight modules. Each module included text equivalent to 13 word.doc pages. The modules comprise information about depression, tasks and exercises, frequently asked questions, and assessment of symptoms. Moreover, themes, concepts, and tasks typically included in face-to-face CBT are also part of the program. Modules 1–2: Psychoeducation about the cognitive behavioral model for depression. The participants learn about CBT models for development and maintenance of depressive symptoms, as well as how punishment and negative and positive reinforcement can relate to the symptoms. The main task in the first module is setting treatment goals. Modules 3–4: The participants learn how to get an overview of their existing activities, and how to enhance positive, rewarding activities and reduce negative, punishing activities. Module 5: The participants are introduced to the impact of selective, negative interpretations and negative automatic thoughts on depression, and are provided with strategies to evaluate these interpretations and replace them with more realistic ones. Modules 6–7: Sleep and relaxation, including information about sleep, as well as sleep strategies. Module 8: Summary of all the previous modules, as well as relapse prevention and future goals. After completing each module, the participant got access to the subsequent module.

Therapist guidance

The student who conducted the face-to-face assessment interviews also was the therapist guiding the participant during the treatment program. After each module, the therapist and the patient had a pre-scheduled manualized telephone appointment that was expected to last for 10 min. Common themes were treatment activity the past week, monitoring of symptoms, potential problems, other treatment needs and support in completing the modules.

Measures

Primary outcome measures of depressive symptoms

MADRS-S (Svanborg and Åsberg, 1994) comprises 9 items measuring depressive symptoms during the past three days on a seven-point scale (0 = normal, 6 = pathological). According to Svanborg and Ekselius (2003), a score of 13 or higher should be used to distinguish individuals with depression from non-depressed individuals, also used by Hedman et al. (2014) as a cut-off in their study.

Beck Depression Inventory (BDI; Beck et al., 1988b) comprising 21 items measuring the intensity of depressive symptoms the past seven days on a scale from 0 (never) to 3 (almost all the time). The cut-off scores are as following: 0–9 = no depression, 10–18 = mild to moderate depression, 19–29 = moderate to major depression, 30–63 = major depression.

The primary outcome measures were assessed before, during, and after treatment, as well as 6 months after completing treatment. Assessments during treatment were used for clinical assessment only.

Comorbid symptoms and complaints

Beck Anxiety Inventory (BAI; Beck et al., 1988a) comprises 21 items measuring the intensity of anxiety symptoms the past seven days on a scale from 0 (never) to 3 (almost all the time).

Generalized Anxiety Disorder 7 (GAD-7; Spitzer et al., 2006) comprises seven items measuring the frequency of GAD the past two weeks on a scale from 0 (not at all) to 3 (nearly every day).

Patient Health Questionnaire 9 (PHQ-9; Kroenke and Spitzer, 2002) comprises nine items measuring the frequency of depressive symptoms and anhedonia the past two weeks on a scale from 0 (not at all) to 3 (nearly every day).

BAI was assessed at pre-treatment, post-treatment and at follow-up. PHQ-7 and GAD-7 with only pre- and follow-up assessment in order to not burden the patients too much.

The data from pre- and post-treatment, as well as 6-month follow-up data, were included in the analysis.

Statistical analysis

SPSS 22.00 was used to analyze the data. We used a mixed linear model fitted with full information maximum likelihood estimation to analyze the change in symptom intensity on group level from pre-treatment to post-treatment and follow-up. By using full information maximum likelihood (FIML) missing data are not imputed, but estimated based on information from all of the observed data. Rather than disregarding incomplete cases, full information likelihood estimation makes use of all available data in the estimation process of parameter values.

Model selection was empirically determined by means of likelihood ratio tests for nested models. The model with the best fit included an unstructured correlation between assessment points. We also used within-group effect sizes (Cohen et al., 2013) to assess the scope of change from pre-treatment to post-treatment and follow-up respectively.

To assess changes in depression symptoms on an individual level, we used Jacobson and Truax' (1991) criteria for identifying clinical reliable change. MADRS-S (Svanborg and Åsberg, 1994) was used, as this instrument has an established cut-off score and often used in public mental health services as an assessment tool for depression (Schulte-van Maaren et al., 2013). Clinical change on MADRS-S was defined as going from above the clinical cut-off MADRS-S > 12; Hedman et al., 2014) at pre-treatment to below the clinical cut-off at post-treatment or follow-up. Reliable changes in pre- to post-treatment MADRS-S scores and pre- to follow-up MADRS-S scores for each patient were calculated using the formulas:

RCI = Reliable Change Index; X = pre-treatment score; X = post-treatment score; SE = Standard error of differences; SDpre = Standard deviation at pre-treatment, rxx = reliability of the outcome measure (Cronbach's Alpha).

Clinical reliable changes were grouped in the following four categories: a) Recovered: reliable and clinical change in MADRS-S, b) Improved: reliable change, but not clinical change, c) No change: no reliable nor clinical change, and d) Deteriorated: reliable negative change in MADRS-S. The analyses on reliable and clinical change are based on the intention to treat principle assuming that drop-outs had no change of status of reliable and clinical change after the drop-out, i.e. last observation carried forward.

In order to investigate whether the change in depressive symptoms in the present study was different from what has been reported in previous studies, we did a benchmarking (Minami et al., 2008) at post-treatment and at follow-up. At post-treatment, the present results were compared to four previous controlled efficacy-studies (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010), and three of these studies at 6-month follow-up with available data (Andersson et al., 2005, Johansson et al., 2012, Vernmark et al., 2010) (please see Table 4 for overview). All of the previous studies reported group level changes in depressive symptoms based on MADRS-S from pre-treatment to post-treatment and follow-up respectively. We first calculated the unbiased within-group effect-size for each of the studies, and then found the mean of the unbiased within-group effect-sizes across these four studies, as recommended by Minami et al. (2008). Thereafter, we used the same procedure to calculate the unbiased within-group effect-size in the present pilot study. The unbiased mean within-group effect-size for MADRS-S in the original studies was large at post-treatment (d = 1.50) and follow-up (d = 1.47). In order for the unbiased within-group effect-size in the present pilot study not to be considered as significantly different from the four benchmarking studies, it could not deviate more than ± 0.20 from the unbiased mean within-group effect-size. This value is used because an effect-size of ± 0.20 or less is considered as clinically trivial (Minami et al., 2008). Thus, the effect-sizes in the present study needed to be within the range 1.30–1.70 at post-treatment and 1.27–1.67 at follow-up in order not to be significantly different from the original studies.

Table 4

Benchmarking analysis.

	Pilot studies				Efficacy studies
	N = 22		Andersson et al., 2005N = 36		Vernmark et al., 2010N = 29		Johansson et al., 2012N = 37		Andersson et al., 2013bN = 33
	M (SD)	ES	M (SD)	ES	M (SD)	ES	M (SD)	ES	M (SD)	ES
MADRS-S pre	21.1 (4.7)		20.1 (5.7)		21.2 (4.1)		22.5 (5.7)		23.6 (4.8)
MADRS-S post	11.2 (7.1)	2.11	12.7 (8.3)	1.30	15.0 (7.0)	1.49	15.2 (7.7)	1.28	13.6 (9.8)	2.08
MADRS-S 6 month f-u	15.3 (10.0)	1.24	14.6 (9.2)	0.96	12.4 (9.3)	2.12	14.4 (8.3)	1.42	*
BDI pre	22.3 (6.2)		20.5 (6.7)		22.2 (6.3)		25.3 (8.0)		24.0 (7.0)
BDI post	11.3 (8.5)	1.76	12.2 (6.8)	1.24	12.3 (7.3)	1.57	16.1 (10.4)	1.15	13.6 (10.1)	1.49
BDI 6 month f-u	13.6 (11.0)	1.38	13.1 (9.1)	1.10	10.9 (9.8)	1.79	13.0 (9.7)	1.20	*
BAI pre	15.3 (6.9)		17.1 (8.2)		13.6 (6.8)		16.6 (9.2)		15.7 (6.7)
BAI post	11.9 (7.4)	0.5	14.1 (8.4)	0.37	8.8 (6.3)	0.71	12.4 (7.9)	0.46	8.6 (7.8)	1.06
BAI 6 month f-u	13 (7.5)	0.3	15.1 (9.3)	0.24	10.7 (9.4)	0.43	11.4 (8.3)	0.57	*

Note. MADRS-S = Montgomery and Åsberg Depression Rating Scale Self-report, BDI = Beck Depression Inventory, BAI = Beck Depression Inventory. ES = Effect size Mpost,- Mpre/SDpre.* = data not available.

Results

A total of 22 participants with mild (n = 7) and moderate (n = 15) were included in the study. There were 6 men (27%) and 16 women (73%) (see Table 1). More than two thirds of the sample had experienced depression previously (68%) and had sought help for their depressive symptoms prior to the study (73%). Almost half (41%) of the participants had at least one comorbid anxiety disorder; social phobia (n = 5), agoraphobia (n = 2), generalized anxiety disorder (n = 2) and panic disorder (n = 1).

Table 1

Demographic description of participants.

	Mean/SD/%
Age	43.5 (11.6)
Female	72.7
Cohabitant/married	63.6
Education at college + level	59.1
Have sought help for depression previously	72.7
Former depression episodes	68.2
Comorbid anxiety disorder	45.5
Former depression treatment at psychologist/psychiatrist	9.1
Former depression treatment at general practitioner	68.1
Medication	18.2
Employed	68.2
Presently on sick leave, vocational rehabilitation, disabled	31.8

Note. N = 22; SD = Standard deviation.

Adherence and attrition

In total, 17 of 22 (77%) participants completed the treatment. The average number of modules completed was 7 (range 2–8). Dropout during treatment (n = 5) was due to the following reasons: in need of other treatment (n = 2), did not have enough time (n = 1), severe somatic illness (n = 1) and did not give any reason (n = 1).

One participant did not attend neither post-treatment nor follow-up assessment. In total 18 participants (82%) were assessed at post-treatment and 18 (82%) participants at follow-up. Participant flow is presented in Fig. 1.

Fig. 1

Participant flow.

Effect of treatment on group level

Primary and secondary outcome measures, including Cohen's d, are presented in Table 2. The main effects of time were statistically significant for the primary outcome measures BDI, F(1,9) = 34.91; p < .001, and MADRS-S F(1,2) = 29.44; p < .001). Both outcome measures showed large within-group effect sizes from pre-treatment to post-treatment (BDI d = 1.76; MADRS-S d = 2.11) and to follow-up (BDI d = 1.38; MADRS-S d = 1.24). The main effects of time were also significant for the secondary outcome measures (PHQ-9, F(1,5) = 13.38; p < .001) and GAD-7, F(1,5) = 3.31, p < .05), but not for the secondary outcome measure BAI (F(1,2) = 1.67; p = .21). PHQ-9 had a large effect-size at follow-up (d = 1.32), whereas the effect sizes for BAI and GAD-7 were small to medium (BAI: post-treatment d = 0.49; follow-up d = 0.33. GAD-7: follow-up d = 0.51).

Table 2

Primary and secondary outcome measures.

	M (95% CI)	SD	ESw	Linear mixed models
Beck Depression Inventory
Pre-treatment	22.26 (19.65–24.87)	6.23
Post-treatment	11.30 (7.76–14.84)	8.47	1.76
6-month follow-up	13.64 (9.04–18.24)	11.01	1.38	F = 34.91, p < .001
MADRS-S
Pre-treatment	21.13 (19.07–23.21)	4.69
Post-treatment	11.22 (8.08–14.37)	7.10	2.11
6-month follow-up	15.32 (10.87–19.77)	9.98	1.24	F = 29.44, p < .001
Beck Anxiety Inventory
Pre-treatment	15.26 (12.22–18.29)	6.87
Post-treatment	11.88 (8.59–15.17)	7.35	0.49
6-month follow-up	13.00 (9.66–16.35)	7.52	0.33	F = 1.67, p = .21
Patient Health Questionnaire 9
Pre-treatment	15.72 (14.14–17.29)	3.55
6-month follow-up	11.27 (8.17–14.37)	6.98	1.25	F = 13.38, p < .001
Generalized Anxiety Disorder 7
Pre-treatment	8.68 (7.33–10.03)	3.05
6-month follow-up	7.13 (5.4–8.87)	3.87	0.51	F = 3.31, p = < .05

Note. MADRS-S Montgomery and Åsberg Depression Rating Scale – Self-report; M Mean; Cl = confidence interval; SD = standard deviation, ESw = Cohen's d within-group effect size.

Effect of treatment on the individual level

Clinical change

Clinical change (Table 3), defined as MADRS-S ≤ 12 (Hedman et al., 2014), was observed in 72% (13/18) of the completers at post-treatment, and 59% (13/22) in the intention-to-treat analysis. At 6-month follow-up a total of 9 of 18 (50%) completers had a clinical change, and 41% (9/22) in an intention-to-treat analysis.

Table 3

Clinical reliable changes from pre-treatment to post-treatment and 6-month follow-up, as measured by MADRS-S.

	Recovered	Improved	Unchanged	Deteriorated	Total
Pre-treatment to post-treatment
Completers	12 (67 %)	1 (6%)	5 (28%)	0	18
Intention-to-treat	12 (55%)	1 (5%)	5 (23%)	0	22
Pre-treatment to 6-month follow-up
Completers	9 (50%)	1 (6%)	6 (33%)	2 (11%)	18
Intention-to-treat	9 (41%)	1 (5%)	6 (27%)	2 (9%)	22

Note. MADRS-S Montgomery and Åsberg Depression Rating Scale – Self-report.

Reliable change

A reliable change (Table 3) on MADRS-S from pre to post was observed in 72% (13/18) of the completers, 59% (13/22) in the intention-to-treat analysis. From pre to follow-up a reliable change was observed in 56% (10/18) of the completers, 46% (10/22) in the intention-to-treat analysis. No reliable change from pre to post was observed in 28% (5/18) of the completers, 23% (5/22) in the intention-to-treat analysis. From pre to follow-up 33% (6/18) completers remained unchanged, 27% (6/22) in the intention-to-treat analysis.

None of the participants had a reliable deterioration from pre to post. From pre to follow-up 11% (2/18) had a reliable deterioration, 9% (2/22) in the intention-to-treat analysis.

Clinical reliable change

A recovered status (Table 3) was obtained for 66.7% (12/18) completers, 55% (12/22) for the intention-to-treat analysis. At follow-up, 50% (9/18) had a clinical reliable change, 41% (9/22) for the intention to treat analysis.

Benchmarking

Post-treatment and follow-up results were benchmarked against the original studies. The unbiased effect size of MADRS-S at post-assessment was in the present study d = 2.03. The unbiased within-group effect size at post-treatment from the four previous iCBT studies (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010) was d = 1.50 (range 1.30–1.70). Our unbiased within-group effect size (d = 2.03) did exceed the upper limit (d = 1.70) of the unbiased mean within-group size in the Swedish studies. This indicates that the effect on MADRS-S in our pilot study was somewhat better than the benchmarking studies at post-assessment. At follow-up, the unbiased effect size on in the present study was d = 1.19, below the unbiased mean within-group effect size of d = 1.47 (range 1.27–1.67) in the three previous studies having a 6-month follow-up (Andersson et al., 2005, Johansson et al., 2012, Vernmark et al., 2010). This indicates that the effect measured by MADRS-S in our pilot study was somewhat inferior compared to the Swedish benchmarking studies at follow-up.

Discussion

The first aim of this study was to investigate the effects of guided iCBT on depressive symptoms. The results showed large effects (2.11–1.24) on depressive symptoms from pre-treatment to post-treatment and at follow-up. Small to moderate effects were identified from pre to post and 6-month follow-up on comorbid anxiety symptoms.

More than half of those included were recovered at post-treatment, and none deteriorated during treatment. At follow-up, improvement was lost in three participants, two of which had deteriorated.

Only two of the benchmarking studies (Vernmark et al., 2010; Johansson et al. (2012) provided data of recovered participants. In the study by Vernmark et al. (2010) at post-treatment, 37% of the participants were recovered and at 6-month follow-up 56% were recovered. Johansson et al. (2012) found at post-treatment a recovery rate of 30% and at 6-month follow-up 41%. We found that a higher percentage of our participants did achieve recovery at post-treatment (55%) and lower at follow-up (41%). We should however be cautious of making direct comparisons, as the outcome measures and clinical cut-off scores were not the same. For example, different BDI scores as cut-off (8 and 10 respectively) were used by Vernmark et al. (2010) and Johansson et al. (2012).

Two participants deteriorated from post-treatment to follow-up. It is well documented that depression is a recurrent disorder (Andrews, 2001, Posternak and Miller, 2001), and the risk of relapse after the first episode is approximately 50%, rising to about 70% after a second episode, and about 90% after three episodes (Barnhofer and Crane, 2009). As a majority of our sample (68%) reported having experienced depression previously, there is a relatively high risk of relapse. Furthermore, residual symptoms are common after completing treatment (Menza et al., 2003, Paykel, 2008). Although effective treatment exists, many patients with depression do not achieve full remission. In a meta-analysis, Casacalenda et al. (2014) found that 46.3% of patients with major depressive disorder achieve full remission following psychotherapy.

As there is a relatively high risk of relapse in depression after completing treatment, it is important to improve the treatment outcome in order to enhance remission and prevent relapse. One way of enhancing relapse prevention and sustain long-term effects is by adding booster programs, in which treatment is continued after the acute treatment phase. To our knowledge, relatively few studies in the field of guided iCBT have investigated the effect of such booster features, so future studies should address this important issue. In line with this, E. Andersson et al. (2014) investigated the effects of an Internet-based booster program for OCD patients, and found that adding a booster program improved the long-term effects and prevented relapse in some of the patients. Furthermore, Johansson and Andersson (2012) suggest that the effects of iCBT could be improved by being integrated in the day-to-day technology, such as smartphones and artificial intelligence.

Small to medium effects were also observed for anxiety symptoms. This indicates that not only is the guided iCBT treatment effective for depressive symptoms, but also anxiety symptoms. This is important, as it often is a great degree of comorbidity between mood and anxiety disorders (Regier et al., 1998) and according to MINI almost half of our sample had comorbid anxiety disorders. Our findings of improvements on anxiety symptoms are in line with the Swedish original studies (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010), which also reported small to moderate effects on comorbid anxiety symptoms.

Our second aim of this study was to benchmark the effects from the present study to effect reported from the developers of the program (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010). The results indicate that at post-treatment the effects in the present study were somewhat higher than the four benchmarking Swedish studies. At follow-up our effects were somewhat lower. This indicates that the short term effect of Norwegian version of the iCBT program translation is at least as effective as the original Swedish version, but that the improvement is less well maintained in the Norwegian version. The benchmarking results should, however; be interpreted with caution. We only had four comparison studies at post-treatment and three studies at 6-month follow-up, and estimates of effects may be unstable due to sampling errors or differences in samples, design and implementations. Also our sample size is small.

Treatment implementation in the present study differs somewhat from the Swedish studies. First and foremost, in the present study therapist guidance was delivered once a week via telephone, whereas the Swedish studies guidance was provided in unlimited e-mail contact between the therapists and patients. This was not possible to do in the present study due to restrictions in Norwegian laws. As a consequence, the contact between therapist and participant in the Swedish studies had been more frequent, whereas in our study the contact may have been less frequent. However, according to Lindner et al. (2014), guidance by telephone and e-mail has equal impact on symptoms reduction for depression, therapeutic alliance and treatment engagement. Another difference is the use of an online discussion group in addition to the guided iCBT treatment. In the study by Andersson et al. (2005), the participants also took part in a discussion group online, where they could discuss the self-help program with other participants. This may have facilitated the experience of receiving support and encouragement among the participants. None of the other Swedish studies reported any use of an online discussion group.

Our study shares several common features with the Swedish original studies. First and foremost, the study samples were quite similar, in terms of age, gender, educational level and employment. However, in the present study, participants more often had sought help previously and were more often on sick leave/vocational rehabilitation compared to the participants in the Swedish studies. Second, in all four of the Swedish studies (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010), students served as therapists, as with the present study.

A relatively high number of individuals were excluded from the guided iCBT treatment in the present study. In total, 22 of 57 interested persons (39%) were included in the study. The most common reasons for exclusion was that the participants did not meet the diagnostic criteria for mild – moderate depression (n = 12), were currently receiving or had previously received CBT (n = 6) and did not have enough time/did not give any specific reason (n = 7). It is important to ensure that the participants included in the treatment program, in fact present the problems the treatment is intended to treat. According to Johansson and Andersson (2012), reliable diagnostic procedures, as used in the present study, are important in order to ensure that the participants included have problems that are relevant for the treatment program. When participants have other problems or meet the criteria of other severe diagnoses (i.e. psychosis or bipolar disease) other treatments are more beneficial for the participant. The percentage of included participants in the present study of 39% is rather similar to the mean across the four Swedish original studies of 36%.

One concern in the iCBT literature is the relative high amount of dropouts and low treatment adherence. The attrition in the present study was 23% (5 of 22), which is comparable with what has been observed in the Swedish studies (M = 23.8%) (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010). This is however somewhat higher than what has been found for individual CBT in a meta-analysis by Cooper and Conklin (2015). Nordgreen and Havik (2011) identified this concern in a survey among Norwegian psychologists, in which a majority of the sample reported that they believed that client compliance would be higher in individual therapy compared to guided self-help treatment. However, the findings in the present study and a recent meta-analysis (e.g., van Ballegooijen et al., 2014) indicate that treatment adherence in guided iCBT is at a comparable level as of individual face-to-face CBT.

The present study has some major limitations. Due to small sample size, we lack the power to identify moderate to weak effects. However, our results are comparable to the results in the original studies investigating the same treatment, where in total 136 participants have received the treatment (Andersson et al., 2005, Andersson et al., 2013b, Johansson et al., 2012, Vernmark et al., 2010). Another limitation is that we did not have a control group. Therefore we cannot conclude that the positive outcomes in the study were due to the treatment or due to other factors. As depression is prone to spontaneous remission and recurrence (Andrews, 2001), we cannot rule out that some of our participants improved as a consequence of spontaneous remission. Another limitation is that only one follow-up assessment was conducted at 6 months, thus we cannot draw conclusions about the truly long-term effects of the treatment. Yet, positive effects of the treatment have been reported to be maintained up to 3,5 years follow-up (Andersson et al., 2013a).

The present study demonstrates several strengths. First, in the present study we used face-to-face structured diagnostic interviews before treatment, in order to ensure the quality of the diagnostic assessments. To date there is little that suggests that self-assessments can replace structured diagnostic interviews (Andersson and Titov, 2014). Second, we used sound benchmarking methodology in our study in order to compare our results with results from previous studies (Minami et al., 2008). Benchmarking is a recommended and useful method when evidence-based data is available from different settings.

In summary, the present study gives further support that guided iCBT for depression is an effective treatment, as indicated in several previous studies. Our findings indicate that significant reductions in depressive symptoms can be obtained via guided iCBT. The results also suggest that the treatment has not lost its efficacy via dissemination from Sweden to Norway, as our results are comparable to those found in previous studies. Our next step is to implement iCBT for mild and moderate depression in the secondary mental health care services in Norway. As part of this work we will address barriers to implementation of iCBT in Norway such as providing patients with the option of self-referring themselves to these services and establishing public reimbursement for iCBT in the same way as for face-to-face CBT.

Disclosure statement

The authors declare that there are no conflicts of interest.

Tine Nordgreen on behalf of the authors.