Effectiveness of guided internet-delivered cognitive behavior therapy for depression in routine psychiatry: A randomized controlled trial.

Depression is one of the most common health problems worldwide but is often undertreated. Internet-delivered cognitive behavioral therapy (ICBT) appears to be an effective treatment option, with the potential to reach a larger proportion of individuals suffering from depression. While many studies have examined the efficacy of ICBT for depression in randomized controlled trials, fewer have focused on the effectiveness of ICBT when used as an integral part of routine health care. In this study the effectiveness of an 8-week ICBT program was examined when delivered in a routine psychiatric setting. A total of 108 patients were referred and 54 were then included and randomized to either ICBT or a waitlist control condition. The sample had a lower education level and a higher proportion of individuals were on sick leave than comparable previous efficacy trials of ICBT for depression conducted in Sweden. Measures assessing depression, anxiety and psychiatric symptoms were administered before and after treatment, follow up was performed at 6- and 12 months after treatment had ended. ICBT resulted in significant reductions of depressive symptoms in the treatment group when compared to a waitlist control group with a large effect size (Cohen's d = 1.6). Treatment gains were maintained at 6- and 12 months after the treatment had ended. In terms of clinical significance, 58% of the sample had improved or recovered after treatment. The study was small, and patients received general psychiatric care after the ICBT treatment had ended which limits the implications. We conclude that ICBT appears to be an effective treatment for depression when delivered as an integral part of routine psychiatric care.

RCT Entities:   Population Interventions Outcomes

Introduction

Major depression with the core symptoms of depressed mood and loss of interest in activities is one of the most common health problems worldwide, affecting approximately 5% of the world population at any given time (Ferrari et al., 2013; Whiteford et al., 2015). Psychological and pharmacological treatments have been found to be effective in the treatment of depression (Cuijpers et al., 2009). While several forms of psychotherapy have been found effective for treating major depression, cognitive behavior therapy (CBT) is to date the most well studied form of psychotherapy for the disorder (Cuijpers, 2017). Many individuals suffering from major depression do however not receive adequate treatment (Kessler et al., 2003). In Sweden approximately 50% of those diagnosed with depression during a 12-month period received adequate treatment during that period (Johansson et al., 2013; Kessler et al., 2003). Johansson et al. (2013) concluded that depression is an undertreated condition in the Swedish population.

CBT delivered via the internet (ICBT) has since its introduction in the beginning of the 1990s gained popularity due to its acceptability, effectiveness and feasibility (Andersson et al., 2019). A large number of randomized controlled trials examining ICBT for depression have since its introduction established the method as an effective and viable treatment option for major depressive disorder (Andersson et al., 2016; Andrews et al., 2018; Carlbring et al., 2018). ICBT is also effective from a health economy perspective (Donker et al., 2015). In all, ICBT can be regarded as a promising treatment of a generally undertreated condition. ICBT is gradually disseminated from research settings to routine care (Titov et al., 2018). In Sweden this has brought on (among several caregiver initiatives) the development of a publicly funded national digital platform for ICBT and other forms of internet-delivered treatments. A rapid increase in the use of the platform could be seen during the first years of use. The platform was introduced in 2016 and monthly activity has increased fivefold until today (Invånartjänster 1177 Vårdguiden, 2018). A risk during the transition from research settings to regular clinical settings is that the participants in efficacy trials differ from patients in routine health care in characteristics relevant to treatment outcome (Westen et al., 2004), such as for example baseline severity, comorbidity, education level or treatment expectations. Another risk is that procedures vary between settings in a way which hinders generalizability from efficacy studies. An example of such a variation is that between self-selected samples and referral samples, the former being the most common type of recruitment in research settings and the latter a more traditional recruitment strategy in a clinical setting. Because procedures as well as patient characteristics can be expected to vary more in a clinical setting effectiveness studies pose a good opportunity to observe if and how the effect of ICBT differs when used in the more varied clinical context. Especially in psychiatry, characteristics such as severity, comorbidity and duration of mental health problems are expected to vary more than in the primary care context. Presently only one large effectiveness study on depression has been conducted in Sweden. Although the patients showed large improvements, these could not be clearly attributed to the treatment, as it did not include a control group (Hedman et al., 2014). In view of the relative absence of controlled studies of ICBT for depression in regular clinical settings, it is important to carry out such studies to see if the effects from research settings generalize to clinical settings.

Hence the aim of this study was to examine the effectiveness of ICBT on depressive symptoms, as demonstrated in a multitude of efficacy trials, when implemented in an outpatient psychiatric clinic using the clinic's routine practices and using a psychiatric referral sample instead of a self-selected sample and with the inclusion of a control group.

Material and methods

Design

A randomized controlled design was used. Participants were randomized into two groups: 1) an experimental group who received therapist supported internet delivered CBT for depression and 2) a waitlist control group.

Aims of the study

The hypothesis was that participants in the treatment group would show a decrease in depressive symptoms compared to a waitlist control group at post treatment. Also, reductions of anxiety related symptoms and general psychiatric symptoms were expected.

Further this study aimed to examine symptom levels of the participants 6- and 12 month after ending ICBT treatment.

Procedure and participants

A flowchart of the procedure, in line with the recommendations of the CONSORT statement (Schulz et al., 2011) is provided in Fig. 1. The intervention was performed at a psychiatric out-patient clinic in Sweden (“secondary” mental health care). Patients at psychiatric clinics in Sweden are normally referred from primary care in cases where the psychiatric treatment is expected to be long and when a more specialized care is required. The out-patient clinic treats adult patients with a range of psychiatric problems, but where no inpatient care is required.

Fig. 1

Flow of the participants through each stage of the trial process.

All personnel at the clinic were briefed about the study and the ICBT treatment option, and also about the criteria for participation. Caregiver personnel were asked to inform patients that potentially could benefit from treatment. Treatment options and plans were normally discussed in cross professional team meetings. All participants in the study were recruited via a within-clinic referral from a caregiver at the psychiatric hospital; no participants were recruited from primary care or via self-referral. Patients from the clinic were, if considered eligible, informed in general terms about the ICBT treatment option via their caregiver (e.g. the treating physician or psychologist informed verbally and by providing patients with a short information booklet). Patients were then contacted by study personnel to obtain verbal consent.

Potential participants were mailed information about the study and written informed consent was collected from all participants. A structured interview was performed at the clinic to assess eligibility. For diagnostic assessment the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders Axis I Disorders (First and Gibbon, 1997) was used. The manual was also used for assessing previous depressive episodes. Participants also completed a screening process that included all outcome measures. Participants excluded from the study were informed verbally or via letter. The participants included in the study were randomized to either immediate treatment or waitlist control group. The treatment lasted 8 weeks after which post treatment data was collected from all participants. Directly after post treatment data had been collected the control group received treatment. Follow up data on outcome measures were collected from all participants 6 and 12 month after the treatment had ended. At follow up both the control group and the intervention group had received treatment. The intervention was performed at the psychiatric clinic, not as an external treatment. The treatment was given within the frame of the psychiatric care, by psychologists at the hospital. After the ICBT intervention, patients continued their psychiatric care, and were normally assessed for their need of further interventions. Other treatment options were made available for those patients who were not interested in ICBT treatment. The trial was approved by the Regional Ethical Review Board (Dnr 2014/575).

Criteria for participation

Inclusion criteria entailed: being minimum 18 years of age, a primary diagnosis of major depressive disorder either single episode or recurrent episodes (without psychotic features). In case of comorbidity (i.e. when a patient meets diagnostic criteria for several diagnoses) the primary diagnosis was defined as the one that was perceived as most distressing and disabling in everyday life and having an earlier onset. Depressive disorder with postpartum onset was not included. If using antidepressants, a minimum of 30 days of stable dosage prior to inclusion was required. This criterion was verified with the participant's treating physician. Any changes in medication during the trial were to be reported by the treating physician. Inclusion criteria also required: that the participants agreed to refrain from psychotherapy during the participation in the study, were fluent in Swedish and had access to a computer, smartphone or tablet with internet connection.

Exclusion criteria included the following: an ongoing alcohol- or substance abuse disorder, being assessed as high-risk suicidal patient (having suicidal thoughts were not an exclusion criteria), being actively engaging in self-harm, having a current eating disorder, bipolar disorder or ongoing psychotic symptoms.

Randomization

Blocked randomization with randomly varied block sizes was used to avoid predictability in the allocation process. After the initial interview and screening, randomization was performed using an online randomization service (random.org). Blocks with two, four and six individuals were created, and assignment information was placed in sealed envelopes by a researcher not involved in the trial. Eligible participants were then consecutively assigned to either experimental- or wait list condition.

Outcome measures

All outcome measures were collected digitally via the same digital platform where patients accessed the treatment. The primary outcome measure was the self-rated version of the Montgomery Åsberg Depression Rating Scale (MADRS-S, Svanborg and Åsberg, 1994). MADRS-S is a questionnaire consisting of nine items measuring symptoms of depression. The digital version (used in this trial) has been compared to the paper and pencil version with regard to its psychometric properties by Holländare et al. (2010). Internal consistency of the online version was similar to the paper and pencil version (Cronbach's α between 0.73 and 0.81). The self-rating version of MADRS-S has showed high correlation to expert ratings (Svanborg and Åsberg, 2001). In this trial Cronbach's α for MADRS-S was calculated to 0.81.

Additionally the Hospital anxiety and depression scale, HADS (Zigmond and Snaith, 1983) was used. HADS consists of two subscales assessing depression (HADS-D) and anxiety (HADS-A). The psychometric properties for HADS in a Swedish sample have been examined by Lisspers et al. (1997), reporting Cronbach's α 0.84 for HADS-A and 0.82 for HADS-D. In this trial Cronbach's α was calculated to 0.74 for HADS-A and 0.78 for HADS-D.

The Outcome Questionnaire-45, OQ-45 (Lambert et al., 1996) was used as a measure of psychiatric symptoms. The OQ-45 is a general measure assessing a wide range of psychiatric symptoms and thus constitutes a good general measure. The OQ-45 has good reliability and high test-retest reliability (Beckstead et al., 2003; Lambert et al., 1996). The psychometric properties for OQ-45 in a Swedish sample have been examined by Strid et al. (2014), reporting Cronbach's α 0.88 for the total OQ-45. In this trial Cronbach's α for OQ-45 was calculated to 0.90.

Treatment

A treatment program originally developed by Andersson et al. (2005) was used in this trial. It has been used and updated in several trials targeting mild to moderate depression (Vernmark et al., 2010; Johansson et al., 2012; Andersson et al., 2013b; Jakobsen et al., 2017; Hedman et al., 2014), and with effects maintained at 3.5 year follow-up (Andersson et al., 2013a).

The treatment program consists of 8 weekly modules, to be completed during 8 weeks of treatment. The program is based largely on Becks cognitive therapy for depression (Beck, 1979; Burns, 1999) and behavioral activation (Lewinsohn, 1992; Martell et al., 2001). The program commences with psychoeducation of the behavioral and cognitive model of depression along with defining goals. Week 2 to week 6 focuses on, through practical exercises, applying these CBT based self-help strategies in participants' everyday life. Week 7 covers sleep management (Morin, 1996) and the last module consists of relapse prevention (Gortner et al., 1998). The treatment was guided with at least one weekly a-synchronic contact between participant and therapist. Participants worked with the weekly modules and each Sunday they reported their assignments to the therapist. The therapist gave text-based feedback (within a 24-hour time frame) focusing on encouraging the participants work and keeping the participant focused on the program. If participants did not report their assignments, the therapist phoned the participants and attempted to problem-solve how participants could adhere better to the treatment protocol. If the assignment was not performed as expected the clinician made an assessment if to continue with the next weekly module or to repeat the previous. Two licensed psychologists acted as therapists in this study.

Statistical analyses

All statistical analyses were performed with SPSS 25. Power was estimated by assuming an effect size (Cohen's d, defined as the standardized difference between groups obtained by calculating the mean difference and dividing by their pooled standard deviation) of 0.80, which would require 52 participants to obtain a power of 80% with a conventional alpha level of 0.05.

Analysis of variance and chi-square tests were used to analyze group differences in demographic data. Outcome data were collected at pre- and post-treatment, and at 6-month, and 12-month follow-up. Post treatment outcome measures were collected from all participants regardless if they completed all treatment modules or not. Multiple imputation was used to replace missing post treatment values, however missing follow-up measures (6-month, and 12-month) were not replaced. Main outcomes were calculated using one-way analyses of covariance (ANCOVA) with group variable as independent variable and outcome variables as fixed factors. The pre-treatment measures were used as covariates.

Follow-up at 6 and 12 months were performed outside the experimental design; both groups had received treatment at that point and were therefore treated as one group at follow-up assessments. No values were imputed for the analysis of follow-up measures. Analysis of variance (ANOVA) was used to compare group means at follow up to pre-treatment means.

Reliable change was assessed using the Jacobson and Truax method (1991). The method consists of two parts. The first part of the method is to establish a Reliable change index (RCI). RCI expresses the change from pre- to post- measurements using the formula RCI = (Xpost − Xpre) / SEdiff where Xpost is the post treatment score of a participant and Xpre the pre-treatment score. SEdiff “Standard error of the difference” is a function of the test's standard deviation and reliability and is calculated using the formula SEdiff = SD√(1 − rxx) where SD is the standard deviation of the baseline scores and rxx is internal reliability for the specific measure (Cronbach's α). Reliable improvement is considered a change that definitely exceeds measurement error. For measuring improvement an RCI exceeding a factor of 1.96 is considered to indicate reliable within a 95% confidence interval. For deterioration a factor of 0.84 is considered reliable for avoiding roof effects when baseline measures are high, as discussed by Rozental et al. (2017).

The second part is choosing a cut-off for the measurement, where a participant scoring below this is assessed as having no symptoms at a clinical level. Participants having a reliable improvement and a post score below the established cut-off for the specific measure (i.e. in the “no symptoms range”) are categorized as “recovered”. Participants with a reliable improvement but a post score above the established cut-off are categorized as “improved”. Participants with no reliable change from pre-treatment scores are categorized as “no change”. Participants with a reliable increase in the measurement are categorized as “deteriorated”. The following cut-off scores were used to define minimal- or below clinical threshold symptoms for each outcome measure. For the MADRS-S a score below 12 was used as suggested by Bandelow et al. (2006). For the OQ-45 a score below 64 was considered non-clinical (Lambert et al., 2004). A score below 8 was considered non-clinical symptoms on HADS-D and HADS-A respectively (Lisspers et al., 1997; Zigmond and Snaith, 1983). For improvement a change of 9 points were required for MADRS-S, 15 points for OQ-45, 5 points for HADS-D and HADS-A. For deterioration a change of 4 points was required for MADRS-S, 7 points for OQ-45, 2 points for HADS-D and HADS-A.

Results

Baseline characteristics

A total of 108 patients were referred, and of these 80 individuals responded and were assessed for eligibility via online questionnaires and live diagnostic assessment. A total of 26 individuals were excluded either for not matching inclusion criteria or matching exclusion criteria. The most common reason for exclusion was not meeting the criteria for a depressive episode.

We included 54 individuals who were randomly assigned to either ICBT or a waitlist control condition. The differences between the randomization groups are presented in Table 1. Analysis of variance and chi-square tests showed no significant difference (all p's > .05) among demographic variables. Hence, no statistically or clinically meaningful differences were detected between experimental group and waitlist control with regard to clinical baseline characteristics.

Table 1

Clinical and sociodemographic characteristics.

	Intervention (n = 27)	Control (n = 27)	Total sample (n = 54)
Women	18 (67%)	13 (48%)	31 (57%)
Mean age (years)	37,4	40,6	39.0
Min–max	19–75	23–64	19–75
Marital status
In relationship	11 (41%)	10 (37%)	21 (39%)
Single	8 (30%)	13 (48%)	21 (39%)
Separated/widowed/divorced	8 (30%)	4 (15%)	12 (22%)
Education
9-year compulsory school	1 (4%)	1 (4%)	2 (4%)
Secondary complementary	13 (48%)	16 (59%)	29 (54%)
College/university (not completed)	4 (15%)	1 (4%)	5 (9%)
College/university (completed)	9 (33%)	9 (33%)	18 (33%)
Antidepressant treatment
Ongoing	22 (81%)	23 (85%)	45 (83%)
Earlier	5 (19%)	1 (4%)	6 (11%)
None	0 (0%)	3 (11%)	3 (7%)
Current sick leave	13 (48%)	15 (56%)	28 (52%)
History of unemployment	16 (59%)	19 (70%)	35 (65%)
Debut (years)	23,3	24,6	24.0
Treatment credibility (yes & partly)	25 (93%)	26 (96%)	51 (94%)
Recurrent episode (yes): n (%)	26 (96%)	25 (93%)	51 (94%)
Previous episodes: 0	1 (4%)	2 (7%)	3 (6%)
1	0	2 (7%)	2 (4%)
2	2 (7%)	0	2 (4%)
3 or more	24 (89%)	23 (85%)	47 (87%)

The general psychiatric symptoms in the sample (as measured with OQ-45) were just above the clinic's total average (for all patients, as measured in Hansson et al., 2013), baseline for the sample was 97 and clinic average 92.

No dose changes in medication were reported by the treating physicians during the ICBT intervention.

Outcome of self-report measures

Post treatment measures were completed by 24 individuals in the experimental group and 23 individuals in the wait-list control group. Missing values were replaced using multiple imputation. Mean matching was used with five iterations; all outcome variables were used in the iteration process. To analyze differences between the groups on the outcome measures, one-way ANCOVAs were performed. For each analysis the corresponding pre-treatment measure was controlled for, baseline measures were used as covariates. Statistically significant differences between control group and treatment group at post treatment follow up were obtained for depressive symptoms as measured with MADRS-S F(1,51) = 35.9, p < .001 and as measured with HADS_D F(1,51) = 37.0, p < .001, for general psychiatric symptoms (OQ-45) F(1,51) = 22.8, p < .001 and for anxiety (HADS-A), F(1,51) = 28.6, p < .001. An overview of outcomes and effect sizes are presented in Table 2. No significant differences on outcome were detected between genders.

Table 2

Means for primary and secondary outcome measuresa.

Measure	n	Pre-treatment scoreMean (SD)	Post-treatment scoreMean (SD)	Effect size (Cohen's d) between groups, post treatment	95% CI of effect size	Control group after receiving treatmentMean (SD)n = 21
Montgomery Åsberg Depression Rating Scale
Intervention	27	26.2 (8.7)	13.6 (6.1)	1.6	1.0–2.2
Control	27	23.4 (5.1)	23.1 (5.7)			13.6 (7.0)
Hospital Anxiety and Depression Scale – Depression
Intervention	27	12.3 (4.1)	6.2 (3.6)	1.6	0.8–2.4
Control	27	11.7 (3.7)	11.1 (2.6)			6.9 (2.6)
Hospital Anxiety and Depression Scale –Anxiety
Intervention	27	11.9 (3.4)	7.0 (3.6)	1.5	0.7–2.2
Control	27	11.7 (4.1)	11.3 (4.5)			6.8 (3.4)
Outcome Questionnaire – 45
Intervention	27	98.2 (19.7)	75.7 (18.0)	1.3	0.6–1.9
Control	27	94.9 (19.0)	93.7 (15.3)			70.5 (19.1)

Missing values replaced with multiple imputation.

Clinically significant change

Clinical significance was assessed and is presented in Table 3. Using the Jacobson and Truax (1991) method for assessing clinically significant change 58% of the sample reached the status recovered or improved when assessing depressive symptoms with MADRS-S.

Table 3

Clinical significance according to Jacobson and Truax (1991) criteria.

Measure	n	Recovered (n)	Improved (n)	No change (n)	Deteriorated (n)c
Montgomery Åsberg Depression Rating Scale
Posta	24	29% (7)	29% (7)	38% (9)	4% (1)
6-Monthb	42	50% (21)	17% (7)	29% (12)	5% (2)
12-Monthb	41	54% (22)	17% (7)	29% (12)	2% (1)
Outcome Questionnaire – 45
Posta	24	21% (5)	29% (7)	50% (12)	0
6-Monthb	42	36% (15)	29% (12)	29% (12)	7% (3)
12-Monthb	41	46% (19)	24% (10)	24% (10)	5% (2)
Hospital Anxiety and Depression Scale – Depression
Posta	24	46% (11)	21% (5)	29% (7)	4% (1)
6-Monthb	42	45% (19)	5% (2)	50% (21)	12% (5)
12-Monthb	41	41% (17)	5% (2)	48% (20)	5% (2)
Hospital Anxiety and Depression Scale –Anxiety
Posta	24	54% (13)	13% (3)	29% (7)	4% (1)
6-Monthb	42	38% (16)	7% (3)	50% (21)	5% (2)
12-Monthb	41	37% (15)	10% (4)	46% (19)	7% (3)

Clinical change in Experimental group between pre- and post-treatment measures.

Clinical change in total sample between pre-treatment and follow-up- measures.

Deterioration was calculated using RCI = 0.84 as threshold, improved category was calculated with standard 1.96 threshold.

Follow up at 6- and 12 month

Follow up is reported in Table 4. Follow up at 6 months and 12 months were completed by 42 and 41 individuals respectively. At this stage the control group had also received treatment. With comparison to pre-treatment scores both groups combined showed a significant reduction on all outcome measures (depressive symptoms, general psychiatric symptoms and anxiety) both at 6, and at 12-month follow up (all comparisons at p < .001).

Table 4

Follow up measures for total sample (no imputation performed), both groups had received treatment.a

Measure	Baseline (SD)n = 54	Post treatment (SD)an = 48	6-Month follow up (SD)n = 42	12-Month follow up (SD)n = 41
Montgomery Åsberg Depression Rating Scale
Total sample	24.8 (7.2)	13.6 (6.3)	12.6 (7.8)	11.2 (6.9)
Hospital Anxiety and Depression Scale – Depression
Total sample	12.0 (3.9)	6.5 (3.2)	7.1 (3.9)	6.5 (3.7)
Hospital Anxiety and Depression Scale –Anxiety
Total sample	11.8 (3.7)	6.9 (3.5)	7.4 (3.9)	7.2 (3.8)
Outcome Questionnaire – 45
Total sample	96.5 (19.2)	73.4 (18.3)	68.9 (21.8)	64.1 (22.3)

The waitlist control group was given access to treatment after 8 weeks. At this measure point both the control and the experiment group had received treatment.

Adherence, dose effects and acceptability

The average module completion rate within the 8-week time frame was 6.3 (completing weekly assignments was used as a measure of completion) and the proportion not completing all 8 modules was 54%. There was no significant relationship between the amount of treatment completed and change in MADRS-S score.

Missing data and non-response analysis

A total of seven individuals did not complete post treatment follow up (treatment and control group combined, not including 6- and 12 month follow). This group was compared to the group completing its post measures on clinical and sociodemographic characteristics. The pre-treatment scores of the non-completers were elevated on all measures when compared to the completers group. A significant difference was found between groups on baseline depression scores as measured with HADS-D p = .016. No other significant differences were found, all p's > .05. No significant difference was found between the groups with regard to sociodemographic characteristics, all p's > .15.

Negative effects

Adverse events were not monitored in a structured way during this trial other than via comments from participant surveys. 4% of the participants were classified as deteriorated on the main outcome variable at post treatment assessment using the Jacobson & Truax methodology with an RCI of 0.84 as cut off for deterioration. A proportion between 5% and 12% reliably deteriorated during the follow up period, the deterioration rates for each outcome measure are reported in Table 3. Participants were able to comment on the treatment experience but very few reported any negative effects. Negative experiences reported concerned almost exclusively feelings of stress with regard to completing the weekly treatment modules in time. No serious adverse events (e.g. suicide attempts, self-harm) were recorded during the trial.

Discussion

The aim of this study was to examine the effectiveness of ICBT on depressive symptoms when treating patients in a psychiatric setting using the clinic's routine practices. Results on the primary outcome measure (MADRS-S) showed that depressive symptoms in the experimental group were significantly decreased as compared to the control group with a large effect size (d = 1.6). Similar results were obtained for depressive symptoms as measured with HADS-D (d = 1.6). The results also showed significantly reduced symptoms on the secondary outcome measures HADS-A and OQ-45 with large between-group effect sizes (d = 1,5 and 1,3 respectively). The gains were maintained at 6 and 12-month follow up. The results show that the version of ICBT used in this trial can be effective when delivered in routine psychiatry using the standard practices such as internal referral for recruiting patients. The use of intentional waiting time (as for the waitlist control group) is not standard procedure for psychiatric care. Still long waiting times are a common problem within Swedish psychiatric care and patients regularly have to wait for psychological treatments.

The same treatment program has been used in several prior trials for depression (trials referenced in Section 2.7 Treatment). To our knowledge all these trials have treated self-selected individuals, i.e. individuals responded to different types of advertisements for ICBT. In this study individuals sought treatment for depression through public health care and were thereafter offered ICBT. As this trial aimed to examine the effectiveness of ICBT specifically in a routine psychiatric setting, besides methodological differences it is also relevant to examine if results and sample characteristics in fact did differ in comparison to these similar trials.

Comparison with the mentioned trials (i.e. those using the same treatment program) shows that the sample in this study were less educated. In the present study 42% of the total sample had ongoing or completed university degrees (close to the national average of 43%, Statistics Sweden SCB, 2018). In prior trials with the same program the average for this variable has been between 59 and 89.7% for the total sample. A higher proportion was on sick leave, 52% in this study as compared to 11.6–24.1% in the compared trials. Baseline severity in terms of MADRS score is somewhat higher in this trial than in the compared trials, also the number of previous episodes are higher in this study than those reported in the compared trials (Andersson et al., 2005; Andersson et al., 2013b; Vernmark et al., 2010).

When comparing the outcomes of this trial to the earlier studies where the same treatment was used the results are similar but not equivalent. In the current study the treatment effect size was d = 1.6 (with regard to depressive symptoms). Effect sizes in the reference trials ranges from d = 1.27 (Hedman et al., 2014) to d = 2.11 (Jakobsen et al., 2017).

The proportion recovered was somewhat lower in this study. At post treatment 29% were categorized as recovered. The prior trials reported recovery rates ranges from 26.5% (Johansson et al., 2012) to 56% (Jakobsen et al., 2017).

When comparing adherence rates among these studies we see that 54% completed all modules in this study as compared to 65–94% in prior trials.

The treatment is generally somewhat less effective when compared to earlier studies using the same treatment program, but the results still compares well.

The acceptance rate (satisfied or very satisfied) in this study was 89%. Worth noting is that several individuals in the no change category were satisfied with the treatment. Several attributed lack of success to internal factors such as “not engaging sufficiently in treatment” (rather than experiencing treatment as not being engaging enough) or “being unfocused” (rather than experiencing treatment as too difficult). It is arguably problematic that this could be a result of a continued depressed attribution style among non-responders.

Participants not completing follow-up measures had generally more severe symptoms at baseline, however significant differences were only detected on one measure. Further studies would be needed to understand if more severe baseline symptoms predicts less favorable outcomes for this context specifically, as it does not appear to hold true in other settings (Bower et al., 2013; Webb et al., 2017).

Approximately 25% of the remitted patients could not be reached. No data was collected from these individuals and therefore no clear conclusions can be drawn on why they chose not to proceed with assessment. The fact that a considerable part of the patients that potentially could benefit from treatment for different reasons did not access the treatment limits generalizability and constitutes a form of barrier for treatment that should be investigated further, to gain knowledge about whether it is related to the context and specific patient demographics or if it is related to how the patients were informed about the study. Of the 80 patients that did consent to participating 26 (33%) were assessed as non-eligible. The rate assessed as eligible was higher than what is normal in Swedish ICBT trials using self-selected samples (e.g. Andersson et al., 2005; Andersson et al., 2013b; Vernmark et al., 2010).

There was no spontaneous improvement in the wait list control condition. This is unusual and could potentially be due to the fact that these were patients in secondary mental health care, which means that symptomatology generally is more severe or more persistent than among patients in primary health care. As proposed by Whiteford et al. (2013) spontaneous improvement may be smaller among more severely depressed patients.

Judging from the results in this trial, the most important steps to increase the usefulness of this method and improve its effectiveness in a psychiatric setting would be to improve adherence during treatment and informing and motivating patients before treatment.

Limitations

An important limitation is the small sample size. Small sample studies tend to report larger treatment effects than large sample trials (Sterne et al., 2000) it also limits the possibility to analyze moderators. No information was collected regarding interventions performed during follow-up. It is therefore not possible to separate the effect of ICBT from other treatments and natural remission in the 6- and 12-month outcome. The pharmacological treatment was not analyzed beyond measuring the proportions in the treatment group and control group that received pharmacological treatment. Therefore, differences between the groups regarding pharmacological treatment, such as length, dosage, and type of medication could have impacted the groups in different ways.

Only participants who could read and write in Swedish could participate which limits the sample and thereby the generalizability of the study. After treatment had ended patients continued with psychiatric routine care such as follow up visits to psychiatrists or visits to counselors and therapists when deemed suitable. A large part of the sample had ongoing antidepressant medication.

About 25% of the patients that were informed and referred about the study chose not to proceed with assessment, which may limit the generalizability of the results.

Conclusions

This study suggests that ICBT for depression is effective when delivered in a regular psychiatric setting using routine recruitment strategies (i.e. not using a self-selected sample). Treatment gains are maintained up to one year after treatment, when participants continue with regular psychiatric care. In comparison to earlier trials using the same treatment program the sample in this study was less well educated and a larger proportion of the sample was on sick leave. The treatment program was somewhat less effective in this particular setting compared to efficacy studies using the same treatment program, showing somewhat smaller effects, lower rates of treatment completers and smaller proportion recovered. While the effect size in this study is smaller than obtained in other studies using the same treatment program, the results however compares well to ICBT major depression-efficacy trials in general. The sample is too small to detect if variation among the demographic variables or clinical variables specific to this context moderates the effect of the treatment. Further effectiveness studies should explore if certain aspects of the clinical routines (e.g. way of informing and referring patients), and clinical or sociodemographic factors moderates the outcome of ICBT for depression.

Conflict of interest statement

Declaration of interest: none.