An Internet-based program for depressive symptoms using human and automated support: a randomized controlled trial.

PURPOSE: The purpose of this study was to analyze the efficacy of an Internet-based program for depressive symptoms using automated support by information and communication technologies (ICTs) and human support. PATIENTS AND METHODS: An Internet-based program was used to teach adaptive ways to cope with depressive symptoms and daily problems. A total of 124 participants who were experiencing at least one stressful event that caused interference in their lives, many of whom had clinically significant depressive symptoms, were randomly assigned into either an intervention group with ICT support (automated mobile phone messages, automated emails, and continued feedback through the program); an intervention group with ICT support plus human support (brief weekly support phone call without clinical content); or a waiting-list control. At pre-, post-, and 12-month follow-up, they completed depression, anxiety, positive and negative effect, and perceived stress measures. Results were analyzed using both intention-to-treat and completers data. The majority were women (67.7%), with a mean age of 35.6 years (standard deviation =9.7).
RESULTS: The analysis showed that the two intervention groups improved significantly pre- to posttreatment, compared with the control group. Furthermore, improvements were maintained at the 12-month follow-up. Adherence and satisfaction with the program was high in both conditions.
CONCLUSION: The Internet-based program was effective and well accepted, with and without human support, showing that ICT-based automated support may be useful. It is essential to continue to study other ICT strategies for providing support.

RCT Entities:   Population Interventions Outcomes

Introduction

Depression is one of the most prevalent and disabling psychological disorders worldwide1,2 and is found to be the second leading cause of disability worldwide.3 The epidemiological magnitude of this disease,1 its special tendency toward chronicity,4 the high rate of comorbidity,5 and the high personal, social, and economic costs6 suggest that prevention and treatment of depression should be a health priority.7 Some research studies have shown a significant relationship between stress, clinical symptomatology, and depression.8,9 It is universally accepted that adverse life events, along with biology and the environment, can cause mental health problems.10 Stressful circumstances, such as difficult social and psychological circumstances, make people feel anxious, sad, worried, and unable to cope, damaging their physical and psychological health. Thus, if people feel tense too often or the tension goes on for too long, they become more vulnerable to suffer from depression.11 Psychological processes have been found to mediate the impact of familial risk, social circumstances, and life events on mental health.10 Therefore, in order to prevent depression, it is important to develop strategies to improve these psychological processes through the promotion of adaptive strategies to regulate emotion and resilience in people exposed to high levels of stress. As the European Pact for Mental Health and Well-being states, prevention of depression is a key issue and a health priority.12

There are evidence-based psychological interventions for depression.13 In fact, depression can be treated effectively with pharmacological interventions, psychotherapy, especially cognitive behavioral therapy (CBT), or a combination of the two.14,15 However, these interventions have an important limitation: the mental health services provided are generally less than adequate in terms of accessibility and quality.16 Less than 50% of people with depression are treated by a health professional or general practitioner, and only a quarter receive appropriate treatment, because of the costs and demands of face-to-face treatments, the time required for their application, or the lack of well-trained professionals.17,18 Therefore, individual psychotherapy, the dominant model in providing services, is not likely to meet these needs.16,19 Information and communication technologies (ICTs), especially with the use of the Internet to support the implementation of interventions, have proven to be a powerful vehicle for their effective deployment in providing general mental healthcare.18,20 Today, Internet-based interventions seem to be a very promising alternative to current routine treatment strategies for depression.21 Meta-analyses have demonstrated the clinical effectiveness of Internet-based treatments for depression.22,23 They have been shown to be a feasible solution and a useful strategy to promote dissemination, solve accessibility problems, and facilitate the delivery of interventions.19,24 According to the stepped-care model considered by the National Institute for Health and Care Excellence guidelines for people with sub-threshold depressive symptoms or mild to moderate depression (step 2), it is important to consider offering individual guided self-help based on the principles of CBT or computerized CBT.25

An additional benefit of Internet-based interventions is that therapist time is greatly reduced compared with face-to-face therapy.26 Although data on Internet-based interventions are consistent and promising, this field is still new,27 and there are important problems to solve. For instance, it is unclear which patients can benefit the most from Internet-based interventions, and dropout rates are higher than those in face-to-face treatment. Dropout rates in Internet-based treatment programs ranges from 2% to 83%, with a median of 19% and a weighted average of 31%,28 which are due to various causes, one of which may be human support not being provided.29 Recently, a growing amount of research has been conducted to determine the role of human support in these interventions, and the literature shows the importance of providing this support.23 Meta-analytic studies have found that Internet-based treatments with support produce greater effect sizes and lower dropout rates than Internet-based programs without any support.23,30 Higher dropout rates have been found in unguided web-based interventions for depression, compared with guided web-based interventions, with average adherence levels estimated at 26% in unguided interventions and 72% in guided interventions.23 However, studies usually refer to support that is administered by a person (therapist, consultant, or researcher), rather than support that can be provided automatically by the system through ICTs (automated reminders, feedback provided by the program, emails, or mobile SMS).23 Unguided interventions have been shown to be much easier to implement and less costly than guided Internet-based interventions,31 and hence, it is important to continue to study their effectiveness. As Newman et al32 noted, minimal contact therapies have been proposed as effective and low-cost interventions for anxiety and mood disorders. According to these authors, it is essential to continue to study whether human support is necessary for therapeutic efficacy and differentiate this human support (which consumes more resources) from the automated support provided by the system itself.

Taking all of these into consideration, the main objective of the present study is to investigate the efficacy of an Internet-based program for depressive symptoms applied using only ICT-based automated support or using this same kind of support, but enhanced by human support. To do so, an Internet-based program designed to provide automated support and feedback to patients throughout the intervention was developed, and a randomized control trial (RCT) with three experimental conditions was performed: two intervention groups (one with ICT support alone and the other with ICT support plus human support) and a waiting list control group. The main hypothesis is that the participants in both the intervention groups, regardless of the type of support received (ICT-based automated support or ICT plus human support), will show significant improvements compared with the control group, and the improvements will be maintained at the 12-month follow-up. Furthermore, it was hypothesized that adherence and satisfaction will be high in both the intervention conditions.

Patients and methods

Participant characteristics and sampling procedures

Participants were recruited through announcements in the university community, advertisements in the media (Internet and newspapers), and posters in the Valencia area (Spain). Sample selection and recruitment began in March 2012 and ended in December 2012. People who were interested contacted the authors through phone or email. Inclusion criteria were age between 18 and 65 years, willingness to participate in the study, ability to use a computer and having an Internet connection at home, ability to understand and read Spanish, currently experiencing at least one stressful event in their lives that produced interference, and having depressive symptoms (score of ≤28 on the Beck Depression Inventory-II [BDI-II]). Exclusion criteria were currently receiving psychological treatment, having received another psychological treatment in the past year, and having a severe Axis I mental disorder: abuse or dependence on alcohol or other substances, psychotic disorder or dementia, and the presence of suicidal ideation or plan (evaluated by Mini-International Neuropsychiatric Interview [MINI]). The study was approved by the Ethics Committee of Jaume I University.

The total final study sample was composed of 124 participants, mostly women (67.7%). The mean age was 35.60 years (standard deviation [SD]: 9.70; range: 20–59 years). Table 1 provides additional demographic information of the participants.

Table 1

Demographic characteristics of participants at pre-assessment

Variables	IG	IGHS	WL	Total
Gender
Male	13 (36.1%)	15 (34.1%)	12 (27.3%)	40 (32.3%)
Female	23 (63.9%)	29 (65.9%)	32 (72.7%)	84 (67.7%)
Marital status
Single	20 (55.6%)	23 (52.3%)	20 (45.5%)	63 (50.8%)
Married/with partner	12 (33.3%)	19 (43.2%)	20 (45.5%)	51 (41.1%)
Divorced	4 (11.1%)	2 (4.5%)	4 (9.1%)	10 (8.1%)
Education level
High school graduate or less	1 (2.8%)	3 (6.8%)	3 (6.8%)	7 (5.6%)
Some college	9 (25%)	12 (27.3%)	7 (15.9%)	28 (22.6%)
College graduate or graduate degree	26 (72.2%)	29 (65.9%)	34 (77.3%)	89 (71.8%)
Age
Rank	20–59	20–53	21–58	20–59
M	35.22	35.05	36.48	35.60
SD	9.70	9.36	10.17	9.70

Abbreviations: M, mean; SD, standard deviation; IG, intervention group (N=36); IGHS, intervention group with human support (N=44); WL, waiting list control condition (N=44).

All the participants experienced one or more stressful events in their lives which produced interference, and many of them had clinically significant depressive symptoms. The most common stressful event was unemployment (83.6%). The sample also presented other stressful situations (unemployed relatives [63.7%], debts [47.58%], disease [own or relatives, 62.90%], conflicts [work/family, 64.52%]).

In addition to experiencing these complicated situations with low mood and depression symptoms, 33 participants met the emotional disorder (ED) criteria, based on the Diagnostic and Statistical Manual of Mental Disorders, 4th edition, Text Revision (DSM-IV-TR) criteria (American Psychiatric Association):33 14 participants had depression (mild to moderate, 8 of them only had depression and 6 had comorbid depression with another anxiety disorder); 4 participants had dysthymia (3 of them had only dysthymia and 1 had comorbid dysthymia with another anxiety disorder); and 15 participants had anxiety disorder combined with mild to moderate depressive symptoms (13 of them had only one diagnosis and 2 of them had a comorbid diagnosis).

Sample size, power, and precision

To collect the sample, power calculations indicated that a sample size of 26 participants in each group (78 in all) would be sufficient to detect an effect size of d =0.70 with a power of 80%, which was the minimum expected, based on a similar study.34 As Internet interventions tend to result in relatively high dropout rates of up to 30%,24 this study aimed for 34 participants in each group, that is, taking into account the three experimental conditions, at least 102 participants in all.

Intervention

A manualized treatment protocol called Sonreír es Divertido (in English: Smiling is Fun) was developed, which included therapeutic components of evidence-based treatments and had previously been tested in another study within the framework of the European online predictive tools for intervention in mental illness project.35 The program is based on the transdiagnostic perspective. Specifically, it has some of the components of the unified protocol: motivation, psychoeducation, cognitive therapy, and relapse prevention.8 Furthermore, it incorporates a behavioral activation component.36 The program also includes a component of positive psychology, offering strategies to promote and enhance positive mood.37,38 The whole protocol stresses the importance and benefits of being active and being involved in life, values, and goals. It allows the individual to learn and practice adaptive ways to cope with depressive and anxiety symptoms and confront daily problems. The protocol was adapted to a completely self-help Internet-based, multimedia (video, image, etc), interactive program. It is designed for optimal use on a personal computer, but it can also be used on a tablet (https://www.sonreiresdivertido.com/).

Intervention protocol modules

The intervention protocol consists of eight interactive modules. As Table 2 shows, each module has specific objectives (for more information about the program, see Botella et al35). The program has two additional modules: the “Home module” (Figure S1), which explains who we are, the terms and conditions, the goal of Smiling is Fun and who can benefit from it; the “Welcome module”, which explains the content of each module and how to benefit from it.

Table 2

Specific objectives, content, and exercises of intervention modules

Intervention modules	Main objective	Module content	Exercises
1. Motivation for change	To analyze the advantages and disadvantages of change	Motivation and its importance for treatmentAmbivalence as a natural part of the process of changeIdentifying your goals	Fill out the Decisional Balance Board and reflect on the pros and cons of changeEstablish specific objectives and manageable goals
2. Understanding emotional problems	To provide information so that the user can understand emotional problems	Stress, depression, and other emotional responsesWays to respond to stressful events I: refusal or negation of problemsWays to respond to stressful events II: acceptance, coping, and resilience	Know some typical responses to adverse events
3. Learning to move on	To teach the importance of “moving on” to acquire a proper level of activity and involvement in life	The role of the activity in our well-beingThe importance to dare and to get involved in lifeTo look for social support to move on	Know why it is important to register the activities carried out every dayComplete the diary of daily activities every day
4. Learning to be flexible	To teach a more flexible way of thinking	Our interpretation of the situations influences our feelingsIdentify the tricks of our thoughts and learn to make our way of thinking flexible	Practice the identification of Erroneous Thoughts and CognitiveFlexibilityComplete the diary of daily activities every day
5. Learning to enjoy	To promote involvement in pleasant and significant activities and in having contact with other people	The role of positive emotions in our livesThe concept of “savoring” and the importance of “small things”The induction of positive emotions and experiences in a controlled and systematic wayThe importance of the smile: the Duchenne smile	Divide the week between smile days and non-smile daysSearch for a moment of enjoyment, concentrate, and experience it fully every daySavor those little things every day: it becomes a habitComplete the diary of daily activities every day
6. Learning to live	To understand the importance of identifying the individual’s own psychological strengths and carrying out meaningful activities linked to values and goals in life	The concept of well-being and psychological “strengths”Values and goals: identify important vital values and goals	Select and record activities linked to values and significant areas of lifeFocus on the present and catch the good timesComplete the diary of daily activities every day
7. Living and learning	To develop an action plan to boost the individual’s psychological strengths	What strengths are important to you? What strengths do you want to strengthen?Other strategies to develop your strengthsPsychological well-being as a result of being active and practicing all the strategies learned	Gratitude:3 pluses: record once a day three things you are thankful for and why. Re-read and experience that emotion every nightYour best memory: remember something positive and rewarding.Thank life for having itVisit of gratitude: thank you letter to someone else. Experience the feeling shared with the other personGetting to express gratitude: think of a person you want to thank (eg, gift, kiss, hug, etc)Curiosity: search for active interest topics or activitiesHope: encourage optimism and positive affect by writing a short story about your ideal futureDevote 5 min every night to visualizing itComplete the diary of daily activities every day
8. From now on, what else…?	To strengthen what was learned during the program	What have I learned?How do I want my future to be?How do I want my life to be?	Summarizing the things learned in each moduleCarrying out the exercise: what goals would I like to achieve?

ICT-support transversal tools

The intervention program uses three complementary transversal tools that accompany the user throughout its implementation. These tools provide useful feedback that is important for participants’ motivation and reinforcement: 1) The Activity report for self-monitoring gives users feedback, showing them that there is a relationship between their mood and the activities performed, as well as the benefits of being active. First, users have to rate their mood state, stress, and coping ability on a scale from 0 to 10. Then, they have to indicate their degree of satisfaction with each activity in the past 24 h, and how much these activities are related to their life values and goals. Furthermore, users have to indicate what percentage of the day they have been actively involved in their lives (Figure S2). 2) The Calendar allows users to know what step of the program they are on, and it gives them information about homework and tasks already completed, reminding them about those still pending. Furthermore, the Calendar indicates which days each participant has completed the activity report (Figure S3). 3) “How am I?” provides a set of graphs and feedback to chart the user’s progress, showing information about the activity level, emotional distress (anxiety and sadness), positive emotionality (active, enthusiastic, energetic, etc), and negative emotionality (angry, fearful, stressed, tense, moody, etc). This tool allows participants to graphically visualize the relationships among their activity level, mood state, stress, and coping ability (Figure S4). Regarding the intervention use, participants progressed sequentially through the intervention program in a completely self-applied way over the Internet at their own pace, but they were told that they would obtain the most benefit from the program if they did about one module per week. This is the time each module was designed to be completed. The participants knew that they had a maximum of 12 weeks for completing all eight modules, because of the individual differences in everyone’s rhythm. During the progress of the intervention, when they finished one module, if they wanted they were allowed to look over it again (Figure S5). In the control group, the participants completed the pre-assessment, and 12 weeks later, the post-assessment, after which, they can also access the program.

Structured support protocol

We developed a structured protocol for the type of support provided to participants during the implementation of the program (Figure S5 and Supplementary material). All the participants received ICT support. Furthermore, one group of participants received human support as well. The ICT support consisted of two weekly, automated mobile phone messages encouraging them to proceed with the program and reminding them of the importance of doing the tasks in each module and the Activity report every day (Figure 1). Several different alternating messages were sent (Supplementary material). Furthermore, the participants received an automated email encouraging them to continue with the modules if they had not accessed the program for a week. In addition to this automated support, the program offered continued feedback to users through the transversal tools described earlier.

Figure 1

Structured protocol for the type of support provided to participants.

Abbreviation: ICT, information and communication technology.

The human support consisted of a 2 min, weekly support call by the therapist (once a week, with no clinical content). The objective of this brief call is to tell the participants whether they are making good progress through the program modules (motivating the participants if they advance slowly or offering positive reinforcement if they advance at a good pace, etc) and to indicate the importance of doing the tasks in each module and the Activity report every day (Figure 1 and Supplementary material). It should be emphasized that the content of the support was not clinical, and the therapist received training in this type of support.

Therapists

All the therapists (4) involved in the study were experienced clinical psychologists (with at least a master degree or PhD) trained in CBT and with extensive experience in treatment using Internet-based interventions. The clinical psychologists applied the MINI by telephone to establish each participant’s clinical diagnosis under the supervision of two senior clinical psychologists. They also made the two weekly support phone calls to the participants in the “human support” group. They were trained in the use of the “human support protocol” developed in the study.

Measures

The measures used in the study are diagnostic interview (at pre-assessment, by phone or face to face at university clinical center, depending on whether or not participants were close to the university clinical center).

MINI

This is a short, structured diagnostic psychiatric interview that yields key DSM-IV and ICD-10 diagnoses.39 It has excellent interrater reliability, ranging from 0.88 to 1.00.40 It has been translated and validated in Spanish.41 Participants were interviewed by telephone to determine their current ED diagnosis using the MINI, as in previous studies on Internet-based interventions.42

Primary outcome measure (at pre-, post-, and follow-up assessment, over the Internet) BDI-II

The BDI-II is a 21-item self-report scale of depressive symptoms.43 The questionnaire has shown good psychometric properties in several studies.44 The Spanish version of the BDI-II was used.45 It shows good reliability and validity data and provides a bifactorial solution that matches what was found in previous studies.45 It has shown high internal consistency for both general (α=0.87) and clinical populations (α=0.89).45 Cronbach’s alpha in the present study was excellent (α=0.88).

Secondary outcome measures (at pre-, post-, and follow-up assessment, over the Internet)

Overall Anxiety Severity and Impairment Scale (OASIS)

The OASIS46 consists of five items that measure the frequency and severity of anxiety, as well as the level of avoidance, work/school/home interference, and social interference associated with anxiety. It has been found to have excellent test–retest reliability, in addition to good convergent and discriminant validity and high internal consistency (α=0.80).46 The OASIS was translated to Spanish, and a validation process as performed.47 The validation data confirmed the factorial structure and reliability and the validity data obtained by the original authors.46 Cronbach’s alpha in the present study was excellent (α=0.87).

Overall Depression Severity and Impairment Scale (ODSIS)

The ODSIS is a self-report measure with five items that evaluate experiences related to depression.7 It measures the frequency and severity of depression, as well as the level of avoidance, work/school/home interference, and social interference associated with depression. It was found to have good convergent and discriminant validity and excellent internal consistency (α=0.94 in the outpatient sample, 0.91 in the student sample, and 0.92 in the community sample).7 The ODSIS was translated to Spanish, and a validation process was performed.48 The validation data confirmed the factorial structure and reliability and the validity data obtained by the original authors.7 Cronbach’s alpha in the present study was excellent (α=0.92).

Positive and Negative Affect Schedule (PANAS)

The PANAS consists of 20 items that evaluate two independent dimensions: positive affect (PANAS+) and negative affect (PANAS−). The range for each scale (10 items on each) is from 10 to 50.49 It is a brief, reliable, and valid self-report measure. It has shown excellent convergent and divergent validity.49 The validation of the Spanish PANAS revealed, like the original version, a robust and stable two-dimensional structure (PANAS+ and PANAS−) and provided strong support for its validity and reliability (internal consistency: 0.89–0.91 for PANAS+ and PANAS− in women and 0.87 for PANAS+ and 0.89 for PANAS− in men).50 In the present study, Cronbach’s alphas for PANAS+ (α=0.94) and PANAS− (α=0.88) were excellent.

Perceived Stress Scale (PSS)

The PSS is a 14-item self-report questionnaire that assesses the degree to which recent life situations are appraised as stressful.51 In the present study, a 4-item PSS (PSS-4) was used. This PSS-4 was introduced as a brief version for situations requiring a very short scale or telephone interviews.52 It has been validated in different studies, showing an internal consistency reliability of 0.76–0.82.53 The Spanish validation of the PSS-14 and PSS-10 had good psychometric proprieties,54 but the Spanish validation of the PSS-4 did not; however, it was used because it was administrated over the Internet. Cronbach’s alpha in the present study was α=0.65.

Treatment acceptance measures (expectations at the end of module 2 and satisfaction at post assessment, over the Internet)

Treatment expectations and satisfaction scales

This questionnaire measures participants’ expectations before treatment and their satisfaction after receiving it.55 It includes five items rated from 0 (“not at all”) to 10 (“very much”); the questions address how logical the treatment seems, to what extent the patient is expected to be satisfied with it, whether the patient would recommend the treatment to others, whether it would be useful in treating other problems, and the treatment’s usefulness for the patient’s problem. Participants answered the expectations scale after module 2 of the program. The satisfaction scale was completed once the treatment was over. This adaptation has been used in previous studies.56,57

Research design

This is a RCT with three experimental conditions. 1) Intervention group with ICT support (IG): participants had access to the Internet-based intervention program and received only ICT-based automated support. 2) Intervention group with ICT-based automated support plus human support (IGHS): participants had access to the Internet-based intervention program with the same intervention modules and the same transversal tools as the other group, and they received ICT support plus human support. The participants in both the intervention groups had a maximum of 12 weeks to finish the intervention program. 3) Waiting list control group (WL): the participants waited 12 weeks, and then they were offered the chance to receive the treatment. If they agreed, they could access the program and were randomly assigned to one of the intervention conditions (IGHS or IG). Measurements were obtained at three time points: prior to the treatment, at posttreatment, and at 12-month follow-up.

The random assignment of the participants to the different experimental conditions was generated by an independent researcher who had no knowledge about the study or the intervention received in the different groups. This investigator performed the randomized assignment according to a randomization list created by the Random Allocation Software, version 1.0. Participants were stratified based on the severity of depressive symptoms. The study was registered in the National Institute of Health Registration System of the United States of America with Clinical Trial Registration Number: NCT02148354 (http://ClinicalTrials.gov/ct2/show/NCT02148354). The CONSORT-EHEALTH guidelines were followed to compare the three experimental conditions included in the study.

Statistics and data analysis

Group differences in demographic data and pretreatment measures were analyzed with one-way analysis of variance (ANOVA) and chi-square tests, in order to determine whether there were significant differences among the groups. The outcome analyses were limited to participants who entered the intervention and did the pretreatment assessment. Those who did not enter the intervention or complete the pretreatment assessment were excluded, as their outcomes, positive or otherwise, could not be indicative of the effectiveness of the intervention, following other relevant authors in the field.58 Two sets of analyses were carried out. The first one only examined observed data (completer participants); individuals who did not provide data for the follow-up period were excluded. The second set employed an intention-to-treat (ITT) analysis. Thus, posttreatment and follow-up improvement were investigated separately for the completer and ITT samples. Data considering the ITT sample were also analyzed to study whether the results were similar to those obtained with the completer sample. In order to test treatment efficacy, mixed ANOVAs with repeated measures were used to compare the time effect on the measures (pre- to posttreatment, and pre-, post-, 12-month follow-up) and the time interactions among the treatment conditions. The ANOVAs were followed by Sidak’s corrected post hoc tests when needed. Because differences in the three experimental groups were found at pretreatment on the PSS variable, they were analyzed using univariate analyses of covariance (ANCOVAs), assigning pretreatment scores as the covariate. Effect sizes (Cohen’s d) were calculated for within- and between-group changes, based on the pooled SD. The expectation and satisfaction scores were also analyzed by using mixed ANOVAs with repeated measures. Correlation analysis was performed to explore the relationship between adherence and the different outcome measures at posttreatment. All statistical analyses were conducted by using IBM SPSS Statistics 20 (IBM Corporation, Armonk, NY, USA).

Procedure

People interested in the study contacted us by phone or by email. Possible participants were prescreened through phone calls to filter some crucial inclusion and exclusion criteria: age, receiving psychological treatment or having received another psychological treatment in the past year, being able to use a computer, having an Internet connection at home, and being able to understand and read Spanish. Participants who passed the prescreening through phone were given a date for an initial face-to-face interview (or through phone, depending on where the participant lived), where a research team member explained the study. Participants interested in participating signed an informed consent form (in person or on the Internet). Subsequently, the investigator asked for demographic information, administered the BDI-II, determined the current ED diagnosis using the MINI, and gathered further information about inclusion and exclusion criteria. The selected participants were then randomly assigned to one of three experimental conditions, and the participants in the two intervention conditions were given access to the program. With their username and password, they could access the program and perform the pretreatment assessment integrated in the web system (the participants in the WL condition had the same evaluation outside the system, but also over the Internet using Survey Monkey). The IGHS participants received a telephone call reminding them to enter the web system to do the pretreatment assessment. In addition, the WL group participants received a telephone call reminding them to do the pretreatment assessment.

When participants completed the eight treatment modules, they performed the posttreatment evaluation, also integrated in the web system (same self-report questionnaires as in the pretreatment assessment, plus the treatment satisfaction scale). They had a maximum of 12 weeks to complete the intervention program.

Finally, 12 months after the posttreatment assessment, the participants completed the 12-month follow-up assessment. Participants in the control condition performed the pretreatment assessment, but they had no access to the program content and did not receive any intervention for 12 weeks. After this waiting period, they were assessed again, and, if they wished to, they could access the program and were randomly assigned to one of the two intervention conditions.

Results

Participant flow

Regarding the flow of participants throughout the investigation process, Figure 2 shows that 481 people were interested in the study. They were contacted through telephone (prescreening) to explain their participation in the study and filter some crucial criteria, described in the Procedure section. After this phone call, 204 were not interested or did not answer the phone call, and 72 did not meet some of the minimum criteria tracked by phone. Thus, 205 people were invited to participate and performed the initial interview, but 73 of them did not meet the inclusion and exclusion criteria. Finally, 132 participants were assigned to the three experimental conditions: IG, n=44; IGHS, n=44; WL, n=44. We had pretreatment assessments for 36 participants in the IG, 44 participants in the IGHS, and 44 participants in the WL. During the intervention program, 8 participants dropped out of the IG group and 8 dropped out of the IGHS group.

Figure 2

Flowchart of study.

Abbreviations: IG, intervention group; IGHS, intervention group with human support; WL, waiting list control condition.

Pretreatment data

Table 1 shows the demographic characteristics of each experimental condition at pretreatment. The results indicated that there were no significant differences between the experimental groups before treatment on any of these variables: gender (χ2 [1] =0.813; P=0.666); marital status (χ2 [2] =2.351; P=0.671); educational level (χ2 [2] =2.526; P=0.640); and age (F(2,121) =0.276; P=0.759).

Regarding the clinical characteristics of the participants in each experimental condition at pretreatment (Table 3), the data indicated that there were no significant differences among the three experimental groups on any of the variables before treatment, with the exception of perceived stress (F(2,121) =8.53, P<0.001). In relation to this variable, the post hoc analysis (Tukey’s HSD) revealed that the WL group was significantly different from the other two experimental conditions (IG and IGHS). That is, the WL group had a statistically significant higher score on the PSS variable than the two intervention groups, but these two intervention groups were not significantly different from each other. As noted earlier, all the participants (in the three experimental conditions) experienced at least one stressful event in their lives.

Table 3

Clinical characteristics of participants at pre-assessment

Dependent variable	Condition	M	SD	F(2,121)	P-value
BDI-II	IG	9.14	7.66	0.84	0.436
	IGHS	10.91	6.96
	WL	9.11	7.47
	Total	9.76	7.34
OASIS	IG	4.06	3.65	2.02	0.137
	IGHS	4.43	3.66
	WL	3.00	3.04
	Total	3.81	3.47
ODSIS	IG	2.53	3.16	1.10	0.336
	IGHS	3.30	3.52
	WL	2.27	3.34
	Total	2.71	3.35
PANAS+	IG	29.22	8.72	0.47	0.625
	IGHS	27.86	6.91
	WL	29.34	7.92
	Total	28.78	7.79
PANAS−	IG	18.11	6.85	1.00	0.372
	IGHS	19.50	5.95
	WL	17.68	6.10
	Total	18.45	6.28
PSS	IG	5.86	2.74	8.53	<0.001
	IGHS	6.25	3.24
	WL	8.07	1.59
	Total	6.78	2.76

Abbreviations: M, mean; SD, standard deviation; IG, intervention group (N=36); IGHS, intervention group with human support (N=44); WL, waiting list control condition (N=44); BDI-II, Beck Depression Inventory-II; OASIS, Overall Anxiety Severity and Impairment Scale; ODSIS, Overall Depression Severity and Impairment Scale; PANAS+, Positive Affect Scale; PANAS−, Negative Affect Scale; PSS, Perceived Stress Scale.

Differential effectiveness of three experimental conditions: change in primary and secondary outcomes pre- and posttreatment

Means, SDs, within-group and between-group effect sizes, and confidence intervals for all the outcome measures in the three experimental groups are summarized in Table 4, for both completer and ITT analyses.

Table 4

Means, SDs, within-group, and between-group effect sizes for the outcome measures at pre- and posttreatment (completer and ITT sample)

		N; mean (SD)		Within-group effect size, d (95% CI)	Between-group effect size, d (95% CI)
		Pretreatment	Posttreatment
Completers at posttest (N = 108)
BDI-II	IG	28; 8.32 (7.13)	28; 3.04 (2.74)	0.72 (0.17; 1.26)	IG vs IGHS	0.40 (−0.90; 0.10)
	IGHS	36; 10.36 (7.28)	36; 4.56 (4.39)	0.78 (0.40; 1.16)	IG vs WL	0.90 (−1.40; −0.40)
	WL	44; 9.11 (7.47)	44; 8.45 (7.32)	0.09 (−0.10; 0.28)	IGHS vs WL	0.62 (−1.74; −0.17)
	Total	108; 9.32 (7.30)	108; 5.75 (5.93)
OASIS	IG	28; 3.64 (3.68)	28; 1.46 (2.13)	0.58 (0.13; 1.01)	IG vs IGHS	0.19 (−0.29; 0.69)
	IGHS	36; 4.28 (3.87)	36; 1.08 (1.73)	0.81 (0.32; 1.29)	IG vs WL	0.57 (−1.06; −0.09)
	WL	44; 3.00 (3.04)	44; 3.34 (3.77)	0.11 (−0.32; 0.10)	IGHS vs WL	0.74 (−1.20; −0.28)
	Total	108; 3.59 (3.51)	108; 2.10 (2.99)
ODSIS	IG	28; 2.29 (3.16)	28; 0.54 (1.32)	0.54 (0.02; 1.05)	IG vs IGHS	0.11 (−0.60; 0.38)
	IGHS	36; 3.17 (3.68)	36; 0.72 (1.75)	0.65 (0.19; 1.10)	IG vs WL	0.57 (−1.06; −0.09)
	WL	44; 2.27 (3.34)	44; 2.23 (3.57)	0.01 (−0.22; 0.25)	IGHS vs WL	0.51 (−0.96; −0.07)
	Total	108; 2.57 (3.41)	108; 1.29 (2.68)
PANAS+	IG	28; 30.32 (8.91)	28; 32.50 (8.51)	0.24 (−0.18; 0.65)	IG vs IGHS	0.05 (−0.44, 0.54)
	IGHS	36; 28.19 (6.96)	36; 32.97 (8.24)	0.67 (0.29; 1.04)	IG vs WL	0.32 (−0.80; 0.15)
	WL	44; 29.34 (7.92)	44; 29.91 (7.58)	0.07 (−0.31; 0.46)	IGHS vs WL	0.38 (−0.83; 0.06)
	Total	108; 29.21 (7.86)	108; 31.60 (8.10)
PANAS−	IG	28; 18.07 (7.19)	28; 13.04 (3.32)	0.68 (0.22; 1.13)	IG vs IGHS	0.04 (−0.53; 0.45)
	IGHS	36; 19.03 (6.19)	36; 13.19 (3.82)	0.92 (0.51; 1.33)	IG vs WL	0.84 (−1.34; −0.35)
	WL	44; 17.68 (6.10)	44; 17.93 (6.80)	0.04 (−0.44; 0.36)	IGHS vs WL	0.82 (−1.28; −0.37)
	Total	108; 18.23 (6.40)	108; 15.08 (5.63)
PSS	IG	28; 5.64 (2.73)	28; 4.25 (2.35)	0.49 (0.13; 0.85)	IG vs IGHS	0.02 (−0.52; 0.46)
	IGHS	36; 6.03 (3.32)	36; 4.33 (3.23)	0.50 (0.13; 0.86)	IG vs WL	1.92 (−2.49; −1.35)
	WL	44; 8.07 (1.59)	44; 8.09 (1.70)	0.01 (−0.31; 0.29)	IGHS vs WL	1.49 (−1.98; −0.99)
	Total	108; 6.76 (2.78)	108; 5.84 (3.08)
ITT at posttest (N =124)
BDI-II	IG	36; 9.14 (7.66)	36; 5.03 (6.09)	0.52 (0.18; 0.86)	IG vs IGHS	0.20 (−0.63; 0.25)
	IGHS	44; 10.91 (6.96)	44; 6.16 (5.61)	0.67 (0.35; 0.98)	IG vs WL	0.50 (−0.94; −0.05)
	WL	44; 9.11 (7.47)	44; 8.45 (7.32)	0.09 (−0.10; 0.28)	IGHS vs WL	0.34 (−0.76; 0.07)
	Total	124; 9.76 (7.34)	124; 6.65 (6.50)
OASIS	IG	36; 4.06 (3.65)	36; 1.92 (2.52)	0.57 (0.21; 0.93)	IG vs IGHS	0.10 (−0.33; 0.54)
	IGHS	44; 4.43 (3.66)	44; 1.66 (2.47)	0.74 (0.40; 1.08)	IG vs WL	0.43 (−0.88; 0.02)
	WL	44; 3.00 (3.04)	44; 3.34 (3.77)	0.11 (−0.33; 0.11)	IGHS vs WL	0.52 (−0.95; −0.09)
	Total	124; 3.81 (3.47)	124; 2.33 (3.08)
ODSIS	IG	36; 2.53 (3.16)	36; 0.86 (1.78)	0.52 (0.12; 0.90)	IG vs IGHS	0.25 (−0.69; 0.19)
	IGHS	44; 3.30 (3.52)	44; 1.43 (2.56)	0.52 (0.13; 0.90)	IG vs WL	0.50 (−0.95; −0.05)
	WL	44; 2.27 (3.34)	44; 2.23 (3.57)	0.01 (−0.22; 0.24)	IGHS vs WL	0.29 (−0.70; 0.13)
	Total	124; 2.71 (3.35)	124; 1.55 (2.82)
PANAS+	IG	36; 29.22 (8.72)	36; 31.31 (9.08)	0.23 (−0.10; 0.57)	IG vs IGHS	0.03 (−0.41; 0.47)
	IGHS	44; 27.86 (6.91)	44; 31.61 (9.35)	0.53 (0.23; 0.83)	IG vs WL	0.17 (−0.61; 0.27)
	WL	44; 29.34 (7.92)	44; 29.91 (7.58)	0.07 (−0.31; 0.45)	IGHS vs WL	0.20 (−0.62; 0.22)
	Total	124; 28.78 (7.79)	124; 30.92 (8.64)
PANAS−	IG	36; 18.11 (6.85)	36; 13.81 (3.66)	0.61 (0.22; 1.00)	IG vs IGHS	0.08 (−0.51; 0.36)
	IGHS	44; 19.50 (5.95)	44; 14.14 (4.68)	0.89 (0.53; 1.24)	IG vs WL	0.72 (−1.18; −0.27)
	WL	44; 17.68 (6.10)	44; 17.93 (6.80)	0.04 (−0.44; 0.36)	IGHS vs WL	0.64 (−1.72; −0.21)
	Total	124; 18.45 (6.28)	124; 15.39 (5.59)
PSS	IG	36; 5.86 (2.74)	36; 4.78 (2.62)	0.39 (0.01; 0.75)	IG vs IGHS	0.02 (−0.47; 0.41)
	IGHS	44; 6.25 (3.24)	44; 4.86 (3.32)	0.42 (0.09; 0.74)	IG vs WL	1.51 (−2.01; −1.02)
	WL	44; 8.07 (1.59)	44; 8.09 (1.70)	0.01 (−0.30; 0.28)	IGHS vs WL	1.20 (−1.66; −0.75)
	Total	124; 6.78 (2.76)	124; 5.98 (3.05)

Abbreviations: SD, standard deviation; ITT, intention-to-treat; IG, intervention group; IGHS, intervention group with human support; WL, waiting list control condition; BDI-II, Beck Depression Inventory-II; OASIS, Overall Anxiety Severity and Impairment Scale; ODSIS, Overall Depression Severity and Impairment Scale; PANAS+, Positive Affect Scale; PANAS−, Negative Affect Scale; PSS, Perceived Stress Scale; CI, confidence interval.

The analysis to test for mean differences in the different measures was a 3 (group) ×2 (time) mixed ANOVA with repeated measures in the second factor.

Regarding the completer sample, the analyses revealed a significant effect of time on all the outcome measures: BDI-II (F(1,105) =44.63, P<0.001), OASIS (F(1,105) =26.28, P<0.001); ODSIS (F(1,105) =19.58, P<0.001); PANAS− (F(1,105) =22.34, P<0.001) and PANAS+ (F(1,105) =6.50, P=0.012). The effect of group was not statistically significant in any variable. Furthermore, the analyses revealed a significant interaction effect: BDI-II (F(2,105) =8.85, P<0.001), OASIS (F(2,105) =11.87, P<0.001), ODSIS (F(2,105) =5.37, P=0.004), PANAS− (F(2,105) =7.41, P=0.001), and PSS (F(2,105) =5.30, P=0.006), except in the case of PANAS+ (F(2,105) =1.75, P=0.180). Regarding the interactions: Sidak’s corrected post hoc tests indicated that both the intervention groups experienced significant improvements in all variables during the intervention, with the exception of PANAS+. The control group did not improve pre- to posttreatment on any of the variables. Regarding the PSS variable, an ANCOVA was performed to control for differences found among the three experimental groups at pretreatment. Covariate analysis indicated that the groups differed significantly at posttreatment (F(2,104) =18.005, P<0.001). Sidak’s post hoc test indicated that the two intervention groups significantly improved their perceived stress pre- to posttreatment; nevertheless, the control group did not improve pre- to posttreatment on this variable. Regarding the ITT sample, the analysis revealed the same pattern of results as in the completer sample. They showed a significant effect of time on all variables: BDI-II (F(1,121) =35.98, P<0.001), OASIS (F(1,121) =26.35, P<0.001); ODSIS (F(1,121) =16.62, P<0.001); PANAS+ (F(1,121) =5.93, P=0.016), and PANAS− (F(1,121) =22.29, P<0.001). The effect of group was not statistically significant on any variable. Furthermore, analysis revealed a significant interaction effect on these measures: BDI-II (F(2,121) =6.09, P=0.003), OASIS (F(2,121) =10.86, P<0.001), ODSIS (F(2,121) =4.08, P=0.019), PANAS− (F(2,121) =7.10, P<0.001), except in the case of PANAS+ (F(2,121) =1.18, P=0.310). Sidak’s post hoc tests indicated that both the intervention groups experienced significant improvements in all variables. This improvement was not found in the waiting list control group. Regarding the PSS variable, an ANCOVA was performed to control for differences found among the three experimental groups at pretreatment. Covariate analysis indicated that the groups differed significantly at posttreatment (F(2,121) =11.908, P<0.001). Sidak’s post hoc test indicated that the two intervention groups significantly improved their perceived stress pre- to posttreatment, whereas the control group did not improve pre- to posttreatment on this variable.

Differential effectiveness of the two intervention conditions in maintaining the improvements at 1-year follow-up

Means and SDs for all the outcome measures in the two intervention groups at pre-post and follow-up (12 months) are summarized in Table 5.

Table 5

Means, SDs, within-group and between-group effect sizes for the outcome measures at pretreatment, posttreatment, and 12-month follow-up (completer and ITT sample)

		N; mean (SD)			Within-group effect size, d (95% CI)	Between-group effect size, d (95% CI)
		Pretreatment	Posttreatment	Follow-up (12 months)	Pre-follow-up (12 months)	Group	Follow-up (12 months)
Completers at 12-month follow-up (N =45)
BDI-II	IG	20; 8.30 (7.26)	20; 2.60 (2.78)	20; 3.25 (4.10)	0.67 (0.11; 1.21)	IG vs IGHS	0.15 (−0.74; 0.44)
	IGHS	25; 10.52 (7.56)	25; 4.80 (4.59)	25; 4.04 (5.69)	0.83 (0.36; 1.29)
	Total	45; 9.53 (7.42)	45; 3.82 (4.01)	45; 3.69 (5.01)
OASIS	IG	20; 3.35 (3.35)	20; 1.45 (2.04)	20; 1.65 (2.58)	0.49 (−0.13; 1.10)	IG vs IGHS	0.14 (−0.73; 0.45)
	IGHS	25; 3.84 (3.84)	25; 0.80 (1.58)	25; 2.04 (2.95)	0.45 (−0.05; 1.02)
	Total	45; 3.62 (3.62)	45; 1.09 (1.81)	45; 1.87 (2.77)
ODSIS	IG	20; 2.30 (2.72)	20; 0.40 (.883)	20; 0.85 (2.13)	0.51 (0.00; 1.02)	IG vs IGHS	0.17 (−0.76; 0.42)
	IGHS	25; 3.08 (3.99)	25; 0.64 (1.440)	25; 1.24 (2.31)	0.45 (−0.02; 0.92)
	Total	45; 2.73 (3.47)	45; 0.53 (1.217)	45; 1.07 (2.22)
PANAS+	IG	20; 29.95 (10.25)	20; 32.25 (9.65)	20; 34.50 (6.79)	0.43 (−0.08; 0.93)	IG vs IGHS	0.23 (−0.83; 0.35)
	IGHS	25; 28.04 (6.01)	25; 32.08 (8.07)	25; 32.92 (6.34)	0.79 (0.33; 1.23)
	Total	45; 28.89 (8.13)	45; 32.16 (8.71)	45; 33.62 (6.52)
PANAS−	IG	20; 17.35 (6.12)	20; 12.60 (3.12)	20; 13.90 (4.28)	0.54 (0.13; 0.95)	IG vs IGHS	0.10 (−0.69; 0.49)
	IGHS	25; 19.16 (6.74)	25; 13.12 (3.81)	25; 14.28 (2.98)	0.70 (0.29; 1.10)
	Total	45; 18.36 (6.46)	45; 12.89 (3.49)	45; 14.11 (3.58)
PSS	IG	20; 5.60 (2.80)	20; 4.10 (2.34)	20; 4.35 (2.62)	0.43 (0.00; 0.86)	IG vs IGHS	0.02 (−0.60; 0.57)
	IGHS	25; 6.20 (3.27)	25; 4.52 (3.19)	25; 4.40 (2.43)	0.53 (0.10; 0.96)
	Total	45; 5.93 (3.05)	45; 4.33 (2.82)	45; 4.38 (2.49)
ITT at 12-month follow-up (N =80)
BDI-II	IG	36; 9.14 (7.66)	36; 5.03 (6.09)	36; 5.61 (6.44)	0.45 (0.13; 0.76)	IG vs IGHS	0.03 (−0.47; 0.40)
	IGHS	44; 10.91 (6.96)	44; 6.16 (5.61)	44; 5.84 (6.51)	0.72 (0.38; 1.52)
	Total	80; 10.11 (6.96)	80; 5.65 (5.82)	80; 5.74 (6.44)
OASIS	IG	36; 4.06 (3.65)	36; 1.92 (2.52)	36; 2.33 (2.88)	0.46 (0.10; 0.82)	IG vs IGHS	0.06 (−0.50; 0.38)
	IGHS	44; 4.43 (3.66)	44; 1.66 (2.47)	44; 2.52 (3.05)	0.51 (0.14; 0.87)
	Total	80; 4.26 (3.63)	80; 1.78 (2.48)	80; 2.44 (2.99)
ODSIS	IG	36; 2.53 (3.16)	36; 0.86 (1.78)	36; 1.11 (2.19)	0.44 (0.12; 0.75)	IG vs IGHS	0.32 (−0.76; 0.12)
	IGHS	44; 3.30 (3.52)	44; 1.43 (2.56)	44; 1.95 (2.89)	0.38 (0.00; 0.75)
	Total	80; 2.95 (3.36)	80; 1.18 (2.24)	80; 1.58 (2.61)
PANAS+	IG	36; 29.22 (8.72)	36; 31.31 (9.08)	36; 32.53 (7.66)	0.37 (0.02; 0.72)	IG vs IGHS	0.08 (−0.51; 0.36)
	IGHS	44; 27.86 (6.91)	44; 31.61 (9.35)	44; 31.93 (7.97)	0.58 (0.26; 0.89)
	Total	80; 28.48 (7.76)	80; 31.48 (9.18)	80; 32.20 (7.79)
PANAS−	IG	36; 18.11 (6.85)	36; 13.81 (3.66)	36; 14.94 (4.21)	0.45 (0.13; 0.77)	IG vs IGHS	0.07 (−0.50; 0.37)
	IGHS	44; 19.50 (5.95)	44; 14.14 (4.68)	44; 15.25 (4.66)	0.70 (0.36; 1.04)
	Total	80; 18.88 (6.37)	80; 13.99 (4.23)	80; 15.11 (4.44)
PSS	IG	36; 5.86 (2.74)	36; 4.78 (2.62)	36; 4.81 (2.80)	0.37 (0.10; 0.64)	IG vs IGHS	0.01 (−0.43; 0.45)
	IGHS	44; 6.25 (3.24)	44; 4.86 (3.32)	44; 4.77 (2.79)	0.45 (0.15; 0.73)
	Total	80; 6.08 (3.01)	80; 4.83 (3.01)	80; 4.79 (2.77)

Abbreviations: SD, standard deviation; ITT, intention-to-treat; IG, intervention group; IGHS, intervention group with human support; BDI-II, Beck Depression Inventory-II; OASIS, Overall Anxiety Severity and Impairment Scale; ODSIS, Overall Depression Severity and Impairment Scale; PANAS+, Positive Affect Scale; PANAS−, Negative Affect Scale; PSS, Perceived Stress Scale; CI, confidence interval.

Regarding the completer sample, the analysis revealed a significant time effect on all measures: BDI-II (F(2,86) =24.58, P<0.001); OASIS (F(2,86) =10.36, P<0.001); ODSIS (F(2,86) =12.46, P<0.001), PANAS+ (F(2,86) =8.01, P<0.001), PANAS− (F(2,86) =30.08, P<0.001), and PSS (F(2,86) =9.47, P<0.001). The analysis did not reveal a significant group effect or a significant interaction effect on any measure. Regarding the ITT sample, the analysis revealed the same pattern of results as the completer sample, a significant time effect on all measures: BDI-II (F(2,156) =29.03, P<0.001), OASIS (F(2,156) =22.94, P<0.001); ODSIS (F(2,156) =14.74, P<0.001), PANAS+ (F(2,156) =10.41, P<0.001), PANAS− (F(2,156) =39.34, P<0.001), and PSS (F(2,156) =13.73, P<0.001). The analysis did not reveal a significant group effect or a significant interaction effect on any measure.

Adherence and modules completed

Regarding adherence, 77.77% of participants completed all the modules in the IG group, and 81.81% of the IGHS group did so. There were no significant differences between the experimental groups in the dropout rate (χ2=0.202; df =1; P=0.653). The average number of modules completed was 7.30 (SD =1.60). The number of modules completed was weakly correlated with posttreatment BDI scores (Pearson’s r=−0.652, P<0.01), OASIS scores (Pearson’s r=−0.480, P<0.01), PANAS− scores (Pearson’s r=−0.382, P<0.01), and PSS scores (Pearson’s r=−383, P<0.01).

Expectations and satisfaction

Table 6 lists the results of both the intervention groups. All the scores were high before the treatment. The analysis did not reveal significant differences between the two groups on any measures: treatment logic (F(1,74) =0.118, P=0.733), treatment satisfaction (F(1,74) =0.345, P=0.559), recommending the treatment to others (F(1,74) =0.061, P=0.806), usefulness of the treatment for other psychological problems (F(1,74) =1.131, P=0.739), and usefulness of the treatment for the patient (F(1,74) =0.112, P=0.739).

Table 6

Means and SDs for expectations and satisfaction

Statements	Expectations N; mean (SD)	Satisfaction N; mean (SD)
How logical do you think this treatment is?
Total sample	76; 7.57 (1.79)	64; 8.14 (1.56)
IG	33; 7.48 (1.64)	28; 8.21 (1.77)
IGHS	43; 7.63 (1.92)	36; 8.08 (1.40)
How satisfied are you with the treatment received?
Total sample	76; 7.28 (1.92)	64; 7.88 (1.74)
IG	33; 7.42 (1.79)	28; 7.93 (1.92)
IGHS	43; 7.16 (2.02)	36; 7.83 (1.61)
To what extent do you feel confident recommending this treatment to a friend who has the same problems?
Total sample	76; 7.64 (2.12)	64; 8.33 (1.79)
IG	33; 7.58 (2.03)	28; 8.25 (1.96)
IGHS	43; 7.70 (2.21)	36; 8.39 (1.68)
Do you think this treatment could be useful for treating other
Total sample	76; 7.46 (2.00)	64; 8.09 (1.76)
IG	33; 7.18 (1.98)	28; 8.04 (1.92)
IGHS	43; 7.67 (2.02)	36; 8.14 (1.66)
To what extent do you think the treatment was helpful to you?
Total sample	76; 6.86 (2.21)	64; 7.36 (1.99)
IG	33; 6.76 (2.12)	28; 7.46 (2.03)
IGHS	43; 6.93 (2.31)	36; 7.28 (1.98)

Note: Scale ranging from 0 to 10 with higher scores indicating greater satisfaction.

Abbreviations: SD, standard deviation; IG, intervention group; IGHS, intervention group with human support.

After using the program, participants’ satisfaction scores were also high. The analysis did not reveal significant differences between groups on treatment logic (F(1,62) =0.109, P=0.742), treatment satisfaction (F(1,62) =0.046, P=0.830), recommending the treatment to others (F(1,62) =0.093, P=0.761), usefulness of the treatment for other psychological problems (F(1,62) =0.053, P=0.818), or usefulness of the treatment for the patient (F(1,62) =0.137, P=0.713).

Discussion

The main objective of this study was to investigate the efficacy of an Internet-based program for depressive symptoms with ICT-based automated support and human support. Participants were experiencing one or more stressful events in their lives which produced interference and many of whom had clinically significant depressive symptoms. The results, using both the ITT procedure and the completer analysis procedure, showed that, on the primary outcome measure (BDI-II), there was a significant time effect and a significant interaction effect. The effect sizes on depressive symptomatology pre- to postintervention were medium to large in both the intervention groups. In the WL group, the effect size was minimal.

On all the other measures of negative emotionality (ODSIS, OASIS, and PANAS−), the analysis also revealed a significant time effect and a significant interaction effect, and the same pattern occurred for perceived stress (PSS variable). The effect sizes were medium to large in both the intervention groups on all of these variables (with the exception of PSS in the ITT analysis, which was smaller). In the WL group, the effect size was minimal on all variables. In the case of positive emotionality (PANAS+), the analysis revealed a significant time effect, but not a significant interaction effect. Nevertheless, it is necessary to take into account that the mean scores found for PANAS+ before the treatment were high, falling within the range of the population with no mental disorder diagnosis.59 Even so, the effect sizes showed that PANAS+ improved, and it improved more in both the intervention groups than in the waiting list control group, where the change is almost nonexistent. The results are in line with those presented in systematic reviews, showing that treatments over the Internet and computerized treatments are effective interventions.18,20,22,23

Regarding the support given in the Internet-based interventions, the literature shows that Internet-based programs without any human support seem to have fewer benefits.18 The review by Richards and Richardson23 also reveals that self-applied interventions supported by a therapist (or by administrative staff) work best, showing larger effect sizes. This conclusion also coincides with the results of the review by Johansson and Andersson,60 who analyzed whether different degrees of support can influence the results of Internet-based interventions. Their results showed that more human support produced larger effect sizes.

Other studies have examined the efficacy of self-applied treatments without human support. A meta-analysis20 confirmed the efficacy of self-applied treatments (without any contact or support from a therapist, consultant, or researcher), compared with a control condition, for adults who met the criteria for a depressive disorder. Clarke et al61 did not find differences at posttreatment between the group that received the self-applied treatment plus postcard reminders and another group that received brief telephone calls (5 min) with the same content as the postal reminders. As in this study, these phone calls had no clinical content. Furthermore, modest but statistically significant improvements were found in both the intervention groups compared with the control group. Nevertheless, in the Clark study, the support was provided by postcards and not by ICTs. Farrer et al62 also showed the efficacy of a self-applied treatment in reducing depression with and without weekly telephone calls by a therapist, compared with standard treatment. In this case, the weekly calls (10-min long) had no clinical content either. However, in this study, there was no control group or specific protocol for the ICT support. Berger et al34 compared the efficacy of a self-applied program using two experimental conditions (without any support from the therapist or using weekly therapist support through email) versus a control group. They found that in both the intervention groups, there was a significant reduction in the depression symptoms, compared with the control group. The improvements were maintained at follow-up in both the groups, and no statistically significant differences were found. However, in their study, the therapist support was through email, and there was no ICT-based automated support. Our data provide information about this new and necessary topic, showing the possibilities and potential of automated support in reducing the cost of treatment.

It should be noted that the literature suggests that more support is better, but authors understand “support” to occur only through human contact, and different ICT-based automated strategies for providing support are usually not considered. However, in this study, all the participants received automated mobile phone messages, automated emails, and continued automated feedback from the program.

To the best of our knowledge, only one study differentiates between automated and human support in an RCT about an Internet-based intervention for depression.63 The authors concluded that an automated-support web-based intervention for the treatment of depression with persuasive technology may achieve similar adherence and effectiveness as the same intervention with human support.63 However, in order to draw more definitive conclusions about differences between the two intervention groups, they would need more statistical power, as in the present study. Nevertheless, it is important to highlight that in both the studies, the results are similar: large differences were not found between the two intervention conditions. Our data, as in the Kelders et al’s study,63 suggest that the automated support was useful. The participants in both the intervention groups improved significantly pre- to posttreatment, and this did not occur in the WL group.

In addition, there were no significant differences in adherence between the two conditions (automated and human support), like in an earlier study by Kelders et al.63 A possible explanation for the lack of differences in effectiveness and adherence between the two intervention groups can be found in the richness and quality of the ICT support. First, the two weekly feedback mobile messages were implemented to closely resemble human feedback support. They encouraged the participants to proceed with the program and reminded them of the importance of doing the tasks in each module and the Activity report every day. The fact that there were several different alternating messages sent and that the length and presentation of the messages were similar to feedback from a person further increased their resemblance to human support, agreeing with Kelders et al.63 Second, participants received an automated email encouraging them to continue with the modules if they had not accessed the program for a week, which could increase adherence. Third, and quite important, the program offers continued personalized feedback to users through various transversal tools for providing support as described earlier (Activity report, Calendar, and How am I?).

Regarding the expectations and satisfaction expressed by the two intervention groups, the participants reported positive expectations about the program. Furthermore, participants in both the groups reported feeling very satisfied with the treatment, and they found it very logical and useful, even for other psychological problems. In addition, they would recommend it to a friend, regardless of whether the support had been human or through ICTs, which is important because Internet-based interventions are easier to implement and, therefore, may reach more people in need. It is essential to continue to study other ICT strategies for providing support, which could further improve users’ opinions.

Furthermore, it is possible that not all participants require the same amount (or the same kind) of support, and this assistance may only be needed at critical times during the treatment program.64,65 Considering ICTs’ numerous possibilities, it is important to continue to study these aspects. Automated support through ICTs may also have significant benefits and influence the results, performing a similar function to human support (in our study, the weekly support call, so that the call would not be necessary), and saving even more therapist time. The literature shows that Internet-based, self-applied treatments save therapist time compared with traditional, face-to-face treatments.26 Therefore, it is possible that by using different ICT-based strategies to provide support, therapist time can be further reduced. This represents progress along the lines of Kazdin and Blase’s16 and Emmelkamp et al’s66 proposal: psychotherapy research needs to develop interventions that can be applied to more patients in a simpler and more cost-effective way.

Another objective of our study was to test whether the changes obtained at posttreatment remained until the 12-month follow-up. Regarding the primary outcome measure (BDI-II), the ITT and completer analyses revealed that the significant improvements obtained in both the intervention groups at posttreatment were maintained until the 12-month follow-up. A medium effect size on the BDI-II in both the intervention conditions was obtained by using the ITT analysis. However, a large effect size was obtained by using the completer analysis in the group with human support. These results agree with the Kelders et al63 study, although they present data only until the 6-month follow-up. Regarding other clinical measures, in the ITT analysis, the participants improved significantly pre- to posttreatment on all measures, and the improvements were maintained until the 12-month follow-up, with the exception of PANAS−. The completer analysis revealed the same pattern. These findings add data to the literature on the maintenance of effects achieved by self-administered psychological treatments delivered over the Internet.26,34,67 In this study, the results showed that participants who received only ICT-based automated support maintained their improvements up to the 12-month follow-up. As Richards and Richardson23 pointed out in their meta-analysis, it is important to note the differences obtained between posttreatment and follow-ups in the case of self-applied treatments, as this evolution may suggest that the benefits of computerized interventions are more short-term. Thus, these authors emphasize the need for further research about maintaining the benefits of Internet-based treatments, and the results contribute by providing additional data on maintaining the improvements achieved through these types of intervention programs without any human support. Nevertheless, more research is needed to determine whether reminders could help to prevent relapses and maintain the progress made in self-applied interventions, specifically by using ICT-based support that can be perceived by participants as similar to human support (avatars, messages, images, etc). For example, studies could consider whether receiving automatic weekly emails during follow-up periods would enable participants to practice the strategies learned during the treatment program.65

Furthermore, more research is needed regarding who is providing the support. In the present study, the support was provided by a clinician. However, as the telephone call was without clinical content, it would be interesting in future studies to study the efficacy if the support was delivered by technicians or non-clinicians. The studies are in the line that it is not important who is providing the support (researcher, technical or clinical).18,23,68,69

This study has some limitations. First, there was no enough statistical power to study the real differences between the two intervention groups because an equivalence or non-inferiority trial was not performed. Therefore, it cannot be stated with certainty that automated support provided by ICTs and human support are equivalent in terms of effectiveness and adherence. However, although it was not the main goal, the study was able to explore the trend of the results. In the future, we intend to replicate this study in an adequately powered sample in order to adequately test the non-inferiority hypothesis (whether there are differences between the two active conditions). The study and intervention program focused on improving depressive symptomatology, although in the sample there were participants with a clinical diagnosis. It would have also been interesting to specifically assess those participants who met the criteria for an adjustment disorder because all of them were going through a stressful event (the diagnostic interview used, MINI, does not include this disorder).

Conclusion

Despite limitations, the Internet-based program was effective and well accepted, with and without human support, showing that ICT-based automated support may be useful. It is essential to continue to study other ICT strategies for providing support.

Home module, which explains what the program is about.

Activity report, where the participants indicate their coping ability, mood state, stress, and what percentage of the day they have been active.

Calendar, which shows a check mark the days the participant has done the activity report and reminds the participants the tasks that are still outstanding.

How am I? The part in which the patients can see the relationship between their mood state (yellow line) and the level of their activity (green bars).

Look over the modules. The participants can come back to this page and look over the finished modules.