Effects of an internet-based self-help intervention for psychological distress due to COVID-19: Results of a randomized controlled trial.

BACKGROUND: The COVID-19 pandemic and its far-reaching impact on physical and mental health generate high demand and, accordingly, a great need for treatment opportunities that promote well-being and manage psychological distress. Internet-based interventions are particularly suitable for this purpose. They are easily scalable, readily accessible, and the online format allows for adherence to social distancing. For this reason, we developed an internet-based self-help intervention called ROCO to address psychological distress due to the COVID-19 pandemic. This randomized controlled trial aimed to examine the efficacy of the ROCO intervention.
METHODS: A total of 107 German-speaking adults with at least mild depressive symptoms were randomized either to the intervention group with direct access to the three-week ROCO intervention plus care as usual or the waiting control group receiving care as usual. Primary outcome (depressive symptoms) and secondary outcomes (stress, anxiety, resilience, emotion regulation, health-related quality of life, embitterment, loneliness, optimism, and self-efficacy) were assessed pre- and post-treatment and at a 6-week follow-up using self-report questionnaires (e.g. Patient Health Questionnaire-9 for depressive symptoms).
RESULTS: The average age was 40.36 years (SD = 14.59) and 81.3% of participants were female. The intervention did not significantly reduce primary depressive symptoms (between-group effect size: d = 0.04) and secondary outcomes such as anxiety and stress symptoms (between-group effect size: d = -0.19). However, the intervention led to a significant increase in emotion regulation skills (between-group effect size d = 0.35) and resilience (between-group effect size d = 0.38).
CONCLUSIONS: The internet-based self-help intervention cannot be recommended for the purpose of reducing depressive symptoms. However, the increase in emotion regulation skills and resilience suggest that the intervention may be suitable for preventive purposes, like improving overall coping with psychological distress or potential stressors. Future research is needed to examine for whom and how the intervention is most effective.

Introduction

In December 2019, the first cases of pneumonia of unknown origin occurred in Wuhan, China. On March 11, 2020, the World Health Organization (WHO) declared the outbreak of the novel coronavirus disease 2019 (COVID-19) as a global pandemic, and as of July 2020, more than 10 million COVID-19 cases were reported worldwide (Wiersinga et al., 2020). In addition to the threat to physical health, the COVID-19 outbreak may also negatively affect mental health. Research at the onset of the COVID-19 pandemic has already indicated that symptoms of depression, anxiety, and self-reported stress have increased in the general population (Rajkumar, 2020; Wang et al., 2020). Meanwhile, various studies point towards an increase in depression and anxiety symptoms in the general population (Luo et al., 2020; Xiong et al., 2020). Moreover, in a study comparing the prevalence of depression symptoms before and during the COVID-19 pandemic in the U.S. general population, more than a tripling of the prevalence was found (Ettman et al., 2020). A continuation of this rise in the level of depression symptoms can be expected, since ongoing restrictions such as social distancing measures lead to isolation and loneliness (Beutel et al., 2017; Dozois, 2020). Preliminary evidence supports this assumption (de Quervain et al., 2020). An online survey in Switzerland showed an increase in psychological distress from the first to the second COVID-19 wave in the general population. For example, during the first COVID-19 wave in April 2020, 9% of respondents reported severe depression symptoms, compared to 18% during the second COVID-19 wave in November 2020 (de Quervain et al., 2020). However, due to the method of data collection, these results should be considered with caution.

Since the COVID-19 pandemic appears to be associated with high levels of psychological distress in the general population, measures should be taken to diminish and prevent further negative mental health impacts. Accordingly, considering the potential increase in demand for psychological support and a continued requirement for social distancing, easily accessible psychological interventions aiming to reduce COVID-19 related psychological distress are urgently needed (Luo et al., 2020; Xiong et al., 2020).

Information on the development and implementation of psychological interventions during the COVID-19 pandemic is still scarce. However, cognitive behavioral therapy (CBT) focusing on identifying and restructuring thought patterns and traps, relaxation techniques, and activity scheduling has been recommended (Halder, 2020; Wang et al., 2020). Furthermore, digital aids such as internet-based self-help interventions were found to be particularly suitable for the treatment of psychological distress under the given circumstances since they do not require direct on-site contact and are easily scalable (Halder, 2020; Soklaridis et al., 2020; Wang et al., 2020; Wind et al., 2020). Internet-based self-help interventions have proven to be an effective treatment option for various psychological problems, such as depressive symptoms (Andersson and Titov, 2014; Cuijpers et al., 2011).

To date, only a few studies have addressed psychological interventions targeting COVID-19 related psychological distress. In randomized controlled trials (RCT's), so far primarily psychological interventions for patients diagnosed with COVID-19 have been evaluated (Liu et al., 2020; Sotoudeh et al., 2020; Wei et al., 2020). For example, in an RCT, progressive muscle relaxation training over a period of 5 days effectively reduced anxiety and improved sleep quality in patients diagnosed with COVID-19 (Liu et al., 2020). Likewise, in a small RCT, a four-session face-to-face crisis intervention including relaxation, cognitive and metacognitive techniques, and techniques to increase resilience significantly reduced stress, anxiety, and depression in patients diagnosed with COVID-19 (Sotoudeh et al., 2020). With respect to internet-based self-help interventions for COVID-19 related psychological distress, Wei et al. (2020) evaluated the efficacy of an internet-based self-help intervention for patients diagnosed with COVID-19 experiencing psychological distress in a small RCT. The 2-week intervention consisting of breath relaxation training, mindfulness, and self-soothing skills has proven to reduce symptoms of anxiety and depression (Wei et al., 2020). Moreover, three studies evaluated internet-based self-help interventions targeting COVID-19 related psychological distress in the general population. In their pragmatic RCT, Al-Alawi et al. (2021) found preliminary evidence that a 6-week internet-based intervention consisting of weekly online sessions based on CBT and acceptance and commitment therapy (ACT) with a certified psychotherapist significantly reduced symptoms of anxiety and depression. In addition, the control group receiving an internet-based self-help intervention (weekly newsletter based on CBT and ACT) also showed improvement in anxiety and depression. However, the online therapy sessions were found to be superior (Al-Alawi et al., 2021). Wahlund et al. (2021) evaluated a 3-week internet-based self-help intervention for dysfunctional worry related to COVID-19. The CBT-based intervention significantly reduced COVID-19 related worry and improved other outcomes such as mood and insomnia (Wahlund et al., 2021). In a pilot RCT, Aminoff et al. (2021) evaluated a tailored internet-based CBT intervention for psychological distress associated with the COVID-19 pandemic. During the 7-week intervention, participants worked on 7 out of 16 modules selected for them based on a screening and clinical interview. Participants received support from a therapist. The intervention significantly reduced depression and other outcomes such as anxiety and stress symptoms (Aminoff et al., 2021).

Based on this background, we conducted an RCT to evaluate the efficacy of an internet-based self-help intervention for psychological distress due to COVID-19 in the general population. The intervention condition was compared to a waiting control condition with both conditions receiving care as usual (CAU). We hypothesized that the 3-week intervention called “ROCO” would lead to greater reduction of depression symptoms (primary outcome measure) and anxiety and stress symptoms (secondary outcome measures). Furthermore, we hypothesized that the intervention in addition to CAU would lead to greater beneficial effects on well-being, optimism, embitterment, loneliness, optimistic self-beliefs, emotion regulation skills, and resilience (secondary outcome measures) compared to CAU alone. We expected the effects to be stable in the 6-week follow-up.

Methods

Study design

In this parallel group RCT, an immediate treatment group receiving direct access to the 3-week internet-based self-help intervention was compared with a waiting control group. Both groups received CAU. Participants in the waiting control group were given access to the internet-based self-help intervention after 3 weeks. The immediate treatment group was followed up 6 weeks after randomization to evaluate the maintenance of potential treatment effects. We aimed to be able to detect small-to-medium between-group effect sizes of d = 0.35, since smaller effect sizes were considered clinically irrelevant (Donker et al., 2009). A power analysis with an α error level of 0.05 and a power (1-β) of 0.80 indicated a necessary sample size of at least 40 participants per group.

The Ethics Committee of the Canton of Bern approved the protocol of this study, and the trial was registered on ClinicalTrials.gov (NCT04380909). However, there is a slight deviation from the study protocol, as data from a second, 18-week, follow-up will be published at a later time due to the pressing nature of the topic (Brog et al., 2021).

Participants

Recruitment of German-speaking participants took place between April 2020 and February 2021, mainly through newspaper articles and internet self-help forums. All interested participants first visited our study website (https://selfhelp.psy.unibe.ch/roco/). Participants who registered on the study website subsequently received the study information. After returning a written informed consent form signed by the participant, participants were asked to complete an online baseline assessment. The online baseline assessment consisted of the outcome measure questionnaires, questions concerning socio-demographic variables, previous or present psychological treatment, and ongoing medication intake for psychological problems. Eligibility for participation in the study was determined based on this baseline assessment.

Criteria for inclusion were (a) to be at least 18 years old, (b) to have access to the internet, (c) to have sufficient knowledge of the German language, (d) to be able to specify an emergency address in the event of an acute crisis, and (e) to exceed a cut-off value of 4 points on the Patient Health Questionnaire (PHQ-9: Löwe et al., 2002), which is interpreted as the presence of mild depressive symptoms. Criteria for exclusion were (a) the presence of suicidal tendencies (Score ≥ 8 on the Suicide Behavior Questionnaire Revised (SBQ-R); Osman et al., 2001) and (b) a known diagnosis of a psychotic or bipolar disorder.

A detailed description of the participant flow is shown in Fig. 1. A total of 26 participants had to be excluded after they filled out the baseline assessment, mainly due to present suicidal tendencies (n = 15) and falling below the PHQ-9 cut-off (n = 8). Three participants fulfilled both exclusion criteria (suicidal tendencies and known diagnosis of a psychotic or bipolar disorder). A total of 107 participants fulfilled all the inclusion criteria and none of the exclusion criteria and were randomized to one of the two study groups in a 1:1 allocation ratio. Randomization was performed using a computer-generated randomization schedule by Randomization.com (Dallal, 2007, August 3). The allocation list was concealed from the investigators and participants.

Fig. 1

Selection, randomization, and flow of participants throughout the trial.

Participants were informed about their group allocation by e-mail. Participants allocated to the intermediate treatment group received an access code and registration instructions for the ROCO intervention. Three weeks after the start of the intervention the waiting period, all participants were asked to fill out an online post-assessment, consisting of the outcome measure questionnaires. After completing the post-assessment, participants in the waiting control group also were given access to the ROCO intervention. At 6 weeks after randomization, participants were asked to fill out the same outcome measure questionnaires again.

Measures

Primary outcome measure

All assessments were carried out online using self-report questionnaires. Participants filled out self-report questionnaires at pre-treatment, post-treatment (3 weeks) and follow-up (6 weeks after randomization). The primary outcome measure was the 9-item Patient Health Questionnaire (PHQ-9; Löwe et al., 2002), assessing the severity of depressive symptoms. The 9 items of the PHQ-9 correspond to the 9 DSM-IV criteria for depression. In the current sample, Cronbach's α was 0.71.

Secondary outcome measure

Secondary outcome measures include the Depression Anxiety Stress Scale (DASS-21; Lovibond and Lovibond, 1995) and the 12-Item Short-Form Health Survey (SF-12; Ware et al., 1996). The DASS-21 assesses depressive mood, anxiety, and stress and is often used as measure of general psychological distress (Breedvelt et al., 2020). To address general psychological distress, we report the composite scale of the 21-item measure (Cronbach's α = 0.87). To assess quality of life the SF-12 was used. The 12-item measure consists of two subscales, a Physical Component Score and a Mental Component Score. The SF-12 is widely used and has a good test-retest reliability (Gandek et al., 1998).

Further secondary outcome measures are the 10-item Life Orientation Test Revised (LOT-R; Glaesmer et al., 2008), the 6-item version of the Bern Embitterment Inventory (BEI; Znoj and Schnyder, 2014), and the 10-item General Self-Efficacy Scale (GSE; Jerusalem and Schwarzer, 2003). The LOT-R assesses generalized optimism (Cronbach's α = 0.73) and pessimism (Cronbach's α = 0.77), while the BEI assesses embitterment, defined as the feeling of being disadvantaged by others and fate (Cronbach's α = 0.77), and the GSE assesses optimistic self-beliefs (Cronbach's α = 0.88).

Furthermore, emotion regulation skills were assessed using the 27-item Self-report Measure to measure emotion regulation skills (SEK-27; Berking and Znoj, 2011). For this study, the composite score is reported (Cronbach's α = 0.93). Moreover, loneliness and resilience were assessed using the 9-item version of the UCLA Loneliness Scale (ULS; Luhmann et al., 2016) and the 10-item version of the Connor-Davidson Resilience Scale (CD-RISC; Connor and Davidson, 2003), respectively. The internal consistencies in the current sample were Cronbach's α = 0.85 for the ULS and Cronbach's α = 0.85 for the CD-RISC.

In addition, overall satisfaction with and usability of the internet-based self-help intervention were assessed post-treatment using the Client Satisfaction Questionnaire-8 (CSQ-8; Attkisson and Zwick, 1982) and the System Usability Scale (SUS; Brooke, 1996), respectively.

Last, the 4-item SBQ-R (Osman et al., 2001), which has been used to screen for the presence of suicidal tendencies, was also used to assess possible worsening of suicidal tendencies during the use of the internet-based self-help intervention. However, the internal consistency of the SBQ-R was unacceptable in the current sample (Cronbach's α = 0.34), and results concerning the SBQ-R should be considered with caution.

Description of intervention

Participants in the intervention group received access to the internet-based self-help intervention ROCO (stands for resilience and optimism during COVID-19), specifically addressing persons experiencing COVID-19 related psychological distress. ROCO is a 3-week self-help intervention consisting of 6 thematic modules. All modules contain brief texts, videos, illustrations, exercises, and a weekly task. The modules are based on cognitive behavioral therapy, focusing on (a) psychoeducation about COVID-19 related psychological distress, (b) emotion regulation skills, (c) identifying and restructuring thought patterns, (d) strengthening resilience, and (e) fostering relaxation and self-care. For a detailed description of the modules see Table 1.

Table 1

Outline of the content of the internet-based self-help intervention ROCO.

Introduction	Information on the self-help intervention and its handling
1. Identifying consequences and challenges	Psychoeducation on COVID-19 related psychological distress, evaluation of the current well-being (bodily sensations, positive, and negative feelings), resource-oriented weekly task
2. Understanding own feelings	Psychoeducation on emotions such as anxiety, helplessness, anger, sense of shame, and sadness, emotion regulation skills, acceptance-oriented weekly task
3. Changing the perspective	Psychoeducation on the influence of thoughts, automatic thoughts, rumination, and irrational beliefs, restructuring thought patterns, weekly task on rumination
4. Strengthening resilience	Psychoeducation on resilience, promoting coping, joie de vivre, and optimism, resource-oriented weekly task
5. Finding rest	Psychoeducation on sleep, sleep hygiene, and relaxation techniques, progressive muscle relaxation as weekly task
6. Taking care of oneself	Psychoeducation on the concept of post-traumatic growth and the importance of indulgence, gratitude and mindfulness exercises, resource-based weekly task
Conclusion	Information on maintaining and transferring what has been learned into everyday life

The modules are preceded by an introduction and rounded off by a conclusion. The self-help intervention also comprises information on what to do in an acute crisis, including a list of emergency contacts. Furthermore, an overview of the weekly tasks can be found, as well as a symptom-tracking questionnaire, allowing participants to track their self-reported symptoms.

Participants had access to all modules at all times. However, they were encouraged to work through two of the 6 modules per week. The individual modules require 40 to 80 min to complete. Since participants were able to access all self-help intervention content at any time, they could thus determine the timing and order in which they worked through the self-help intervention. While working on the self-help intervention, participants had the possibility to enable reminders that encouraged them to log in to the self-help intervention again after a certain period of inactivity.

Furthermore, a guidance on demand approach was applied. Guidance on demand implies that support is only established when requested by a participant, but there is no scheduled contact per se. Therefore, participants could demand guidance via text-based chat function in the self-help intervention. They were informed that a psychologist would answer their request within 3 working days.

Statistical analysis

Analyses were conducted in SPSS according to an intention-to-treat principle. We conducted independent samples t-tests and χ2 tests (nominal data) to test group differences in demographic data and pre-treatment outcome measures. The efficacy of the intervention was tested with a mixed-model repeated-measures analysis of variance with time (pre-post) as a within-group factor and treatment as a between-group factor. Mixed models offer some advantages: First, in mixed models, all available data from each participant is used. Therefore, missing values are not substituted, but the parameters of the missing values are estimated. Second, mixed models account for the dependence of data and correlation of repeated measures within individuals (Bell and Fairclough, 2014; Gueorguieva and Krystal, 2004).

We computed a separate model for each outcome measure. We used a compound symmetry covariance structure since it provided the best model fit based on Bayesian Information Criteria (BIC). We calculated Cohen's d for within- and between-group effect sizes based on estimated means and the pooled standard deviations of the observed means. In order to control for the baseline measures, we computed effect sizes sensu Morris (2008) for the pre-post comparison for the intervention group and the waiting control group. We calculated a Reliable Change Index (RCI; Jacobson and Truax, 1992) for depressive symptoms to analyze negative effects of the intervention (PHQ-9 = 4.69).

To test the stability of the effects from post-treatment to the follow-up, within-group changes in outcome scores from post-assessment to follow-up assessment were analyzed using paired t-tests. Only completers were included in the analysis of follow-up data. To compare drop-outs and completers we conducted independent t-tests and χ2 tests (nominal data).

Results

Baseline evaluation

The mean age of the 107 German-speaking participants was 40.36 years (SD = 14.59, range = 18–81 years). The majority were female (n = 87, 81.3%), of Swiss origin (n = 78, 72.9%), single (n = 65, 60.7%) went to university (n = 64, 59.8%), and were engaged in full-time (n = 27, 25.2%) or part-time paid work (n = 51, 47.5%). In total, 28 participants (26.2%) were in concurrent psychological treatment and 24 participants (22.4%) were taking medication for psychological problems. A large percentage of the participants had previous experiences with psychological treatment (n = 68, 63.6%). Based on the PHQ-9, the average depression score was 11.07 (SD = 4.23); 38.3% of the participants reported a mild, 39.3% a moderate, and 22.4% a severe depression. Participants initiated the participation in the study after they found the study website through a search on the internet (26.2%), after they read about the study on social media (13.1%) or in newspaper articles (16.8%), saw flyers (6.5%), and as a response to recommendations from a health professional (13.1%) or other sources such as friends or university services (23.4%).

Table 2 presents the baseline characteristics of the participants and between-group comparisons. There was a significant between-group difference in terms of education. Persons in the treatment group were less educated (χ ( = 8.03, p = 0.02, Cramer's V = 0.27). The groups did not differ significantly on any of the remaining demographic characteristics or other variables. Moreover, there were no significant between-group pre-treatment differences on any of the primary or secondary outcome measures (p's > 0.08).

Table 2

Demographics and sample characteristics at baseline for the treatment and waiting control group.

	TotalN = 107	Treatment group n = 53	Control groupn = 54	Statistic
Age, M (SD)	40.36(14.59)	40.68 (15.55)	40.04 (13.73)	t(105) = 0.23, p = 0.82b
Gender, n (%)				χ2(1) = 1.60, p = 0.21a
Male	19 (17.8)	7 (13.2)	12 (22.2)
Female	87 (81.3)	46 (86.8)	41 (75.9)
Non-binary	1 (0.9)	–	1 (1.9)
Current marital status, n (%)				χ2(1) = 0.24, p = 0.63a
Single	65 (60.7)	36 (67.9)	29 (53.7)
Married/Civil Union	30 (28.0)	15 (28.3)	15 (27.8)
Divorced/Civil Union annulled	11 (1.3)	2 (3.8)	9 (16.7)
Widowed/Civil partner died	1 (0.9)	–	1 (1.9)
Education, n (%)				χ2(2) = 8.03 p = 0.02a
Compulsory School	3 (2.8)	2 (3.8)	1 (1.9)
Apprenticeship	21 (19.6)	16 (30.2)	5 (9.3)
Secondary II	19 (17.8)	9 (17.0)	10 (18.6)
University	64 (59.8)	26 (49.0)	38 (70.4)
Employment, n (%)				χ2(3) = 0.86, p = 0.84a
Full-time paid work	27 (25.2)	14 (26.4)	13 (24.1)
Part-time paid work	51 (47.5)	24 (45.3)	27 (50.0)
Unemployed	3 (2.8)	2 (3.8)	1 (1.9)
At-home parent	4 (3.7)	3 (5.7)	1 (1.9)
Student	13 (12.1)	5 (9.4)	8 (14.8)
Retired	9 (8.4)	5 (9.4)	4 (7.4)
Nationality, n (%)				χ2(1) = 0.46, p = 0.50a
Swiss	78 (72.9)	36 (67.9)	42 (77.8)
German Speaking countries	26 (24.3)	14 (26.5)	12 (22.3)
Other Countries	3 (2.7)	3 (5.7)	–
Psychological Treatment, n (%)
Past	68 (63.6)	38 (71.7)	30 (55.6)	χ2(1) = 3.01, p = 0.08
Current	28 (26.2)	14 (26.4)	14 (25.9)	χ2(1) = 0.00, p = 0.95
Current Medications, n (%)	24 (22.4)	14 (26.4%)	10 (18.5)	χ2(1) = 0.96, p = 0.33
Depressive symptoms
Gesamtwert, M (SD)	11.07 (4.23)	11.13 (4.36)	11.00 (4.14)	t(105) = 0.16, p = 0.88b
Mild, n (%)	41 (38.3)	21 (39.6)	20 (37.0)	χ2(2) = 1.54, p = 0.46
Moderate, n (%)	42 (39.3)	18 (34.0)	24 (44.4)
Severe, n (%)	24 (22.4)	14 (6.4)	10 (18.5)

Chi-Square calculations include only categories with a frequency > 3.

Bootstrap 1000 samples.

Drop-out analysis and adherence to treatment

Of the 107 randomized participants, 97 (90.7%) completed the post-assessment, whereas 10 participants (9.3%) did not fill out the post-assessment (see Fig. 1). There were no significant differences in terms of demographics and primary and secondary outcome measures at pre-treatment between participants who did and who did not fill out the post-assessment (p's > 0.08). However, participants who did not fill out the post-assessment spent significantly less time in the self-help intervention (M = 47 min, SD = 1 h 32 min vs. M = 4 h 18 min, SD = 3 h 58 min, t(27.2) = 4.39, p = 0.003, d = 0.95) and completed significantly less modules (M = 2, SD = 2.07 vs. M = 4.53, SD = 2.10, t(9.7) = 3.18, p = 0.004, d = 1.21). A module was considered as completed if there was a timestamp (time at which the module was accessed) for the corresponding module. Since each module consisted of only one page, a timestamp indicated that the module had been consulted. Among participants who completed at least one module (48 of 53 participants in the intervention group), drop-out was 12.5%. Moreover, there was a tendency that participants who did not fill out the post-assessment were more often in the intervention group (15.1% vs. 3.7%).

Out of the 53 participants in the intervention group, 36 completed the follow-up questionnaires (67.9%). Drop-out at follow-up was associated with significantly lower usability ratings of the self-help intervention (M = 66.25, SD = 11.91 vs. M = 87.5, SD = 11.91, t(6.8) = 4.038, p = 0.005, d = 1.8) and fewer completed modules (M = 2.17, SD = 2.2 vs. M = 5.11, SD = 1.66, t(6) = 3.1, p = 0.021, d = 1.6).

On average, participants completed four of the six modules (M = 4.15, SD = 2.27, range = 0–6 modules) and 54.7% of participants completed all modules. Five participants did not log in to the self-help intervention (9.4%). The mean time spent in the self-help intervention was 3 h and 47 min (SD = 3 h 54 min, range: 0 min – 22 h 24 min). Only three participants demanded guidance via text-based chat function and 15 messages were exchanged in total. Pre-post changes of the outcome measures did not correlate with the number of completed modules nor with usage time, with one exception: The pre-post changes in loneliness, assessed by the ULS, correlated significantly with the number of completed modules (r = 0.395, p = 0.009), meaning that the more modules were completed the higher the reduction in loneliness tended to be.

Overall effects at post-treatment

The observed and estimated means for the primary and secondary outcome measures are displayed in Table 3. For each outcome measure, a linear mixed model with group as a fixed factor and time as a repeated factor was calculated (see Table 3).

Table 3

Observed and estimated means for primary and secondary outcome measures and within- and between-group effect sizes.

Outcome	Pre-treatmentM (SD)	n	Post-treatment (observed)M (SD)	n	Post-treatment (estimated)M (SE)	n	Follow up (observed)M (SD)	n	Post-treatment between group comparisonsaF(df), p	Pre-Post within group effect sizes (estimated means)dCohen (95% CI)	Between-group effect sizes at post-treatment (estimated means)dppc2 sensu morris
PHQ-9
Intervention	11.13 (4.36)	53	9.56 (3.70)	45	9.63 (0.59)	53	8.75 (5.07)	36	F(1,97.6) = 0.048,	0.37 (−0.18–0.91)	0.04
Control	11.00 (4.14)	54	9.60 (3.89)	52	9.67 (0.56)	54			p = 0.827	0.33 (−0.21–0.87)
DASS-21
Intervention	21.53 (9.23)	53	20.27 (10.84)	45	20.66 (1.39)	53	17 (10.44)	36	F(1,97.0) = 1.732,	0.09 (−0.45–0.63)	−0.19
Control	22.37 (9.86)	54	19.33 (9.13)	52	19.66 (1.34)	54			p = 0.191	0.29 (−0.25–0.82)
SF-12 MH
Intervention	31.10 (9.10)	53	36.72 (11.01)	43	36.47 (1.35)	53	38.31 (10.40)	36	F(1,98.3) = 1.586,	0.54 (−0.01–1.09)	0.24
Control	28.81 (7.73)	54	32.23 (9.20)	52	32.14 (1.27)	54			p = 0.211	0.39 (−0.15–0.93)
SF-12 PH
Intervention	53.43 (8.79)	53	50.96 (10.03)	43	51.26 (1.16)	53	51.26 (11.24)	36	F(1,96.3) = 0.005	−0.23 (−0.77–0.31)	0.01
Control	56.11 (6.98)	54	53.87 (6.43)	52	53.86 (1.10)	54			p = 0.942	−0.34 (−0.87–0.20)
LOT-R O
Intervention	7.19 (2.73)	53	7.52 (2.62)	42	7.45 (0.38)	53	7.69 (2.86)	36	F(1,92.1) = 0.674	0.10 (−0.44–0.64)	0.12
Control	6.87 (2.33)	54	6.84 (2.65)	50	6.82 (0.36)	54			p = 0.414	−0.02 (−0.55–0.51)
LOT-R P
Intervention	4.75 (2.76)	53	4.67 (2.39)	42	4.63 (0.39)	53	4.58 (2.31)	36	F(1,92.2) = 0.027,	0.05 (−0.49–0.59)	−0.02
Control	4.70 (2.63)	54	4.62 (2.91)	50	4.64 (0.37)	54			p = 0.969	0.02 (−0.51–0.56)
BEI
Intervention	8.75 (4.88)	53	8.45 (4.23)	42	8.52 (0.71)	53	7.61 (4.69)	36	F(1,93.1) = 0.075,	0.05 (−0.49–0.59)	0.04
Control	10.07 (4.96)	54	9.50 (5.22)	50	9.62 (0.68)	54			p = 0.785	0.09 (−0.45–0.62)
ULS
Intervention	21.26 (4.82)	53	19.88 (4.56)	43	20.12 (0.64)	53	19.28 (4.94)	36	F(1,95.3) = 2.155	0.24 (−0.30–0.78)	0.20
Control	20.37 (4.25)	54	20.27 (4.04)	52	20.16 (0.61)	54			p = 0.145	0.05 (−0.48–0.58)
GSE
Intervention	25.91 (4.47)	53	26.88 (4.81)	43	26.88 (0.66)	53	27.03 (5.35)	36	F(1,95.5) = 1.405	0.21 (−0.33–0.75)	0.17
Control	26.56 (4.72)	54	26.69 (4.47)	51	26.74 (0.63)	54			p = 0.239	0.04 (−0.49–0.57)
SEK-27
Intervention	62.64 (15.45)	53	73.33 (15.19)	42	71.68 (2.33)	53	73.92 (17.57)	36	F(1,93.6) = 5.661	0.59 (0.04–1.14)	0.35
Control	59.83 (16.61)	54	62.76 (16.65)	50	63.17 (2.22)	54			p = 0.019	0.20 (−0.33–0.74)
CD-RISC
Intervention	21.87 (6.62)	53	23.48 (6.43)	42	23.47 (0.92)	53	23.11 (6.51)	36	F(1,92.8) = 6.523	0.25 (−0.30–0.79)	0.38
Control	23.78 (5.47)	54	23.10 (6.75)	50	23.05 (0.88)	54			p = 0.012	−0.12 (−0.65–0.42)
SBQ-R
Intervention	4.92 (1.36)	53	5.18 (1.78)	45	5.15 (0.21)	53	5.36 (1.79)	36	F(1,97.3) = 0.010,	−0.15 (−0.69–0.39)	−0.02
Control	4.72 (1.28)	54	4.92 (1.41)	52	4.93 (0.20)	54			p = 0.919	−0.16 (−0.69–0.38)

M, mean; SD, standard deviation; SE, standard error; df, degrees of freedom; CI, confidence interval; PHQ-9, Patient Health Questionnaire; DASS-21, Depression Anxiety Stress Scale; SF-12 MH, Short-Form Health Survey mental health subscale, SF-12 PH, Short-Form Health Survey physical health subscale; LOT-R O, Life Orientation Test Revised optimism subscale; LOT-R P, Life Orientation Test Revised pessimism subscale; BEI, Bern Embitterment Inventory; ULS, UCLA Loneliness Scale; GSE, General Self-Efficacy Scale; SEK-27, Self-report Measure to measure emotion regulation skills; CD-RISC, Connor-Davidson Resilience Scale; SBQ-R, Suicide Behavior Questionnaire Revised.

Intention-to-treat analysis.

The primary outcome measure, the PHQ-9, was not qualified by a significant group x time interaction (F( = 0.048, p = 0.827). Between-group effect size controlling for pre-measurement sensu Morris (2008) for depressive symptoms was d = 0.04. Likewise, the mixed-model analyses revealed no significant group x time interactions for the following secondary outcome measures: DASS-21, mental and physical health subscales of the SF-12, optimism and pessimism subscales of the LOT-R, BEI, ULS, and GSE (all F's (degrees of freedom 1, 92.1–98.3) <2.155, all p's > 0.145). Between-group effect sizes controlling for pre-measurement sensu Morris (2008) ranged between d = 0.01–0.24 (absolute values).

The SEK-27 as measure of emotion regulation skills and CD-RISC as measure of resilience were qualified by significant group x time interactions (CD-RISC: F( = 6.523, p = 0.012; SEK-27: F( = 5.661, p = 0.019). Between-group effect sizes controlling for pre-measurement sensu Morris (2008) were small-to-medium with d = 0.35 (SEK-27) and d = 0.38 (CD-RISC). Within-group comparisons in the intervention group revealed small and medium effect sizes (CD-RISC: d = 0.25; SEK-27: d = 0.59). Within-group effect sizes in the waiting control group were d = −0.12 for the CD-RISC, respectively d = 0.20 for the SEK-27.

To explore whether concurrent psychological treatment or medication intake during the self-help intervention moderated pre-post effects on outcome measures, we included the corresponding variables in the mixed-model analyses and tested the significance of the three-way interaction between time, group, and concurrent psychological treatment or medication intake. None of the three-way interactions were significant (all p's > 0.054) with two exceptions: both three-way interactions for the DASS-21 were significant (psychological treatment: F( = 4.626, p = 0.034; medication intake: F( = 4.526, p = 0.036). For both, concurrent psychological treatment and medication intake, only time x group interactions among participants receiving concurrent psychological treatment/medication became significant (psychological treatment: F( = 6.14, p = 0.021 vs. F( = 0.002, p = 0.962; medication intake: F( = 4.647, p = 0.044 vs. F( = 0.037, p = 0.848). Between-group effect sizes controlling for pre-measurement sensu Morris (2008) were higher among those participants receiving psychological treatment or medication [psychological treatment: d = −0.73 vs. d = 0.01; medication intake: d = −0.85 vs. d = 0.05]. Participants who received both the internet-based self-help intervention and concurrent psychological treatment or medication showed worsening on the DASS-21 (see Table 4 for observed and estimated means).

Table 4

Observed and estimated means for the DASS-21 and within- and between-group effect sizes, considering concurrent psychological treatment and medication intake.

Outcome	Pre-treatmentM (SD)	n	Post-treatment (observed)M (SD)	n	Post-treatment (estimated)M (SE)	n	Post-treatment between group comparisonsaF(df), p	Pre-Post within group effect sizes (estimated means)dCohen (95% CI)	Between-group effect sizes at post-treatment (estimated means)dppc2 sensu morris
DASS-21
Concurrent psychological treatment
Intervention	22.86 (8.51)	14	25.00 (10.58)	12	25.77 (2.69)	14	F(1,23.4) = 6.14,	−0.31 (−1.36–0.75)	−0.73
Control	21.07 (9.68)	14	16.54 (8.59)	13	17.17 (2.64)	14	p = 0.021	0.43 (−0.63–1.49)
No concurrent psychological treatment
Intervention	21.05 (9.53)	39	18.66 (10.56)	33	18.81 (1.62)	39	F(1,71.45) = 0.002,	0.22 (−0.41–0.85)	0.01
Control	22.83 (10.00)	40	20.26 (9.22)	39	20.51 (1.54)	40	p = 0.962	0.24 (−0.38–0.86)
DASS-21
Concurrent medication intake
Intervention	25.29 (7.62)	14	27.91 (12.01)	11	27.53 (2.63)	14	F(1,19.5) = 4.647,	−0.23 (−1.28–0.82)	−0.85
Control	31.20 (9.86)	10	25.90 (11.61)	10	25.90 (2.93)	10	p = 0.044	0.49 (−0.77–1.75)
No concurrent medication intake
Intervention	20.18 (9.59)	38	17.82 (9.48)	33	18.37 (1.56)	38	F(1,72.8) = 0.037	0.19 (−0.45–0.83)	0.05
Control	20.51 (8.83)	43	17.93 (7.83)	41	18.40 (1.43)	43	p = 0.848	0.25 (−0.35–0.85)

M, mean; SD, standard deviation; SE, standard error; df, degrees of freedom; CI, confidence interval; DASS-21, Depression Anxiety Stress Scale.

Intention-to-treat analysis.

Treatment satisfaction

Overall, participants were satisfied with the self-help intervention. The mean score on the CSQ-8 was 3.09 (SD =0.61), corresponding to mostly satisfied (3). In addition, participants were very satisfied with the usability of the self-help intervention. The mean score on the SUS was 84.39 (SD = 14.01), lying between good (71.4) and excellent (85.5; Bangor et al., 2009).

Suicidal tendencies and negative effects

A linear mixed model with group as fixed factor and time as repeated factor (pre-post) was conducted for the worsening of suicidal tendencies. There was no significant group x time interaction on the SBQ-R (F( = 0.010, p = 0.919). Observed and estimated means for the SBQ-R are presented in Table 3. Regarding negative effects, the RCI showed that in the intervention group, 20% of the participants deteriorated on depressive symptoms and in the waiting control group, 19.23% of the participants deteriorated on depressive symptoms.

Stability of effects

Observed means and standard deviations at the 6-week follow-up for the primary and secondary outcome measures are displayed in Table 3. Only participants in the intervention group who completed all three assessments (pre, post, and follow-up) were included. DASS-21 scores decreased significantly from post-treatment to follow-up (t(35) = 2.314, p = 0.027, d = 0.38). There were no significant post-treatment to follow-up changes in the primary and the other secondary outcome measures (t('s = 0.170–1.617, p's = 0.115–0.866).

Discussion

In this trial, the efficacy of an internet-based self-help intervention for COVID-19 related psychological distress – ROCO – was investigated. The results show that the 3-week internet-based self-help intervention was not effective in reducing depressive, anxiety, and stress symptoms. There could be several reasons for this result: First, participants in this trial showed on average moderate depressive symptoms (primary outcome) at baseline. Meta-analyses indicate that the severity of depressive symptoms at baseline influence treatment effects (Bower et al., 2013; Fournier et al., 2010). For example, in their meta-analysis of low-threshold internet-based interventions, Bower et al. (2013) found that participants who are initially more severely depressed show larger treatment effects compared to participants with lower initial symptom severity. Similar results were reported by Fournier et al. (2010) in their meta-analysis on antidepressant medication and depression severity. The benefit of antidepressant medication increased with the severity of depressive symptoms. Such results can be explained by the fact that more severe depressive symptoms offer more room for improvement than mild or moderate depressive symptoms. Second, the ROCO intervention is rather short, with a duration of three weeks. Although internet-based interventions often are shorter compared to face-to-face therapies (van Beugen et al., 2014), it is possible that the ROCO intervention was too short to produce more and stronger changes for example in depressive symptoms. A study by Christensen et al. (2006) suggests that longer internet-based interventions are more effective in reducing depressive symptoms than shorter ones. However, heavy time constraints are one of the most common reasons for high attrition in internet-based interventions (Christensen et al., 2006; Christensen et al., 2009), which in turn would be an argument for shorter interventions. Third, we used the PHQ-9 to assess depressive symptoms. Although short measurement instruments such as the PHQ-9 are widely used, they also carry some risks (Titov and Andersson, 2021). Regarding the PHQ-9, for example, significantly more cases of major depression are detected when using simple cut-off scores than when using additional criteria consistent with DSM-IV (Titov and Andersson, 2021). Therefore, the use of convenient cut-off scores for the PHQ-9 could lead to over-identification of individuals with clinically relevant depressive symptoms. Accordingly, our sample may have included individuals for whom psychological treatment would not be necessary and who, accordingly, would not benefit from such treatment (Titov and Andersson, 2021). Fourth, while several studies suggest that the COVID-19 pandemic has a lasting negative impact on mental health (Daly et al., 2020; Kikuchi et al., 2020), a study from the U.S. reports an initial increase in psychological distress at the onset of the COVID-19 pandemic that was followed by a decline in psychological distress in the months thereafter (Daly and Robinson, 2021). Such findings may indicate that although there was a substantial increase in psychological distress at the onset of the COVID-19 pandemic, there may be a decrease in psychological distress over time. A trajectory like this, which is characterized by a decline in mental health at the time of an adversity followed by a gradual improvement coming close to previous levels, is referred to as recovery in resilience research (Infurna and Luthar, 2018). Recovery is a common response to other major life stressors or potentially traumatic events (Clark and Georgellis, 2013; Galatzer-Levy et al., 2018). In accordance with this assumption of recovery, both groups in this trial show improvements in the primary outcome, depressive symptoms, over time. Within-group effect sizes are small-to-medium (intervention group, d = 0.37, waiting control group d = 0.33). Therefore, it is questionable whether an early intervention to reduce psychological distress is necessary at all or if it is advisable to first observe if recovery occurs. However, since other studies have found that internet-based self-help interventions are effective in reducing COVID-19 related depressive symptoms (Al-Alawi et al., 2021; Aminoff et al., 2021; Wahlund et al., 2021), more research is needed to identify under which circumstances internet-based self-help interventions are effective in reducing COVID-19 related depressive symptoms and for whom. Nonetheless, the ROCO intervention led to an increase in emotion regulation skills (between-group effect of d = 0.35) and resilience (between-group effect of d = 0.38) as early as 3 weeks after treatment initiation. The effects remained stable in the 6-week follow-up. Given the content of the ROCO intervention, the improvement in emotion regulation skills and resilience is plausible. ROCO includes both a module that addresses emotions and emotion regulation skills and a module that focuses on strengthening resilience. Accordingly, the results could be explained by the content of the ROCO intervention. Taking into account that increasing resilience was mentioned as a consideration for dealing with the COVID-19 pandemic, these are encouraging results (Habersaat et al., 2020). Moreover, it has been shown that deficits in emotion regulation skills are associated with psychopathology such as depressive symptoms (Silk et al., 2003; Williams et al., 2004), while successful emotion regulation facilitates emotional adjustment (Berking et al., 2008). In the case of ROCO, these findings could indicate that the intervention only proves effective in the long-term, in particular when a new stressor occurs. Accordingly, the intervention could be particularly useful as first-step measure for preventive treatment.

Negative effect sizes for depressive, anxiety, and stress symptoms as measured by the DASS-21 were found for participants who were concurrently receiving psychological treatment (d = −0.73) or taking medication (d = −0.85). Even though the sample size for the three-way-interaction between time, group, and concurrent psychological treatment was small, these results might suggest that ROCO could be particularly beneficial for people who do not seek concurrent treatment. Lastly, the intervention group showed similar rates of deterioration with respect to depressive symptoms as the waiting control group. This result contradicts meta-analyses that showed that deterioration rates are lower in internet-based self-help interventions compared to control groups (Ebert et al., 2016; Karyotaki et al., 2018). One possible explanation for the similar deteriorations in the two groups is that the COVID-19 pandemic is an ongoing stressor (Kira et al., 2021) and, that the ROCO intervention was not successful in halting the deterioration in depressive symptoms due to the COVID-19 pandemic.

Limitations

Several limitations of our study have to be considered. Participants in the waiting control group received access to the ROCO intervention after completing the post-assessment at the end of the three-week waiting period. For this reason, between-group comparisons are not possible for follow-up measurements, which is why we could only examine the stability of the effects for the intervention group and, moreover, cannot determine, whether the decrease of DASS-21 values from the post to the follow-up measurement was due to the intervention, recovery, or other reasons. Another limitation concerns randomization. The randomization was not ideal, since the analysis of the demographic data revealed a significant group difference regarding educational status. Moreover, although we assessed whether participants used other treatments or took medications in addition to the ROCO intervention at each measurement time point, we do not have information regarding the quantity and quality of those other treatments. Other treatments or medication might also influence the results and limit the generalizability of the study results. In this regard, the self-selection of the participants must be mentioned as another limitation. Due to self-selection, the participants may differ from the general population and the study results may be limited. Furthermore, we did not conduct a diagnostic interview, but used self-assessment questionnaires exclusively. Thus, we were not able to make diagnoses and the results may be affected by the subjective responses. Finally, drop-out rates at follow-up have to be mentioned as a limitation, even though drop-out rates at post-assessment were low.

Conclusions

Despite these limitations, the current trial provides further information on the use of internet-based self-help interventions during the COVID-19 pandemic. The investigated internet-based self-help intervention, ROCO, was not able to reduce primary depressive symptoms and is accordingly not suitable for the treatment of depressive symptoms. However, the present study showed evidence that the intervention has beneficial effects on emotion regulation and resilience. These results suggest that the intervention may be useful for preventive purposes, such as dealing with potential future stressors. Future research is needed to examine for whom and how such an intervention is effective.

Financial support

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sector and was entirely funded by the University of Bern.

Declarations

Ethical approval has been obtained by the Cantonal Ethics committee Bern (BASEC2020-00990). Informed consent was obtained from all participants in the study.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.