Lessons learned from two feasibility trials of a translated and minimally monitored iCBT program for young adults among community and university samples.

Young adults are less likely to use traditional mental health services than adults. iCBT may be more youth friendly, but its access remains limited in several countries. This study aims to evaluate the feasibility of a minimally monitored transdiagnostic iCBT program translated from English to French and offered in Canada for the treatment of anxiety and depression among young adults. The impact of the program on resilience was also assessed. Twenty-five participants from community and university samples were included in the analyses in two separate single-group feasibility trials with a 3-month follow-up. Feasibility outcomes were attrition, adherence, treatment acceptability and treatment gains. Outcome measures included the Patient Health Questionnaire 9-item (PHQ-9), Generalized Anxiety Disorder 7-item (GAD-7) and Connor-Davidson Resilience Scale (CD-RISC). In Trial 1 (N = 10), recruiting among the general population proved to be challenging. Low adherence (20%) and high attrition (80%) rates were observed. Methodological changes to recruitment were made and a second trial was conducted among university students using incentives (N = 15). About half of the participants completed treatment (55%, n = 8/15), 12/15 provided post-treatment data and 8/15 provided 3-month follow-up data. All study completers (100%; n = 14) reported they would recommend the program. Mixed effects model analyses revealed significant and large pre-post treatment reductions on the PHQ-9 and GAD-7 (Cohen's d = 1.09 and 1.64 respectively) with treatment gains maintained at follow-up. Moderate improvements were found on the CD-RISC post-treatment with further gains at follow-up (ds = 0.55 and 1.10 respectively). These finding suggest that future research on this French-Canadian version of the Mood Mechanic Course is feasible with appropriate attention to recruitment. Translating existing iCBT interventions and using minimal monitoring may be an optimal first step to initiate the cross-cultural dissemination of iCBT while increasing access to treatments among young adults. Lessons learned are discussed.

Introduction

Young Canadians aged 15–24 have the highest 12-month prevalence rate of depression compared to other age groups (Statistics Canada, 2017). This may result from having to cope with multiple changes during the transition to adulthood pertaining to social roles, identity, work environment and other aspects of life (Arnett, 2000). In addition, anxiety disorder rates have been increasing over the years in Canadian youth aged 12–24 years old, reaching a prevalence of 12.9% in 2018 (Wiens et al., 2020). Early help seeking may improve the long-term prognosis of these disorders in this population (Mitchell et al., 2017). However, young adults between 18 and 25 years of age are the least likely to use traditional mental health resources with less than half of them with depression seeking professional help (Rickwood et al., 2007; Statistics Canada, 2014; Statistics Canada, 2018). Percentages of help seeking as low as 18–34% have been reported for anxiety and depression among this population (Gulliver et al., 2010).

Stigma, embarrassment, problems recognizing symptoms, and a preference for self-reliance have all been identified as barriers perceived by young people to seeking help for mental health problems (Gulliver et al., 2010; Rickwood et al., 2007). A preference for self-reliance could stem from a need to assert their independence and autonomy which is part of this life phase (Wilson et al., 2005). Perhaps as a result of this need for autonomy and preference for self-reliance, young adults are increasingly consulting Internet-based information and support and integrating this understanding into service delivery may prove helpful (see Rickwood et al., 2007, for a review). The provision of Internet-delivered cognitive-behavioral therapy (iCBT) may be an option for facilitating access to treatment (Hadjistavropoulos et al., 2014; Andersson and Titov, 2014). Reviews and meta-analyses support the use of clinician-guided iCBT among adults for the treatment of anxiety and depression (Andrews et al., 2018; Newby et al., 2016; Titov et al., 2018). A few studies on clinician-guided iCBT have been conducted among young adults in research and routine care settings with encouraging findings (Benton et al., 2016; Day et al., 2013; Dear et al., 2018, Dear et al., 2019; Johnston et al., 2014; Mullin et al., 2015; Olthuis et al., 2016; Staples et al., 2019). Significant symptom reductions have been reported with gains maintained up to 12 months in a routine care setting, although patients in such a setting are less likely to complete post-treatment or follow-up outcome measures compared to those under strict research trial conditions (Staples et al., 2019).

Evidence-based iCBT programs have been implemented in many countries (Titov et al., 2018). However, their implementation remains limited in Canada (Hadjistavropoulos et al., 2021), even more so for young adults and French speakers. French is an official language in Canada and spoken by an estimated 300 million speakers worldwide (Organisation internationale de la francophonie, n.d.). Translating available programs has been advocated by the International Society for Research on Internet Interventions as a strategy to increase access to iCBT (Ritterband et al., 2006) and has been conducted in several countries (e.g., Knaevelsrud et al., 2015; Kayrouz et al., 2016; Wagner et al., 2012). In countries sharing a similar culture, translating iCBT programs would only require a minimum of resources and may not require cultural adaptations (Lintvedt et al., 2013a, Lintvedt et al., 2013b). The use of a minimally monitored delivery, which involves no contact with a clinician but a number of strategies to promote engagement and adherence (e.g., preparatory telephone interview, automated emails, symptom monitoring), may also be a realistic first step in the implementation process. A growing number of self-guided iCBT trials have more recently been conducted using such strategies designated as minimally monitored interventions (Robichaud et al., 2020). Outcomes similar to clinician-guided iCBT have been found among adults (DaPonte et al., 2018; Dear et al., 2015, Dear et al., 2016; Fogliati et al., 2016; Robichaud et al., 2020; Titov et al., 2013, Titov et al., 2015). The use of a minimally monitored delivery of iCBT has also been supported among young adults in a large trial in which the efficacy of a clinician-guided iCBT was compared to a minimally monitored delivery (Dear et al., 2018). Large improvements in anxiety and depression were found post-treatment using both methods of delivery. No marked or consistent differences in clinical outcomes were found at 3- and 12-month follow-ups. A minimally monitored delivery may better align with young adults' need for self-reliance.

Beside reducing anxiety and depression, increasing resilience, commonly defined as a “dynamic process encompassing positive adaptation within the context of significant adversity” (Luthar et al., 2000, p. 543), may be of relevance for young adults given the prevalence of anxiety and depression among this population and the role changes associated with this developmental transition. University students with low levels of resilience report significantly higher levels of psychological distress in comparison to those with high levels of resilience (Pidgeon et al., 2014). Cognitive behavioral therapy has been shown to be an effective treatment for anxiety and depression, but also to enhance adaptive cognitive coping and improve problem solving skills (McCann et al., 2016; Padesky and Mooney, 2012). Active coping is associated with higher levels of resilience, effectively managing stressful situations, and lower levels of depression (Southwick et al., 2005). To our knowledge, no studies have yet documented the impact of iCBT on resilience among young adults.

This study includes two trials that aim to assess the feasibility of a translated and minimally monitored iCBT program, the Mood Mechanic Course, among French speaking young adults in Canada. Based on the Wellbeing Course, an established transdiagnostic iCBT for adults (Dear et al., 2011; Titov et al., 2013, Titov et al., 2014, Titov et al., 2015, Titov et al., 2020), the Mood Mechanic Course (Dear et al., 2018; Johnston et al., 2014; Staples et al., 2019) targets core symptoms of anxiety and depression in people aged 18 to 25 years. Like the Wellbeing Course, it has been successfully implemented at the MindSpot Clinic in Australia with 2405 young adults who started treatment from 2013 to 2019 (Staples et al., 2019). Its use with a minimally monitored delivery has been supported (Dear et al., 2018). In addition, the transportability of the French-Canadian version of the Wellbeing Course using a minimally monitored delivery model was previously supported at the Université de Moncton Telepsychotherapy Unit in a feasibility and randomized controlled trial (DaPonte et al., 2018; Robichaud et al., 2020). In the present study, a first feasibility trial of the French-Canadian translation of the Mood Mechanic Course was conducted among the general population. As a result of several challenges encountered, a second trial was conducted among university students. In both trials, assessed feasibility outcomes included the ease of recruitment, treatment acceptability, attrition, treatment adherence and a preliminary assessment of the program's efficacy on anxiety, depression, resilience and life satisfaction. It was hypothesized that the majority of participants would (1) complete the study, (2) complete the five lessons of the Mood Mechanic Course within eight weeks (3) report being satisfied with the program and (4) show statistically and clinically significant reduction in symptoms of depression and anxiety, including general anxiety, social phobia and panic. No hypotheses were made regarding resilience and life satisfaction given the exploratory nature of this assessment.

Methods

To facilitate comparison to past literature, the methodology was similar to previous studies on the minimally monitored Mood Mechanic Course (Dear et al., 2018).

Participants

In Trial 1, participants were recruited in New-Brunswick using advertisements in various media. Interested young adults applied online to participate via an established website (www.etherapies.ca). A total of 10 participants were included in the analyses. The eligibility criteria were: (1) between the ages of 18 and 25; (2) residents of New Brunswick; (3) French as the first official language spoken; (4) good understanding of written French; (5) reliable internet access; (6) clinical levels of depression on the Patient Health Questionnaire- 9 item (PHQ-9 ≥ 10), or of anxiety on the Generalized Anxiety Disorder Scale -7 item (GAD-7) ≥ 8, Social Phobia Inventory (SPIN) ≥ 19 (Connor et al., 2000) or Panic and Agoraphobia Scale (PAS) ≥ 19 (Bandelow, 1999); (7) not currently engaged in another cognitive behavioral therapy; (8) no change in medication for anxiety or depression at least one month prior to the study and no change expected during the study and (9) willing to share personal information with the research coordinator. Applicants with severe depression (score ≥ 20 or a score > 2 on item 9 (suicidal ideation) of the PHQ-9) or currently experiencing mental health problems with psychotic symptoms were not eligible to participate. Within five business days of applying, individuals meeting the criteria were contacted by telephone to discuss the study and to answer any questions about the trial. Participants were also asked to provide the name and phone number of an emergency contact (either their family physician or significant other). Despite the use of SMS texts, less than half of eligible participants could be reached for a telephone interview. In consideration of the challenges encountered during recruitment, a decision was made to include these applicants (60%; n = 6/10). Those who could not be reached by telephone were sent an email requesting that they provide an emergency contact.

In Trial 2, recruitment took place in undergraduate university courses. Changes were made to the eligibility criteria so that young adults showing mild levels of anxiety or depression could participate (PHQ-9 ≥ 5 or GAD-7 ≥ 5). The initial telephone interview was reinstated as inclusion criteria and participants were compensated with course credits based on the time they invested in the study (4 course credits maximum based on 4 h of investment). Applicants not meeting the study criteria were sent an email to thank them for their interest in the study and were provided a list of mental health resources. Individuals showing severe depression or suicidal thoughts were encouraged to contact their family physician and were sent a list of mental health resources as well. Participant flow is shown in Fig. 1. Table 1 presents the sociodemographic characteristics of the participants.

Fig. 1

Participant flow in Trial 1 (left) and Trial 2 (right).

Table 1

Sociodemographic and mental health characteristics of the participants in each trial.

Variable	Trial 1(N = 10)		Trial 2(N = 15)
	n	%	n	%
Gender
Man	2	20	3	20
Woman	8	80	12	80
Age
Mean (SD)	21.6 (1.9)		19.4 (2.1)
Range	18 to 25		18 to 25
Marital status
Married/common law	1	10	13	86.7
Single	9	90	2	13.3
Education (completed)
High school	4	40	11	73.4
College	1	10	1	6.7
University	5	50	3	20
Occupation
Full-time student	7	70	14	87.5
Part-time student	1	10	1	6.3
Full-time employment	2	20	0	0
Previously diagnosed with a mental health disorder	3	30	3	20
Previously received psychotherapy	5	50	8	53.3
Currently taking psychotropic medication	4	40	2	13.3

Measures

Treatment adherence

The percentage of participants who completed all five lessons of the Mood Mechanic Course within eight weeks was used to measure adherence. Lesson completion was defined as viewing all pages of each lesson and was monitored using participants' activity logs.

Attrition

Attrition was assessed using the percentage of participants who did not complete the post-test or follow-up outcome measures.

Treatment acceptability

Treatment acceptability was evaluated post-treatment using multiple choices or yes/no questions (adapted from Titov et al., 2013): (1) ‘Overall, how satisfied are you with the Course?’ (totally unsatisfied/generally unsatisfied/mostly unsatisfied/neither satisfied nor unsatisfied, mostly satisfied/generally satisfied and totally satisfied), (2) ‘How would you evaluate the quality of the material?’ (poor/acceptable/good/very good/excellent), (3) ‘Would you recommend this treatment to a friend suffering from anxiety or depression?’ (yes or no), and (4) ‘Was it worth your time doing the Course?’ (yes or no). Four open-ended questions were also included to get participants' feedback on the intervention, on what was the most and the least helpful to them and any suggestions that they might have to improve the program. As well, participants provided feedback about the program over the telephone at post-treatment at which time their answers to the previous questions were reviewed.

Primary psychological outcome measures

Outcome measures to assess preliminary treatment efficacy were brief self-report questionnaires administered online using the software Ninja Forms and through the secured study website. Pre-treatment Cronbach's alphas are reported for each measure. The primary psychological outcome measures assessed anxiety and depression and were the same as the ones used by Dear et al. (2018) to facilitate comparison of the findings. A secondary measure was used to assess the impact of the program on resilience while tertiary measures were comprised of disorder-specific measures of anxiety and of a measure of life satisfaction. French translations or adaptations of psychometrically sound English questionnaires were used. Primary outcome measures were translated using an internationally accepted translation methodology (Acquadro et al., 2012; 2010. Instructions for Patient Health Questionnaire PHQ and GAD-7 Measures TOPIC PAGES Background 1 Coding and Scoring 2, 4, 5 Versions 3 Use as Severity and Outcome Measures 6-7.). They showed adequate internal consistency and sensitivity to change during therapy (DaPonte et al., 2018; Robichaud et al., 2020).

Patient Health Questionnaire 9-item (PHQ-9; Kroenke et al., 2001, translation by MAPI Research Institute)

The PHQ-9 is comprised of nine items assessing the symptoms and severity of depression based on the major depression diagnostic criteria of the DSM-IV. Item scores are summed with higher scores representing higher levels of depression (α = 0.91).

Generalized Anxiety Disorder Scale 7-Item (GAD-7; Löwe et al., 2008; Spitzer et al., 2006, translation by MAPI Research Institute)

The GAD-7 is a 7-item measure of the symptoms and severity of generalized anxiety based on the DSM-IV diagnostic criteria, with higher scores representing higher levels of anxiety (α = 0.87).

Secondary psychological outcome measure

Connor-Davidson Resilience Scale (CD-RISC; Connor and Davidson, 2003; French-Canadian adaptation by Hébert et al., 2018)

Measure of 25 items assessing resilience or “personal qualities that enable one to thrive in the face of adversity” (Connor and Davidson, 2003, p. 76). Higher scores represent higher levels of resilience (α = 0.85).

Tertiary psychological outcome measures

Panic and Agoraphobia Scale (PAS, Bandelow, 1999; French-Canadian adaptation by Roberge et al., 2003)

Measure comprised of 13 items assessing the frequency and intensity of panic based on the DSM-IV diagnostic criteria for panic disorder with or without agoraphobia. Higher scores indicate more severe symptoms of panic (α = 0.83).

Social Phobia Inventory (SPIN; Connor et al., 2000; translation by Radomsky et al., 2006)

The SPIN is a 17-item questionnaire measuring the severity of fear, avoidance and physiological symptoms associated with social anxiety disorder. Higher scores indicate higher levels of social anxiety (α = 0.93).

Life Satisfaction Questionnaire (LSQ; Diener et al., 1985)

Five-item measure assessing global life satisfaction, with higher scores representing higher levels of life satisfaction (α = 0.78).

Translation

The translation of the Mood Mechanic Course into the Cours Mécanique de l'Humeur was performed by a professional translator Bourret Translation Inc. and reviewed by Professor France Talbot, a bilingual clinical psychologist with expertise in the treatment of anxiety and depression using CBT. Differences of opinion were limited. The names in the vignettes were changed and statistics pertaining to Australia were replaced by statistics relevant to Canada.

Intervention

The Mood Mechanic Course (Dear et al., 2018; Johnston et al., 2014; Staples et al., 2019) is a transdiagnostic iCBT intervention designed to treat core symptoms of anxiety and depression in people aged 18–24 years. It is comprised of five lessons completed over an 8-week period. It includes didactic text, photos, images and case-enhanced learning examples of young adults illustrating the use of CBT techniques to manage challenging situations commonly faced at that age (e.g. managing the demands of tertiary education, leaving the family home, limiting alcohol consumption to manage emotional wellbeing, and balancing workplace and other demands) (see Table 2 for a description). For each lesson, a “Do It Yourself” guide is included with homework and exercises to complete. Supplementary material on related topics is also included, such as problem solving, challenging unhelpful beliefs and overestimation of risk and underestimation of ability to cope.

Table 2

Content of the Mood Mechanic Course.

Lesson	Lesson content	Duration	Additional resources
Lesson 1	Psychoeducation on anxiety and depression, CBT model and symptom identification. Information on the prevalence of anxiety disorders and depression and on how they can interfere with life goals. Description of how cognitive, behavioral and physical symptoms can contribute to poor emotional health.	1 week	Sleep management
Lesson 2	Thought monitoring and challenging. Strategies to generate realistic cognitions.	2 weeks	Structured problem solvingWorry timeChallenging beliefs
Lesson 3	Hyperarousal/relaxation and hypo-arousal/pleasant activities scheduling. Strategies for physical de-arousal and for re-engaging in reinforcing activities.	1 week	Risk calculationCoping calculationShifting attention
Lesson 4	Avoidance and safety behaviors in relation with anxiety and depression along with graded exposure.	2 weeks	Assertive communication
Lesson 5	Information about problem-solving, relapse prevention, and emotional well-being maintenance.	2 weeks

Procedure

The study was approved by the Human Research Ethics Committee of the Université de Moncton (Moncton, New Brunswick, Canada), file number n° 1617-070.

Memorandum of understanding

A memorandum of understanding was signed between Macquarie University and the Université de Moncton to allow access to the Mood Mechanic Course and its translation for the purposes of this study.

Design

Two separate single group open trials including a 3-month follow-up were conducted. Primary and secondary outcome measures were administered at pre-treatment, post-treatment (Week 9) and at the 3-month follow-up. The SPIN, PAS and LSS were completed at post-treatment only to increase the likelihood that participants would complete the follow-up assessment. At the beginning of the course, participants received an email reminding them of how to proceed. A recommended timetable for each lesson was also included. The program was offered using the software Moodle hosted by Accra Solutions Inc. Personalized automated emails consisting of one to two paragraphs of three to four sentences written in a warm and supportive way were sent on a weekly basis (Titov et al., 2013). Instructions and reminders were provided as well as reinforcement of practice explaining that symptom reduction requires gentle, but consistent, practice of the skills over time, especially when exposure is introduced. Normalization of the challenges of learning new skills was also offered. A monetary compensation for completing the weekly assessment (5$/week) and posttest (20$) was offered in Trial 2 to encourage participation. Because this was a test of feasibility, we wanted to encourage participation, but do not anticipate that this is an ongoing requirement. A safety protocol similar to the one used by Titov et al. (2013) was used. No participant had to be excluded from the study and referred for face-to-face mental health services.

Statistical analysis

Changes in outcomes and maintenance of therapy gains were evaluated using linear mixed effect model analyses. As a result of attrition in Trial 1, a decision was made a posteriori to enter data from the pretest and the first three weeks of the program (60% Lesson 1 and 2 completion rate) into the analyses using the PHQ-9 and GAD-7 which were administered on a weekly basis. To compare the results with Trial 2, data from Week 3 of Trial 2 were also entered in the analyses. For both Trial 1 and Trial 2, a diagonal covariance matrix provided the best fit to the data based on the Akaike Information Criterion (Vrieze, 2012). Normality of data distribution was supported. One outlier, found in Trial 1 on the GAD-7, was eliminated.

Data on all outcome measures were imputed and were analyzed using the intent-to-treat principle. In the presence of missing data in clinical trials, complete case analysis and single imputation methods such as the last observation carried forward (LOCF) have often been used. However, removing incomplete cases from the analysis can lead to biases as incomplete cases can represent a non-random sample from the study. Estimates of the treatment group means and standard errors are affected by missing data approaches (such as LOCF) which can bias the estimated treatment effect (Olsen et al., 2012). Mixed-effect models and multiple imputations have been recommended as an acceptable alternative and are useful in studies with high rates of dropouts and small sample sizes (Barnes et al., 2006; Hoyle, 1999, as cited in Olsen et al., 2012). In such cases, multiple imputation of missing data can be helpful in increasing statistical power. Gains in statistical power were found to be pronounced for small samples ranging from 10 to less than 50 cases (Cheema, 2014). In multiple imputation, missing data are “filled in” or imputed with plausible values, thus preserving the overall sample size of the study. In the present study, the results of both complete case and imputed data analyses were reported for comparison purposes. Little's tests indicated they were missing at random in both trials (Trial 1: χ2 (15) = 10.392, p = 0.794; Trial 2: χ2 (15) = 13.079, p = 0.596), an assumption to be met for multiple imputations to be performed (Soley-Bory, 2013). A maximum of about 50% of cases had missing data on primary or secondary outcome measures so 50 imputations were specified for each trial as recommended by Manly and Wells (2015). Because there was a maximum of 27% of cases missing for the tertiary psychological outcome measures (SPIN, PAS and LSQ) in Trial 2, 27 imputations were specified for those. Overall, findings from mixed effect models using the observed data were comparable to those using the imputed data. Within-group effect sizes using Cohen's d were calculated based on pooled standard deviations. All analyses were performed using SPSS, version 27 (SPSS Inc., Chicago, IL).

Results

Attrition rate

For Trial 1, 40% of participants did not complete the Week 3 assessment (4/10) and 80% did not complete the post-treatment assessment (n = 8/10). No participant completed the 3-month follow-up. For Trial 2, an attrition rate of 20% was obtained (n = 3/15). Over half of the participants completed the follow-up (n = 8/15) (see Fig. 1).

Treatment adherence

In Trial 1, 20% (n = 2/10) completed all 5 lessons. On average, participants completed 2.5 lessons. In Trial 2, about half of participants finished all lessons (53%; n = 8/15), with the average of four lessons completed.

Treatment acceptability

Among study completers in both trials, 100% of participants reported that the quality of the material was good to excellent, that they were mostly satisfied to generally satisfied, that the program was worth their time and that they would recommend it to a friend. All study completers reported finding the program very easy to understand. Answers to the open-ended questions in both trials revealed a diversity of comments. A few participants reported finding the case stories most useful along with the “helpful tips and advice” (n = 3). Other comments included “Practical techniques so that I could do something to help myself, while before I would feel overwhelmed”; “Easy to apply in day-to-day life” and “Not having to meet someone to talk about my problems”. No trend was observed either in what participants reported as least helpful. The length of the sessions and emails figured among the comments. Only one participant mentioned the fact that no guidance was provided, and it pertained to doing the homework. A few suggestions were given to improve the program. They were to reduce its duration and the length of the lessons and varying the format by including videos for example (n = 3).

Primary psychological outcome measures

Means for the PHQ-9 and GAD-7 are shown in Table 3, Table 4 and changes in scores illustrated in Fig. 2. In Trial 1, the mixed linear model analyses did not reveal a significant time effect on the PHQ-9 from T1 to W3 (complete case F3, 10.72 = 3.34, p = 0.060), but a significant change was consistently found across the 50 multiple imputations with all p-values less than 0.05. A significant time effect was found on the GAD-7 (complete case F3, 14.26 = 7.17, p = 0.040). The significance of the changes for the GAD-7 was consistent over 50 imputations with all p-values less than 0.001. Pairwise comparisons including complete and imputed data indicated that the GAD-7 scores were significantly lower at W3 than at T1 (all ps < 0.05) as well as on the PHQ-9 (all ps < 0.01). None of the week to week comparisons were significant (all ps > 0.05). Large pre-treatment to W3 effect sizes were found using estimated marginal means (Cohen's d = 1.56 and 1.58 respectively).

Table 3

Means, standard deviations and effect sizes (Cohen's d) for the observed and estimated marginal means for Trial 1.

Variable	Observed Means (SD)				Estimated Mean (SD)				Effect sizes(based on estimated means)
	T1	W1	W2	W3	T1	W1	W2	W3	T1-W3
PHQ	15.00 (4.3)	11.22 (3.11)	10.40 (4.10)	8.67 (5.05)	15.00 (4.08)	11.21 (2.88)	10.39 (3.16)	8.52 (4.05)	1.56
GAD	12.50 (3.17)	10.22 (4.32)	10.29 (5.09)	6.17 (2.86)	12.5 (3.00)	10.25 (4.02)	8.44 (1.87)	6.16 (2.56)	1.58

Note. T1 = pre-treatment; W1 = week 1; W2 = week 2; W3 = week 3.

Table 4

Means, standard deviations and effect sizes (Cohen's d) for the observed and estimated marginal means for Trial 2.

Variable	Observed Means (SD)				Estimated Mean (SD)				Effect sizes(based on estimated means)
	T1	W3	T2	T3	T1	W3	T2	T3	Pre-post	Pre-W3	W3-post	Post-follow-up	Pre-follow-up
PHQ	11.27 (5.93)	6.85 (5.38)	4.55 (3.8)	4.00 (1.51)	11.27 (5.73)	6.92 (4.96)	4.53 (3.45)	3.96 (1.55)	1.09	0.97	0.52	0.15	1.16
GAD	10.8 (3.97)	7.15 (5.05)	3.91 (2.70)	5.13 (3.40)	10.8 (3.87)	7.2 (4.69)	3.96 (2.56)	5.06 (2.90)	1.64	0.71	0.97	0.31	1.1
RISC	59.13 (10.30)	–	64.91 (9.43)	74.25 (8.46)	59.13 (2.57)	–	64.867 (2.07)	74.22 (1.69)	0.55	–	–	1.10	1.59
SPIN	22.2 (13.58)	–	13 (9.75)	–	22.2 (13.13)	–	12.95 (8.25)	–	0.62	–	–	–	–
PAS	22.47 (6.84)	–	17.73 (5.00)	–	22.47 (6.62)	–	17.75 (4.34)	–	0.63	–	–	–	–
LSQ	23.07 (4.37)	–	26.73 (3.58)	–	23.07 (4.22)	–	26.77 (3.25)	–	0.88	–	–	–	–

Note. T1 = pre-treatment; W3 = week 3; T2 = post-treatment; T3 = follow-up.

Fig. 2

Estimated marginal means over time for the GAD-7 and PHQ-9 in Trial 1 and Trial 2.

In Trial 2, the results revealed a significant time effect on the PHQ-9 (complete case F3, 23.40 = 7.90, p = 0.001) and GAD-7 (complete case F3, 20.21 = 10.55, p < 0.001). The significance of the changes was consistent over 50 imputations with all p-values less than 0.01. Pairwise comparisons showed that the PHQ-9 and GAD-7 scores were significantly lower at post-treatment than at pre-treatment(all ps < 0.01) Scores on the PHQ-9 were significantly lower at W3 than pre-treatment (ps < 0.05 for complete cases and across 47 imputations) as well as scores on the GAD-7 (ps < 0.05 for complete cases and across 48 imputations). GAD-7 scores were also significantly lower at post treatment compared to W3 (ps < 0.05 for complete cases and across 47 imputations), whereas PHQ-9 scores did not lower significantly from W3 to post-treatment (all ps > 0.05). Gains were maintained between post-treatment and follow-up (all ps > 0.05). Large pre-treatment to post-treatment (Cohen's d = 1.09 to 1.64) and pre-treatment to 3-month follow-up (Cohen's d = 1.1 to 1.16) effect sizes were found using the estimated marginal means. Moderate to large effect sizes were obtained for the PHQ-9 and GAD-7 from pre-treatment to W3 and from W3 to post-treatment (Cohen's ds = 0.52 to 0.97).

Secondary psychological outcome measures (Trial 2 only)

Mixed linear model analyses revealed significant main effects for Time on the RISC (complete case F2, 19.29 = 7.98, p = 0.003). The significance of the change was consistent over 50 imputations with all p-values less than 0.001. Pairwise comparisons showed that RISC scores did not differ significantly from T1 to T2 (all ps > 0.05), but increased significantly from T2 to T3 (all ps < 0.05). A moderate pre-treatment to post-treatment effect size (d = 0.55) and large post-treatment to follow-up effect size (Cohen's d = 1.10) were found.

Tertiary psychological outcome measures (Trial 2 only)

Significant main effects were found for Time on the SPIN (complete case F1, 25.97 = 4.38, p = 0.046), PAS (complete case F1, 25.99 = 4.51, p = 0.043) and LSQ (complete case F1, 25.75 = 5.97, p = 0.022) d. The significance of the changes was consistent over 27 imputations with all p-values less than 0.05. Moderate pre-treatment to post-treatment effect sizes (d = 0.62 to 0.88) were observed.

Discussion

This study assessed the feasibility of an iCBT program, the Mood Mechanic Course, translated from English to French and offered in Canada using a minimally monitored delivery model for the treatment of anxiety and depression. Feasibility outcomes included treatment adherence, attrition, acceptability, and a preliminary assessment of the program's efficacy. Several challenges were encountered in Trial 1 and as a result, the hypotheses were not supported. In contrast, all study hypotheses were supported in Trial 2. A majority of participants completed the program and the study. Among study completers, treatment satisfaction was high with all of them reporting that they were mostly satisfied to generally satisfied with the program, that it was worth their time and that they would recommend it to a friend. Comparable results were found with the original English version of the program (Dear et al., 2018). Findings from the present two feasibility trials provided important information on the use and transportability of iCBT programs for young adults.

Preliminary support was found for the transportability of the Mood Mechanic Course. This is in line with Canada's priority to increase access to psychotherapy, including access to online therapy, based on models from Australia and the United Kingdom (Mental Health Commission of Canada, 2014, Mental Health Commission of Canada, 2018, Mental Health Commission of Canada, 2021). Translating existing iCBT programs may be an optimal and acceptable first step to increase access to iCBT in the face of limited cultural differences. In the present study, no cultural adaptations were made in consideration of previous findings pertaining to the Wellbeing Course, on which the Mood Mechanic Course is based. Its transportability in routine care was supported in Saskatchewan, Canada, using a clinician-guided delivery (Hadjistavropoulos et al., 2014; Hadjistavropoulos et al., 2016; Hadjistavropoulos et al., 2017) as well as a French-Canadian translation with no cultural adaptations delivered using a minimally monitored model in New Brunswick (DaPonte et al., 2018) and in the Atlantic provinces (Robichaud et al., 2020). In these studies, conducted by our team, as well as in the present study, the process of taking an existing intervention to a new country and language using a collaborative approach was straightforward. No issues related to cultural differences were raised by participants. A Telepsychology Memorandum of Understanding having been signed between the psychology regulatory bodies of the Atlantic provinces allowing for the provision of cross-jurisdictional telepsychology services, the transportability of the Mood Mechanic Course could be further increased to these provinces.

Several lessons learned in the present feasibility trials, namely in relation to recruitment and treatment engagement, could be of benefit to optimize the use of the Mood Mechanic Course including the following four:

First, consistent with what was experienced during a trial of the English version of the Mood Mechanic Course (Dear et al., 2018), recruitment was challenging in Trial 1. Both that study and Trial 1 relied primarily on media recruitment. A lesson learned was that even when recruiting a smaller sample, it appears to be easier to recruit young adults in educational settings as done in Trial 2 and in other studies (Benton et al., 2016; Day et al., 2013; Mullin et al., 2015). Offering course credits appears to have been a helpful incentive. An innovative approach may be to offer the Mood Mechanic Course as part of the students' academic curriculum with the aim of promoting their wellbeing. Such an approach was used at Yale University where an undergraduate course on the science of happiness entitled Psychology and the Good Life was offered and became the most popular course ever offered (Bridges, 2020). If such an approach were to be used, it is believed that credits for taking the course would be sufficient. There would be no need for a monetary compensation. In Trial 2, such a compensation was added mostly for the post-test because none of the participants would have completed it before the end of their university course. Since course credits had to be given before the post-test, we were concerned that it would negatively impact data collection. In a large-scale study, the influence of an online version of Psychology and the Good Life on students' subjective wellbeing was found to be significantly greater compared to a control course, Introduction to Psychology (Yaden et al., 2021). When recruiting in the general population, a marketing approach targeting families instead of directly targeting young adults may also be more helpful to promote young adults' engagement in iCBT (Wilson et al., 2011).

Second, in Trial 1, despite the use of SMS reminders for the pre-treatment telephone interview, less than half of eligible participants could be reached. As found in Australia, this interview turned out to be a barrier to begin the Course (Dear et al., 2018). Paradoxically, strategies aimed at increasing adherence may be, by the individual who does not engage, considered as a factor contributing to the decision to not begin or complete treatment (Johansson et al., 2015). In Trial 1, a decision was made to include the applicants who could not be reached by telephone. A clear link could not be established between participating in the telephone interview and treatment adherence and study completion rates in Trial 1 as a result of the small sample, but the interview was reinstated in Trial 2 as a participation criterion. Feasibility outcomes improved, but it is not clear that it was because of this procedure since participants in Trial 2 were also compensated to complete the program and the study. The decision to reinstate the initial telephone interview in Trial 2 was made since it was used in studies on the self-guided Mood Mechanic Course program in Australia, and while causation cannot be established, lower attrition rates than in Trial 1 were reported (Dear et al., 2018). Other authors have also postulated that brief 5 to 10 minute pre- and post-treatment telephone calls may encourage engagement in self-guided iCBT (Carlbring et al., 2006; Nordin et al., 2010). Of interest, Draucker et al. (2019) conducted a feasibility study of an online dating violence program among emerging adults. Attrition was elevated and occurred early in the program similarly to what was found in Trial 1. The authors concluded that strategies to increase retention early in the program may promote study completion. This may be the role of an initial telephone interview. This remains to be further assessed. Furthermore, young adults may be more willing to engage in such an interview if it has clinical relevance to them (e.g., to establish a diagnosis; Johnston et al., 2014; Mullin et al., 2015).

Third, although a lower adherence rate was found in Trial 1, young adults who did not complete treatment may have stopped because of having sufficiently engaged with the intervention to get the intended benefits. This has been referred to as effective engagement (Yardley et al., 2016). Treatment outcomes obtained in Trial 1 from pretreatment to Week 3 suggest this may have been the case. Large improvements in anxiety and depression were found in both trials from pretreatment to Week 3. Although the large improvements found from pre-treatment to W3 on the GAD-7 and PHQ-9 in Trial 1 may indicate that completing two lessons is an adequate therapeutic dose, data from Trial 2 suggest that further improvements may be occurring from W3 to post-treatment as moderate to large effect sizes were found. Also, post-treatment GAD-7 and PHQ-9 scores were in the normal range while they remained in the minimal range at Week 3 in Trial 1. For participants in Trial 1, going from moderate to severe symptoms to minimal ones may however have been perceived as satisfactory for them. The expectation that young adults would benefit from completing as many lessons as possible may not be realistic nor required. In contrast to a more traditional therapy approach with a predetermined number of sessions, a treatment model based on effective engagement allowing young adults to determine the duration of therapy as per their needs may be more efficient and further increase access to treatment. Although studies vary in quality, several reviews of single session interventions would support the assessment of such a model (e.g., Aafjes-van Doorn and Sweeney, 2019; Bloom, 2001; Cameron, 2007; Hymmen et al., 2013). A meta-analysis of 50 randomized trials further suggest that single session interventions would be beneficial to youth presenting different mental health problems (Schleider and Weisz, 2017). An online delivery was also supported among this population for the management of depression with gain maintained at a 3-month follow-up (Schleider et al., 2022). Face-to-face single session interventions have also been recently implemented in all community addictions and mental health clinics in New Brunswick, Canada, where the present study was conducted with, according to a news release, significant reduction of waitlists (MacIsaac, 2022).

Fourth, the large improvements observed on all outcome measures of anxiety and depression with gains maintained at follow-up support the transdiagnostic nature of the Mood Mechanic Course. Therapeutic gains may extend to positive psychological variables as well, namely on resilience. Scores on the RISC kept increasing from pre-treatment to follow-up with a large effect size from post-treatment to follow-up. The mean resilience score at follow-up was below the mean of 75.7 reported among non-help seeking university students (Hartley, 2011). These results are in line with the previously stated relationship in the literature between resilience and mental health (Hu et al., 2015). Future studies are needed with larger samples and comparisons groups to further examine the impact of iCBT on resilience, how it may contribute to relapse prevention and help young adults to better face life's challenges. iCBT may also prove to be helpful to young adults low in resilience to prevent the development of significant symptoms of anxiety and depression. A significant increase in life satisfaction was also observed suggesting that the Course may have an overall positive impact on young adults' life.

Despite the lessons learned and encouraging findings reported in this study, several significant limitations need to be acknowledged. Being feasibility trials, the samples were small and control groups or comparison groups were not included. It is therefore not possible to draw conclusions about the efficacy of the iCBT program. However, the magnitude of changes on the outcome measures were large and clinically significant, and comparable to the ones found in research and applied settings in Australia (Dear et al., 2018; Staples et al., 2019). No diagnostic interviews were conducted, but the use of self-report measures was more in line with a minimally monitored model of delivery. The generalizability of the results is limited. A majority of participants were women and highly educated, incentives were used in Trial 2 and the samples recruited may not be representative of young adults presenting to mental health services. Since only study completers provided feedback on the intervention, the acceptability of the program may be overestimated. Suggestions made by a few study completers may be helpful including reducing the duration of the program. The follow-up was brief, and no follow-up data was available for Trial 1. Results from the original version of the minimally monitored Mood Mechanic Course nevertheless suggest that treatment gains can be maintained for up to 12 months (Dear et al., 2018).

In conclusion, findings from the present study suggest that future research on the French-Canadian and minimally monitored version of the Mood Mechanic Course is feasible with appropriate attention to recruitment. As found for the French-Canadian translation of the Wellbeing Course (DaPonte et al., 2018; Robichaud et al., 2020), the results highlight the potential of translating an already implemented iCBT program and using a minimally monitored delivery model to facilitate the transportability of iCBT in countries and linguistic communities where its access is limited. Future studies are needed on the assessment of effective engagement and ‘youth-centered’ treatment models fulfilling young adults' need for autonomy and self-reliance while promoting access to evidence-based psychological care.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.