Transdiagnostic Internet-delivered cognitive behaviour therapy with therapist support offered once-weekly or once-weekly supplemented with therapist support within one-business-day: Pragmatic randomized controlled trial.

In routine care, internet-delivered cognitive behaviour therapy (iCBT) regularly includes therapist support delivered via secure email, but the optimal response time to emails is unknown. In this study, we compared the benefits of therapists providing support once-weekly versus therapists providing support once-weekly supplemented with a one-business-day response to all patient emails. This pragmatic randomized controlled trial included therapists employed by a specialized iCBT clinic or community mental health clinics, where providing iCBT is a secondary service. Patients with depression and/or anxiety who enrolled in transdiagnostic iCBT (5 core lessons over 8 weeks) were randomized to: 1) once-weekly support supplemented with a one-business-day response to patient emails by specialized therapists (n = 233); 2) once-weekly support also offered by specialized therapists (n = 216); or 3) once-weekly support offered by community clinic therapists (n = 226). Outcomes were measured at 8, 12, 24, and 52-weeks post-enrollment. Patient engagement and treatment experiences (e.g., treatment satisfaction, therapist alliance) were also assessed and a focus group was conducted with therapists. Supplementing once-weekly therapist support with a one-business-day response to patient emails resulted in therapists sending more emails to patients (M: 13 versus 9) and required more therapist time over treatment (M: 155 versus 109 min), but was not associated with improved outcomes, patient engagement or treatment experiences. All groups showed large improvements in symptoms of depression and anxiety maintained at 52-week follow-up, strong engagement and positive treatment experiences. Therapists viewed challenges of responding to patient emails within one-business-day to outweigh benefits. Contrary to expectations, supplementing once-weekly therapist support with a one-business-day response to all patient emails did not benefit patients and increased therapist time as well as therapist challenges when delivering iCBT in routine care.

RCT Entities:   Population Interventions Outcomes

Introduction

Depression and anxiety are highly prevalent and disabling conditions (Vos et al., 2015). In Canada, it is estimated that up to 40% of individuals with these conditions do not receive formal treatment (Urbanoski et al., 2017). Cognitive behaviour therapy (CBT) is recognized as one of the most effective treatments with large effect sizes observed for symptom improvement (Cuijpers et al., 2016). Several barriers limit access to face-to-face CBT, however, including location, mobility or time constraints, stigma, or a desire to self-manage symptoms (Andersson et al., 2019).

Therapist-assisted internet-delivered cognitive behaviour therapy (iCBT) is a promising alternative to face-to-face CBT that addresses these barriers (Andersson et al., 2019). In iCBT, patients receive similar content to that shared in face-to-face CBT, except therapeutic content is delivered in an online format, typically via weekly lessons. Lessons are often comprised of text but can also include audio-visual content, and homework is typically assigned to assist in skill acquisition. In many iCBT programs, patients receive support from a therapist via secure email systems or phone calls estimated to take 10–15 min on a weekly basis (Andersson et al., 2014; Titov et al., 2018). The availability of therapist support alongside iCBT programs is associated with higher completion rates and larger effect sizes than when therapist support is not provided (Baumeister et al., 2014). Reviews of the literature also suggest iCBT is as effective as face-to-face CBT (Carlbring et al., 2018) and that results tend to be maintained over the longer term (e.g., several years post-intervention; Andersson et al., 2018). Of importance, there is also evidence that results for iCBT are similar when offered in routine care clinics (e.g., Titov et al., 2018), wherein patients often report greater symptom severity and comorbidity (Kazdin, 2015).

Optimizing amount of therapist support

While therapist support appears to be preferable to no therapist support (Baumeister et al., 2014), limited research has investigated the impact of varying how quickly therapist support is provided. Understanding the optimal responsivity of therapist support is critical because it represents a major resource requirement in delivering iCBT. Recently some attention was given to offering therapist support once-weekly (1W; i.e., where a therapist initiates contact 1W) compared to optional weekly therapist support (i.e., no contact is offered unless the patient requests support; Hadjistavropoulos et al., 2017b). Similar outcomes were found for both approaches at 3-month follow-up, although optional therapist support resulted in lower iCBT completion rates (56.6% versus 82.4%). In contrast to 1W therapist support, there are patients in routine practice who express a preference for more frequent and immediate contact with therapists during iCBT, such as one-business-day (1BD) therapist response to patient emails (Hadjistavropoulos et al., 2018a). Similarly, therapists have also suggested that a more timely response to patient emails would improve patient engagement and outcomes (Hadjistavropoulos et al., 2017a), especially for patients with greater symptom severity or who express a strong preference for frequent therapist support. Therapist support in 1BD may represent a more personalized, patient-centred approach, which refers to care that responds to patient preferences, needs, and values (Institute of Medicine, 2001). Some of the documented benefits of patient-centred care that could facilitate iCBT outcomes include increased adherence, self-efficacy, self-management, and satisfaction (Preference Collaborative Review Group, 2008). Of note, 1BD therapist support is different than planned therapist contact multiple times per week. For example, Klein et al. (2009) compared 1W contact with three-weekly contacts, and found no differences in clinical outcomes. However, in 1BD therapist support, therapists check in 1W, but also respond within 1BD when patients send emails during the week. In essence, responding in 1BD to patients who send emails recognizes that not all patients need additional therapist support, but support is available if patients initiate contact.

Study purpose

To our knowledge no research has examined 1W therapist support supplemented with 1BD response to patient emails (referred to throughout this paper as 1W + 1BD therapist support) even though this approach has been suggested by both patients and therapists, and it represents a potential approach for delivering iCBT that might be expected when implemented in routine care. A comprehensive comparison of 1W + 1BD therapist support to 1W therapist support provides valuable information for decision-making around implementation of iCBT in routine care, and can provide an understanding of patient benefits but also implementation costs and feasibility (Hermes et al., 2019). Thus, the objective of this study was to understand the impact of offering 1W + 1BD therapist support versus 1W therapist support on both patients and therapists. Patients were randomly assigned to one of three groups: (1) iCBT with 1W + 1BD therapist support offered by therapists employed by a specialized iCBT clinic, whereby therapists specialize in the delivery of iCBT and do not deliver face-to-face care (1W + 1BD specialized); (2) iCBT with a standard 1W therapist support also offered by therapists employed by a specialized iCBT clinic (1W specialized); or (3) iCBT with 1W therapist support offered by therapists employed by community mental health clinics whereby iCBT is a secondary task and therapists typically spend most of their time delivering face-to-face care (1W community). The community clinic therapists were included for comparative purposes as this group routinely offers iCBT in Saskatchewan, but could not offer 1W + 1BD therapist support without having evidence of clinical benefits relative to costs of the approach. Groups were compared in terms of patient engagement with iCBT (e.g., lessons completed, emails, log-ins), treatment experiences (e.g., satisfaction, therapist alliance, negative effects), as well as symptom improvement (primary outcomes were depression and general anxiety) at 8, 12, 24 and 52-week follow-up. Consistent with an implementation research approach (Hermes et al., 2019), we measured therapist time and therapist perceptions of 1W + 1BD approach as these represent important considerations when implementing iCBT in routine care. It was hypothesized that patients receiving the 1W + 1BD therapist support would show greater engagement, improved treatment experiences, and larger symptom improvements (depression and anxiety were primary outcome measures) at all time periods compared to patients receiving 1W support; no differences were hypothesized between 1W therapist support offered by therapists employed by the specialized clinic versus community mental health clinics based on past cohort comparisons showing no differences between these two groups in routine practice (Hadjistavropoulos et al., 2016). Because preference for offering 1W + 1BD therapist support has been expressed by therapists in past research (Hadjistavropoulos et al., 2017a), it was predicted that 1W + 1BD therapist support would be preferred by therapists.

Method

Ethics statement and study design

This study was approved by the Research and Ethics Board at the University of Regina and was registered as a clinical trial (Clinicaltrials.gov: NCT03304392). This study had a three-arm, randomized, controlled, superiority, pragmatic trial design whereby patients were randomly assigned to three groups (1W + 1BD specialized, 1W specialized, 1W community). The original trial design was to compare 1W + 1BD specialized with 1W specialized. Because iCBT is also routinely delivered by clinicians working in community clinics, this third group was added to the study design. It was felt to be methodologically advantageous to compare 1W + 1BD specialized relative to both 1W specialized and 1W community to ensure there was not bias introduced by having the specialized clinic therapists deliver both 1W + 1BD and 1W therapist support. The study was powered to detect between a small and moderate size difference in outcomes. With a 0.05 significance level and 80% power, an estimated total sample of 787 was required to detect a small effect size and a total sample of 128 was estimated to detect a moderate effect size.

Setting, participant recruitment, screening, and randomization

All patients began by applying for iCBT through the Online Therapy Unit website (www.onlinetherapyuser.ca); the unit is a government-funded clinic that specializes in iCBT. Patients learned about the Online Therapy Unit through a variety of sources including: medical professionals (53.2%; n = 361), mental health professionals (21.63%; n = 146), word of mouth (12.59%; n = 85), and online searches and email announcements (6.37%; n = 43). Other sources of referral (e.g., media, poster) were rare or not specified (6.21%; n = 40).

Patients who completed the online screening process between October 15, 2017 and September 14, 2018 were included in the trial. The online screening questionnaire assessed whether patients met the following inclusion criteria including: (1) being 18 years of age or older; (2) residents of Saskatchewan; (3) endorsing symptoms of anxiety and or depression; (3) able to access a computer with reliable internet access; and (5) willingness to provide a physician as an emergency contact. After the online screening questionnaire was completed, a staff member from the Online Therapy Unit contacted the patient by telephone to discuss treatment appropriateness and further assess exclusion criteria. Individuals were excluded if they: (1) had current high suicide risk, or a suicide attempt or mental health hospitalization in the previous year; (2) had primary problems with psychosis, alcohol or drug problems, or mania; (3) were currently receiving regular (>twice a month) psychological treatment for anxiety and or depression; (4) would not be present in the province during the 8-week treatment period; or (5) had concerns about participating in iCBT.

Following eligibility assessment, screeners randomized patients who were deemed eligible for the trial to one of the three groups using Research Electronic Data Capture (REDCap) in a 1:1:1 ratio in blocks of 24 without matching. Therapists in the specialized clinic had a background in social work (n = 3) or psychology (n = 1), whereas therapists from the seven community mental health clinics had a background in psychology (n = 7), social work (n = 18), nursing (n = 3), or addictions counseling (n = 3). All therapists received training and supervision in iCBT through the Online Therapy Unit (Hadjistavropoulos et al., 2012). The nature of the groups did not allow for therapists or patients to be blinded to condition. See Fig. 1 for participant flow and Table 1 for sample characteristics.

Fig. 1

Patient flow from screening to 52-week follow-up.

Table 1

Pre-treatment patient characteristics by group.

Variable	All groups (n = 675)		Once-weekly plus one-business-day support specialized clinic (n = 233)		Once-weekly support (n = 442)				Statistical significance
					Specialized clinic (n = 216)		Community clinics (n = 226)
	n	%	n	%	n	%	n	%
Age
Mean (SD)	36.92 (13.02)	–	37.37 (13.15)	–	37.03 (13.40)	–	36.34 (12.56)		F2,673 = 0.40; p = .69
Range	18–84	–	18–84	–	18–78	–	18–77
Gender
Male	162	24	54	23.2	61	28.2	47	20.8	χ2(2, N=675) = 3.49; p = .18
Female	513	76	179	76.8	155	71.8	179	79.2
Marital status
Single/never married	172	25.5	54	23.2	61	28.2	57	25.2	χ2(4, N=675) = 6.71; p = .75
Married/common-law	431	63.8	150	64.4	134	62.1	147	65.1
Separated/divorced/widowed	72	10.6	29	12.5	21	9.7	22	9.7
Education
Less than high school	19	2.8	7	3.0	5	2.3	7	3.1	χ2(6, N=675) = 4.92; p = .96
High school diploma	109	16.1	39	16.7	36	16.7	34	15.0
Post high school certificate/diploma	196	29.0	71	30.5	57	26.4	68	30.1
University education	351	52.0	116	49.8	118	54.6	117	51.9
Employment status
Employed part-time/full-time	415	61.5	144	61.8	135	62.5	136	60.2	χ2(10, N=675) = 6.10; p = .81
Unemployed	45	6.7	19	8.2	13	6.0	13	5.8
Homemaker	83	12.3	25	10.7	26	12.0	32	14.2
Student	53	7.9	17	7.3	16	7.4	20	8.8
Disability	47	7.0	13	5.6	17	7.9	17	7.5
Retired	32	4.7	15	6.4	9	4.2	8	3.5
Ethnicity
Caucasian	616	91.3	217	93.1	195	90.3	204	90.3	χ2(4, N=675) = 19.28; p = .26
Indigenous	36	5.3	13	5.5	10	4.6	13	5.7
Other	23	3.3	3	1.2	11	5.1	9	3.9
Location
Large city (over 200,000)	274	40.6	94	40.3	92	42.6	88	38.9	χ2(4, N=675) = 5.07; p = .75
Small to medium city	179	26.5	60	25.8	52	24.1	67	29.6
Small rural location (under 10,000)	222	32.9	79	33.9	72	33.3	71	31.4
Mental health characteristics
Taking psychotropic medication	372	55.1	137	58.8	113	52.3	122	54.0	χ2(2, N=675) = 2.08; p = .35
Pre-treatment GAD-7 ≥ 10	446	66.1	152	65.2	151	69.9	143	63.3	χ2(2, N=675) = 2.28; p = .32
Pre-treatment PHQ-9 ≥ 10	440	65.2	162	69.5	139	64.4	139	61.5	χ2(2, N=675) = 3.35; p = .19
Pre-treatment PDSS-SR ≥ 8	330	48.9	119	51.1	98	45.4	113	50.0	χ2(2, N=675) = 1.63; p = .44
Pre-treatment SIAS-6 ≥ 7 & SPS-6 ≥ 2	354	52.4	120	51.5	112	51.9	122	54.0	χ2(2, N=675) = 0.33; p = .85
LEC-5 trauma and PCL-5 ≥ 33	195	28.9	58	24.9	61	28.2	76	33.6	χ2(2, N=675) = 4,33; p = .12
No clinical scores	79	11.7	26	11.2	24	11.1	29	12.8	χ2(2, N=675) = 6.02; p = .81
Mean measures above cut-off (SD)	2.63 (1.51)	–	2.62 (1.45)	–	2.60 (1.52)	–	2.62 (1.57)	–	F2,673 = 0.02; p = .98

Note. GAD-7 = Generalized Anxiety Disorder-7; PHQ-9 = Patient Health Questionnaire-9; PDSS-SR = Panic Disorder Severity Scale-Self Report; SIAS-6/SPS-6 = Social Interaction Anxiety Scale-6 and Social Phobia Scale-6; LEC-5 = Life Events Checklist for DSM-5; PCL-5 = Posttraumatic Stress Disorder Checklist for DSM-5.

Intervention

All patients received the same iCBT course, called the Wellbeing Course, which was developed by the eCentreClinic at Macquarie University, Sydney, Australia and has been described in detail elsewhere (Titov et al., 2015b). This course is appropriate for individuals with and without a DSM-5 diagnosis of depression or DSM-5 anxiety disorder. The course contains five lessons that cover (1) the cognitive behavioural model and symptom identification; (2) thought monitoring and challenging; (3) de-arousal strategies and pleasant activity scheduling; (4) graduated exposure; and (5) relapse prevention. Each lesson includes psychoeducational material in a slideshow format, patient stories, and downloadable lesson materials and assignments to facilitate skill acquisition. Extra resources that can be accessed at any time are also available related to a variety of topics (i.e., assertiveness, communication skills, managing beliefs, mental skills, managing panic attacks, managing PTSD, sleep, structured problem solving). The five lessons are released to patients gradually over 8 weeks with automated emails used to notify patients about the availability and content of upcoming lessons. Lessons are released based on elapsed time and the patient having accessed the previous lesson and not based on therapist review of patient progress; specifically, lessons 1, 2, 3, 4, and 5 are available at the beginning of weeks 1, 2, 4, 5 and 7.

Therapist support

In all groups, patients were informed that they could send as many emails as they wanted to their therapist over the course of treatment. In 1W conditions, patients were informed that their therapists would respond to their emails 1W on a pre-specified day for 8 weeks and typically would spend 15 min per week on these emails. In the 1W + 1BD condition, patients were informed that in addition to 1W therapist support, their designated therapist would respond to all patient emails within 1BD. In all groups, patients were informed that most contact would be by email but phone calls would also be made if therapists felt it was indicated, such as when patients had not logged in for over a week, had a significant increase in symptoms, were perceived to be at increased suicide risk, had difficulty with the website, or requested a phone call.

In all conditions, the assigned therapist would send an email to the patient on the designated day each week. Therapists were instructed to spend ~15 min on this weekly email to each patient and to: (1) show warmth and concern; (2) engage the patient by asking about their understanding of the material and need for help; (3) provide feedback on symptom questionnaires that preceded each lesson; (4) highlight lesson content; (5) answer any patient questions about lessons to assist with skill development; (6) reinforce progress and practicing skills; (7) manage any risks (e.g., suicide); and (8) remind patients of course procedures as needed (e.g., timelines, next check-in). In 1W + 1BD, the first email of the week was to contain the above features and additional emails, sent within 1BD of receiving a patient email, were designed to be supportive, answer patient questions or respond to comments in patient emails. The 1BD emails could also include additional features as needed (e.g., feedback on questionnaires, clarify procedures, encourage reading and practice).

Outcomes1

Primary outcome measures were administered at screening, prior to lessons 1 to 5 and then at 8, 12, 24, and 52-week follow-up after enrollment. Secondary measures were obtained at all periods but not prior to lessons 2 to 5. The exception were measures related to trauma, which were administered at screening and at 12, 24 and 52-week follow-up in order to reduce some burden related to measure completion. Patients received 1–3 automated email reminders to complete follow-up measures and 1–3 reminder telephone calls when outcome measures were not completed (automated emails and phone calls were not used if measures were completed). To improve measure completion at 52-week follow-up, participants were entered into a draw to win a $150.00 Visa gift card with a 1 in 50 chance to win. The screening and 12, 24, and 52-week measures were administered via REDCap while pre-treatment and 8-week measures were administered using the Online Therapy Unit web platform.

Primary outcomes

Patient Health Questionnaire – 9-item (PHQ-9; Kroenke et al., 2001)

The PHQ-9 is a validated self-report questionnaire that measures depression severity over the past two weeks with nine items rated on a 0 to 3 scale resulting in a total score ranging from 0 to 27. A score of 10 or greater is used to identify a likely diagnosis of depression (Manea et al., 2012) and symptom severity is interpreted as follows: mild (5–9), moderate (10–14), moderately-severe (15–19), and severe (20 or above). The PHQ-9 has good psychometric properties (Kroenke et al., 2010). Cronbach's α for the PHQ-9 in this study ranged from 0.83 to 0.89.

Generalized Anxiety Disorder – 7-item (GAD-7; Spitzer et al., 2006)

The GAD-7 is a validated self-report questionnaire that screens for anxiety severity over the past two weeks using seven items rated 0 to 3 creating a total score ranging from 0 to 21. Scores of 10 are greater have been used to identify cases of generalized anxiety disorder (Spitzer et al., 2006) and symptom severity is interpreted as follows: mild (5–9), moderate (10–14), and severe (15–21). The GAD-7 has strong psychometric properties (Spitzer et al., 2006). Cronbach's α for the GAD-7 in this study ranged from 0.88 to 0.90.

Secondary outcomes

Kessler Distress Scale (K10; Kessler et al., 2002)

The K10 is a 10-item well validated self-report questionnaire measuring psychological distress; items are rated 1 to 5 resulting in a total score ranging from 10 to 50 (Kessler et al., 2002). Cronbach's α for the K10 in this study ranged from 0.88 to 0.92.

Panic Disorder Severity Scale Self-Report (PDSS-SR; Shear et al., 2001)

The PDSS-SR is a psychometrically sound 7-item questionnaire assessing symptoms of panic disorder on a 0 to 4 scale, with the total score ranging from 0 to 28. A score of 8 or greater is used to identify those who likely have panic disorder (Shear et al., 2001). Cronbach's α for the PDSS-SR in this study ranged from 0.89 to 0.91.

Social Interaction Anxiety Scale-6 and Social Phobia Scale-6 (SIAS-6/SPS-6; Peters et al., 2012)

The SIAS-6/SPS-6 consists of 12 items rated 0 to 4, summed to create a total score ranging from 0 to 48 (Johnston et al., 2013). The SIAS-6/SPS-6 has been found to be a reliable and valid measure of social anxiety (Peters et al., 2012). A cut-off score of ≥7 on the SIAS-6 and ≥2 on the SPS-6 is used to identify those likely experiencing social anxiety disorder (Peters et al., 2012). Cronbach's α in this study ranged from 0.86 to 0.93 for the SIAS-6 and from 0.91 to 0.93 for the SPS-6.

Life Events Checklist for DSM-5 (LEC-5; Weathers et al., 2013a)

The LEC-5 assessed for history of 16 potentially traumatic experiences (e.g., natural disaster, fire or explosion, transportation accident) as well as presence of other extraordinarily stressful events not captured by the 16 items. If patients endorsed more than one event, they were asked to indicate which of the events was most distressing.

Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5; Weathers et al., 2013b)

Patients who endorsed distress related to a traumatic event on the LEC were administered the PCL-5, which is a 20-item self-report questionnaire assessing symptoms of posttraumatic stress disorder using a 0 to 4 scale. Items are summed to create a score ranging from 0 to 80. As recommended by Weathers et al. (2013b) a cut-off of 33 or greater was used in this study to identify those with a likely diagnosis of PTSD. The PCL-5 has strong psychometric properties (Blevins et al., 2015). Cronbach's α for the PCL-5 in this study ranged from 0.93 to 0.95.

Sheehan Disability Scale (SDS; Sheehan, 1983)

The SDS is a 3-item validated measure of functional impairment in work/school, social life, and family life with items rated 0 to 10, resulting in a total score ranging 0 to 30. Cronbach's α for the SDS in this study ranged from 0.82 to 0.89.

Treatment perceptions and experiences

Credibility and Expectancy Questionnaire (CEQ; Devilly and Borkovec, 2000)

During the online screening and at 12-week follow-up, the 3-item treatment credibility CEQ subscale (rated 1 to 9) was administered along with one additional item assessing treatment expectancy from 0% to 100%. The first three items were summed to create a total score ranging from 3 to 27. Cronbach's α for the credibility score ranged from 0.79 to 0.82 in this study.

Working Alliance Inventory – Short Revised (WAI-SR; Munder et al., 2010)

At 8-weeks post-treatment, the 12-item WAI-SR was administered with items rated 1 to 5. The WAI-SR results in three subscales assessing agreement on therapy tasks, agreement on therapy goals, and bond between the therapist and patient. Subscale scores range from 4 to 20 and the total score ranges from 12 to 60. In this study, Cronbach's α for the total score and subscale sores ranged from 0.89 to 0.94.

Treatment experiences

Patients answered a series of questions to assess treatment experiences at 8-weeks post-treatment. Using a scale from 1 (“very dissatisfied”) to 5 (“very satisfied”), patients rated satisfaction with treatment overall, satisfaction with materials, and satisfaction with emails from and telephone calls with their therapist. Patients also rated if treatment was worth their time (yes/no) and if they would recommend the course to a friend (yes/no). Two additional questions assessed confidence to manage symptoms (rated 1 “greatly reduced” to 5 “greatly increased”) and motivation to seek treatment if needed in the future (rated 1 “greatly reduced” to 5 “greatly increased”). Patients were also asked about preferences for frequency of therapist emails (none, automated, once a week, twice a week) and phone contact (none, occasional if requested, occasional if requested or clinically indicated, regular weekly phone calls). To assess negative effects, patients were also asked whether they experienced any unwanted negative effects or events that they associated with taking part iCBT (yes/no), how much of an impact the negative effects/events had on their life (rated 0 to 3), and how much the negative effect or event continued to impact their life (rated 0 to 3).

Engagement

Engagement was measured by examining the percentage of patients who accessed each lesson, as well as number of emails sent to therapist, emails from therapist, phone calls with therapist, and log-ins to the web application.

Therapist time

Therapists working in the specialized clinic tracked the amount of time they spent providing support to patients each week. Included in the timing was time to review the web application for patient progress, questionnaires and emails, time to compose emails, and time to speak with the patient by telephone and complete associated notes.

Therapist focus group

Once all patients were enrolled in the trial, the four therapists, employed by the specialized iCBT clinic, who provided 1W + 1BD therapist and 1W therapist support participated in a 90-minute focus group held at the specialized clinic. The focus group was led by an experienced female researcher with an MA in applied psychology employed by the specialized clinic to conduct qualitative research; the researcher was well known to the therapists. Topics of interest were circulated to therapists in advance including: therapist perceptions of positive and negative aspects of 1W + 1BD therapist support compared to 1W therapist support for both therapists and patients; the impact of 1W + 1BD support on therapist email behaviours, such as building rapport, providing psychoeducation and facilitating understanding (Hadjistavropoulos et al., 2018b); and therapist perspectives of the ratio of treatment cost to treatment benefit for 1W + 1BD therapist support.. An audio recorder and scribe/note taker were used to ensure data accuracy.

Data analysis

The groups were compared on background variables, engagement, and treatment experiences using general linear models for continuous variables and chi-square tests for categorical variables. When tests were significant, post hoc analyses were conducted to examine group differences. Alpha was adjusted from 0.05 to 0.01 as a partial control for the number of analyses conducted.

A series of generalized estimation equation (GEE) models were then used to examine average group change in symptoms over time while accounting for within-subject variance through an unstructured working correlation with robust error estimation (Hubbard et al., 2010; Liang and Zeger, 1986). Each model specified a gamma scale to address the positive skewness of scores within bounded symptom scales together with a log link function to account for the pattern of proportional symptoms reduction in line with past internet-delivered therapy research (Karin et al., 2018b).

Consistent with intention-to-treat principles (Hollis and Campbell, 1999), missing values were generated for all dependent variables. Demographic and symptom variables predicting missing values were examined to determine suitability of a missing at random assumption (MAR; Little et al., 2014). Consistent with previous missing cases' research (Karin et al., 2018a), these analyses identified the number of lessons completed as the single dominant predictor of missing data at post-treatment (Wald's χ2 = 347.6, p < .001, Nagelkerke R Square = 60.7%) and 12, 24, and 52-week follow-up respectively (Wald's χ2 = 188.0, p < .001, Nagelkerke R Square = 34.7%; Wald's χ2 = 167.10, p < .001, Nagelkerke R Square = 31.3%; Wald's χ2 = 112.6, p < .001, Nagelkerke R Square = 21.8%), suggesting that a MAR assumption would be suitable on the condition that the replacement of missing cases outcome would be stratified (adjusted) by an individual's rate of lesson completion. Considering this, a multiple imputation procedure was employed to replace missing cases, controlling for the variables of group, time-point, lesson completion rate, and all possible two-way interactions.

To compare the outcomes among the conditions, GEE analyses were conducted examining treatment group (1W + 1BD specialized; 1W specialized; 1W community) and time (pre-treatment, 8, 12, 24, and 52-weeks post-enrollment). Estimated marginal means from the GEE analyses were used to test the rate of change over time within each group, as well as to compare the rate of change between the groups as a percentage change estimate and Cohen's d effect sizes. Estimates were generated for the pre-treatment to 8-week and pre-treatment to 52- week time window together with 95% confidence intervals. The exception to the above was the PCL-5, which was not measured at 8-week follow-up.

Consistent with past research (Hadjistavropoulos et al., 2016; Titov et al., 2015a, Titov et al., 2015b), we compared groups on reliable change on the primary outcome measures at post-treatment and 52-week follow-up. For the PHQ-9, reliable recovery was defined as patients scoring > 9 at pre-treatment, <10 at post-treatment, and having at least a 6-point decrease on the PHQ-9; reliable improvement was defined as a decrease of at least 6 points; deterioration was defined as an increase of 6 points or more; and no change was defined as a change of 5 or fewer points in either direction. For the GAD-7, reliable recovery was defined as patients scoring > 9 at pre-treatment, <10 at post-treatment, and having at least a 4-point decrease on the total GAD-7 score; reliable improvement was defined as a decrease of at least 4 points; deterioration was an increase of 4 or more points; and no change was defined as a change of 3 or fewer points in either direction.

A power analysis was conducted to determine the minimal group differences that the current sample could refute under circumstances where Time by Group tests show non-significance (Type II error). The power analysis followed the DELTA2 guidelines for clinical trials (Cook et al., 2018) and was conducted with the “Longpower” software package in the R language (Donahue and Edland, 2016). The analysis used the pre-treatment to 8-week rate of change within the 1W specialized group as a standard effect. The resulting analysis demonstrated that the current sample was large enough to refute Time by Group differences in the rate of the primary symptom change that were as little as 15% in the total rate of change (25% of the standard effect).

In a third step, a series of sensitivity analyses were conducted to examine if the rate of symptom change among the groups varied as a function of pre-treatment symptom severity, or as a function of patient emails sent to therapist. In these analyses, three-way interactions were modelled, testing for Time by Group by Moderator effects. The variables of baseline symptoms were modelled as minimal/mild, moderate and severe subgrouping moderator variable, and patient emails were modelled as the number of emails patients sent, grouped into categories 0–1, 2–4, and 5 or more emails.

The focus group was transcribed verbatim, de-identified and analyzed using QSR International's NVivo 12 qualitative analysis software. A descriptive, inductive approach to thematic analysis was used to identify major themes within focus group data (Braun and Clarke, 2006). The facilitator read the focus group transcript closely to obtain an initial impression of the data and then engaged in inductive coding, wherein basic codes that represent each unit of meaning were derived. A second qualitative researcher associated with, but not employed by the Online Therapy Unit, reviewed the focus group transcript and themes were refined. A third researcher then reviewed the themes to sort the individual codes into those pertaining to the patient and those pertaining to the therapist. Two focus group participants served as the final check of themes to ensure they were an accurate reflection of the focus group.

Results

Background characteristics

Patients were on average 36.92 years of age (SD = 13.02), 76% (n = 513) were women, 63.8% (n = 431) were married/common-law, 52% (n = 351) reported some university education, 61.5% (n = 415) were employed part- or full-time, 91.3% (n = 616) were Caucasian and 40.6% (n = 274) lived in a large city. The majority of patients reported taking psychotropic medication (55.1%; n = 372). More than half of the patients had a score ≥10 on the PHQ-9 suggestive of a depressive disorder (65.2%; n = 440), a score of ≥10 on the GAD-7 suggestive of generalized anxiety disorder (66.1%; n = 446), and a score ≥7 on the SIAS-6 and ≥2 on the SPS-6 suggestive of social anxiety disorder (52.4%; n = 354). Less than half of patients reported a score ≥8 on the PDSS-SR suggestive of panic disorder was (48.9%; n = 330), and a positive event on the LEC-5 and a score ≥ 33 on the PCL-5 suggestive of posttraumatic stress disorder (28.9%; n = 195). On average, patients scored above the clinical cut-off on 2.63 (SD = 1.51) of measures with 11.7% (n = 79) of patients scoring in the nonclinical range on all measures. No statistically significant group differences were observed (see Table 1).

Primary outcomes

The means, standard deviations, percentage reductions, and Cohen's d effect sizes for the PHQ-9 and GAD-7 for each of the three groups are presented in Table 2. The GEE analyses revealed statistically significant Time effects for the GAD-7 (p < .001) and PHQ-9 (p < .001). There were no statistically significant Time by Group interactions from pre-treatment to post-treatment or pre-treatment to 52-week follow-up (p range: .09–.84) with one exception. There was one Time by Group interaction observed for the PHQ-9 from pre-treatment to 52-week follow-up (p = .009). Subsequent examination of differences showed that the within-group effect sizes from pre-treatment to 52-week follow-up were larger in the 1W + 1BD specialized clinic group (M: 12:45 vs 6.1 resulting in 51% reduction) than the 1W community clinics group (M: 11:63 vs 6.5 resulting in 44% reduction; Cohen's d = 0.21, 95% CI = 0.02–0.40).

Table 2

Means, 95% confidence intervals, and effect sizes (Cohen's d) for primary and secondary outcomes by group with multiple imputations pooling.

	Estimated marginal means					Percentage changes (%) from pre-treatment		Within-group effect sizes from pre-treatment
	Pre-treatment	8-week follow-up	12-week follow-up	24–week follow-up	52-week follow-up	To post-treatment	To 52-week follow-up	To post-treatment	To 52-week follow-up
Primary outcomes
PHQ-9
1W + 1BD S	12.45 (11.7–13.2)	6.22 (5.6–6.9)	6.44 (5.8–7.1)	6.11 (5.4–6.8)	6.10 (5.4–6.8)	50 (45–55)	51 (45–57)	1.20 (1.00–1.40)	1.22 (1.02–1.42)
1W S	12.19 (11.4–13.0)	6.21 (5.5–7.0)	6.13 (5.3–6.9)	6.08 (5.1–7.0)	5.48 (4.9–6.1)	49 (43–55)	55 (50–60)	1.09 (0.88–1.29)	1.27 (1.06–1.48)
1W C	11.63 (10.9–12.4)	6.79 (6.1–7.5)	6.41 (5.6–7.2)	6.52 (5.7–7.4)	6.5 (5.8–7.2)	42 (36–48)	44 (38–50)	0.92 (0.72–1.12)	0.96 (0.76–1.16)
GAD-7
1W + 1BD S	12.1 (11.4–12.8)	5.9 (5.2–6.6)	6.05 (5.4–6.7)	5.08 (4.5–5.7)	5.46 (4.8–6.1)	51 (46–57)	55 (49–60)	1.31 (1.10–1.51)	1.39 (1.18–1.60)
1W S	12.63 (12.0–13.3)	6.11 (5.2–7.0)	5.94 (5.2–6.7)	5.75 (5.1–6.4)	5.33 (4.7–5.9)	52 (45–59)	58 (53–63)	1.30 (1.08–1.51)	1.55 (1.33–1.77)
1W C	11.75 (11.1–12.4)	6.28 (5.5–7.0)	5.85 (5.2–6.5)	5.48 (4.8–6.2)	5.64 (5.0–6.3)	47 (40–53)	52 (47–57)	1.12 (0.91–1.32)	1.27 (1.06–1.47)
Secondary outcomes
K10
1W + 1BD S	28.82 (27.9–29.8)	20.93 (19.9–21.9)	20.91 (20.0–21.8)	19.28 (18.4–20.2)	19.86 (18.8–20.9)	27 (24–31)	31 (28–35)	1.06 (0.86–1.26)	1.24 (1.04–1.44)
1W S	28.20 (27.9–29.8)	20.93 (19.9–21.9)	20.91 (20.0–21.8)	19.28 (18.4–20.2)	19.86 (18.8–20.9)	27 (23−30)	33 (28–37)	0.97 (0.77–1.17)	1.23 (1.02–1.44)
1W C	27.85 (26.9–28.8)	21.49 (20.2–22.8)	20.94 (19.9–22.0)	19.18 (18.2–20.1)	20.18 (19.0–21.3)	23 (18–27)	28 (23−32)	0.86 (0.66–1.05)	1.02 (0.82–1.22)
PDSS-SR
1W + 1BD S	7.68 (6.9–8.4)	4.63 (4.1–5.2)	4.25 (3.7–4.8)	3.28 (2.7–3.8)	3.62 (3.0–4.3)	40 (32–47)	53 (44–61)	0.60 (0.41–0.79)	0.81 (0.61–1.00)
1W S	7.06 (6.3–7.8)	4.57 (3.9–5.2)	3.48 (2.8–4.1)	3.7 (2.9–4.5)	3.16 (2.5–3.8)	35 (26–45)	55 (46–65)	0.47 (0.27–0.66)	0.79 (0.59–0.99)
1W C	7.90 (7.2–8.6)	5.49 (4.8–6.2)	4.7 (4.0–5.4)	3.42 (2.9–3.9)	3.72 (3.1–4.4)	30 (21–40)	53 (45–61)	0.47 (0.28–0.66)	0.84 (0.65–1.04)
SIAS-6/SPS-6
1W + 1BD S	14.98 (13.5–16.4)	10.9 (9.7–12.1)	7.29 (6.3–8.3)	6.33 (5.5–7.1)	6.74 (5.8–7.7)	27 (19–35)	55 (45–61)	0.41 (0.22–0.59)	0.92 (0.72–1.11)
1W S	14.69 (13.2–16.1)	11.69 (10.4–13.0)	6.93 (5.8–8.0)	6.23 (5.3–7.1)	6.36 (5.2–7.5)	20 (12–29)	57 (49–64)	0.29 (0.1–0.49)	0.95 (0.75–1.15)
1W C	14.89 (13.4–16.4)	11.98 (10.6–13.3)	7.36 (6.3–8.4)	6.33 (5.3–7.3)	6.91 (5.9–7.9)	20 (10–29)	54 (47–60)	0.29 (0.1–0.47)	0.89 (0.69–1.08)
PCL-5a
1W + 1BD S	32.98 (30.4–35.5)	–	24.95 (20.8–29.1)	22.1 (19.2–25.0)	22.62 (20.1–25.1)	–	30 (23–38)	0.61 (0.42–0.80)	0.66 (0.47–0.85)
1W S	35.09 (32.2–38.0)	–	24.95 (20.8–29.1)	22.1 (19.2–25.0)	22.62 (20.1–25.1)	–	34 (20–47)	0.71 (0.51–0.91)	0.68 (0.48–0.88)
1W C	36.01 (32.7–39.3)	–	25.41 (21.8–29.0)	22.56 (20.5–24.6)	26.08 (23.5–28.7)	–	27 (20–34)	0.86 (0.67–1.06)	0.61 (0.42–0.80)
SDS
1W + 1BD S	17.31 (16.3–18.3)	11.44(10.2–12.7)	9.87 (8.8–11.0)	8.07 (7.1–9.0)	8.47 (7.5–9.5)	34 (27–41)	51 (45–57)	0.73 (0.54–0.92)	1.14 (0.94–1.34)
1W S	16.51 (15.4–17.6)	11.08 (9.8–12.4)	8.84 (7.8–9.9)	8.19 (6.9–9.5)	8.35 (7.2–9.5)	33 (25–41)	49 (42–57)	0.67 (0.47–0.86)	1.03 (0.83–1.24)
1W C	16.89 (15.9–17.8)	11.76 (10.6–12.9)	10.1 (9.0–11.2)	8.28 (6.9–9.7)	9.12 (8.0–10.3)	30 (23–37)	46 (39–53)	0.67 (0.48–0.86)	0.99 (0.79–1.19)

Note. 1W = once-weekly; 1BD = one-business-day; S = specialized clinic; C = community clinics; PHQ-9 = Patient Health Questionnaire-9; GAD-7 = Generalized Anxiety Disorder-7; K10 = Kessler-10; PDSS-SR = Panic Disorder Severity Scale-Self Report; SIAS-6/SPS-6 = Social Interaction Anxiety Scale-6 and Social Phobia Scale-6; PCL-5 = Posttraumatic Stress Disorder Checklist for DSM-5. SDS = Sheehan Disability Scale; 95% confidence intervals are shown in square parentheses for the estimated marginal means, percentage changes, and effect sizes.

PCL-5 was administered at online screening and not pre-treatment and was not administered at 8-week follow-up.

Secondary outcomes

The means, standard deviations, percentage reductions, and Cohen's d effect sizes for the secondary measures for each of the three groups are also presented in Table 2. The GEE analyses revealed statistically significant Time effects for all variables (p < .001), but no significant interactions between Group and Time from pre-treatment to post-treatment and pre-treatment to 52-week follow-up (p range: .18–.84).

Clinical significance

Percentage change and within group effect sizes from the GEE models are shown in Table 2. For all groups at post-treatment and 52-week follow-up, large within-group effect sizes (Cohen's d: 0.92–1.55) were observed on the PHQ-9, GAD-7, and K10. Medium within-group effect sizes were observed for the PDSS-SR and SDS at post-treatment (Cohen's d: 0.7–0.73), but were large at 52-week follow-up (Cohen's d: 0.79–1.14). On the SIAS-6/SPS-6 effects were initially small at post-treatment (Cohen's d: 0.29–0.41), but were large at 52-week follow-up (Cohen's d: 0.89–0.95). For the PCL-5, the effects were moderate from screening to 52-week follow-up (Cohen's d: 0.61–0.68).

Table 3 includes information about the proportion of patients who demonstrated reliable recovery, reliable improvement, deterioration, or no change on the PHQ-9 and GAD-7 at post-treatment and at 52-week follow-up. In the case of the GAD-7, groups did not differ on these measures at post-treatment or 52-week follow-up. Overall, 45.6% of patients demonstrated reliable recovery, 66% reliable improvement, 31.4% no change and 2.7% reliable deterioration on the GAD-7 at post-treatment. The rates were very similar for the GAD-7 at 52-week follow-up, with 48.3% of patients demonstrating reliable recovery, 65.8% reliable improvement, 29.7% no change and 4.4% reliable deterioration. On the PHQ-9 at post-treatment, while reliable recovery (34.8% all groups) and reliable deterioration (3% all groups) did not significantly differ among groups, patients in the 1W community clinics group had a lower rate of reliable improvement (38.5%) than patients in the 1W and 1W + 1BD specialized clinic groups (50.8% and 52.3% respectively) as well as a higher rate of no change (58.8%) than the 1W and 1W + 1BD specialized clinic groups (46.1% and 44.5% respectively). By 52-week follow-up, however, no group differences were observed among groups on the PHQ-9 reliable change measures. Overall, on the PHQ-9 at 52-week follow-up, 40.2% of patients demonstrated reliable recovery, 49.2% reliable improvement, 47.6% no change and 3.2% reliable deterioration.

Table 3

Reliable recovery, reliable improvement, no change, and deterioration on the PHQ-9 and GAD-7 at post-treatment and at 52-week follow-up using the imputed dataset.

	All groups%	1W + 1BD S%	1W S%	1WC%	Significance
Post-treatment
PHQ-9
In clinical range at assessment	65.2	69.5	64.4	61.5	χ2(2) = 3.35; p = .19
Reliable recovery	34.8	39.6	37.1	27.8	F(2, 84.5) = 3.07; p = .05
Reliable improvement	47.2	52.3	50.8	38.5	F(2, 2103) = 5.05; p = .007
No change	49.8	44.5	46.1	58.8	F(2, 1377) = 5.28; p = .005
Reliable deterioration	3.0	3.2	3.1	2.7	F(2, 7068) = 0.24; p = .79
GAD-7
In clinical range at assessment	66.1	65.2	69.9	63.3	χ2(2) = 2.28; p = .32
Reliable recovery	45.6	47.3	48.2	41.4	F(2, 443) = 1.17; p = .31
Reliable improvement	66.0	66.5	67.2	64.2	F(2, 176) = 0.17; p = .84
No change	31.4	30.5	30.6	33.1	F(2, 97.7) = 0.05; p = .95
Reliable deterioration	2.7	3.0	2.2	2.7	F(2, 34.7) = 0; p = 1
52-week follow-up
PHQ-9
In clinical range at assessment	65.2	69.5	64.4	61.5	χ2(2) = 3.35; p = .19
Reliable recovery	40.2	45.2	40.4	34.8	F(2, 24.9) = 1.42; p = .26
Reliable improvement	49.2	51.7	50.0	45.9	F(2, 351.7) = 0.72; p = .49
No change	47.6	44.8	48.3	49.7	F(2, 42.8) = 2.51; p = .78
Reliable deterioration	3.2	3.5	1.7	4.3	F(2, 16.6) = 0.46; p = .63
GAD-7
In clinical range at assessment	66.1	65.2	69.9	63.3	χ2(2) = 2.28; p = .32
Reliable recovery	51.7	52.9	54.0	48.3	F(2, 57.7) = 0.40; p = .67
Reliable improvement	69.7	70.9	72.6	65.8	F(2, 385.7) = 1.11; p = .33
No change	26.6	24.7	25.4	29.7	F(2, 154) = 0.60; p = .55
Reliable deterioration	3.6	4.4	2.0	4.4	F(2, 31.0) = 0.64; p = .54

Note. 1W = once-weekly; 1BD = one-business-day; S = specialized clinic; C = community clinics; PHQ-9 = Patient Health Questionnaire-9; GAD-7 = Generalized Anxiety Disorder-7.

Sensitivity analyses

The longitudinal Time by Group models, that included a further subgroup analysis of patient emails to therapist (Time by Group by Patient Emails), demonstrated non-significant three-way interactions on primary outcomes at post-treatment (p: .07–.64) and 52-week follow-up (p: .04–.19). Furthermore, the longitudinal Time by Group models that included a further analysis of the severity of pre-treatment depression symptoms (i.e., PHQ-9 minimal/mild, moderate, or severe) demonstrated there were no significant three-way interactions at post-treatment or 52-week follow-up (p < .84–.94). A similar analysis of subgrouping by GAD-7 pre-treatment symptoms (GAD-7 minimal/mild, moderate, severe) identified a significant three-way interaction at post-treatment (p < .01), but not at 52-week follow-up (p = .02). Review of the GAD-7 three-way interaction suggested that patients with severe pre-treatment GAD-7 reported greater GAD-7 symptom improvement at post-treatment when they received 1W + 1BD support (62%) compared to 1W support (specialized clinic 50%; community clinics 52%). Given that the difference was quite minimal (i.e., all three treatment groups reported a ≥50% reduction and the differences were not maintained at 52-week follow-up), it was regarded as non-meaningful. The estimated marginal means, percentage changes from pre-treatment to post-treatment and 52-week follow-up, and significance of the Time by Group by Moderator interactions are included in the Supplementary file.

Engagement

Table 4 provides information on patient engagement in iCBT for all patients. No significant group differences were found in the proportion of patients who completed all 5 lessons or outcome measures at any of the time points. There were also no differences among groups in number of log-ins or days between first and last-log-in. There was a significant difference in number of emails sent to patients with follow-up tests showing therapists in the 1W + 1BD group sent more emails to patients (M: 13.17) than therapists in both 1W groups (M: 8.94 specialized clinic and 8.95 community clinics). Furthermore, there was a significant group difference in number of patient emails sent to therapists, with follow-up tests showing patients in the 1W + 1BD group sent more emails to their therapist (M: 5.11) than patients in both 1W groups (M: 3.78 specialized clinic and 3.94 community clinics). In terms of phone calls, a significant difference emerged, with follow-up tests demonstrating that the 1W community clinics group received fewer phone calls (M: 0.69) than the 1W specialized clinic group (M: 1.06).

Table 4

Program engagement by group.

Variable	All groups (n = 675)		Once-weekly/one-business-day support specialized clinic (n = 233)		Once-weekly support				Statistical significance
					Specialized clinic (n = 216)		Community clinics (n = 226)
	n	%	n	%	n	%	n	%
Engagement
Completion of 5 lessons	476	70.5	162	69.5	149	69.0	165	73.0	χ2(2, N=675) = 1.13; p = .57
Mean number of messages from therapist (SD)	10.40 (3.41) 2–27	–	13.17 (4.01)5–27	–	8.94 (1.75)2–15	–	8.95 (1.79)2–16	–	F2,673 = 179.17; p < .001
Mean number of messages sent to therapist (SD)	4.29 (4.00) 0–35	–	5.11 (5.09)0–35	–	3.78 (2.90)0–14	–	3.94 (3.48)0–27	–	F2,673 = 7.71; p < .001
Mean number of log-ins (SD)	20.49 (12.63)3–87	–	22.39 (14.55)4–87	–	19.65 (11.17)3–57	–	19.34 (11.58)3–84	–	F2,673 = 4.11; p = .02
Mean number of phone calls with therapist (SD)	0.91 (1.32)0–12	–	0.98 (1.32)0–6	–	1.06 (1.55)0–12	–	0.69 (1.01)0–7	–	F2,673 = 4.72; p = .01
Mean days between first and last log-in (SD)	122.66 (109.17)0–466	–	113.33 (102.67)3–426	–	129.66 (109.25)0–421	–	125.58 (115.26)0–466	–	F2,673 = 1.38; p = .25

Treatment experiences

Ratings of treatment experiences made by patients who completed post-treatment measures are reported in Table 5. No group differences were observed on the WAI-SR total scale or subscales, ratings of treatment credibility, treatment satisfaction, improved confidence, motivation to seek additional treatment, or negative effects of treatment. The one exception was that patients in the 1W + 1BD specialized clinic group reported greater satisfaction with therapist phone calls than both 1W groups (85.7% very satisfied/satisfied 1W + 1BD specialized clinic versus 66.7% 1W specialized clinic and 66.3% 1W community clinics). Overall, review of ratings suggested high treatment satisfaction and low level of negative effects. There were also no significant differences among groups on preferences for the frequency of therapist contact. Across treatment groups, most patients indicated a preference for 1W emails from therapists (overall 67%) and occasional phone contact either requested by patient or therapist (overall 48.8%) (Table 5).

Table 5

Treatment experiences reported by participants completing post-treatment measures by group.

Variable	All groups (N = 488)		Once-weekly/one-business-day support specialized clinic (n = 166)		Once-weekly support				Statistical significance
					Specialized clinic (n = 156)		Community clinics (n = 166)
	n	%	n	%	n	%	n	%
Working Alliance Inventory-Short Revised
Mean WAI-SR total score (SD)	46.19 (10.58)12–60	–	47.63 (9.91)13–60	–	46.01 (11.16)15–60	–	44.88 (10.58)12–60		F2,673 = 2.97; p = .05
Mean WAI-SR bond score (SD)	16.91 (3.82)4–20	–	17.23 (3.71)4–20	–	16.88 (4.02)4–20	–	16.60 (3.7)4–20		F2,673 = 1.15; p = .32
Mean WAI-SR task score (SD)	14.95 (3.61)4–20	–	15.36 (3.26)4–20	–	14.95 (3.79)4–20	–	14.52 (3.75)4–20		F2,673 = 2.36; p = .10
Mean WAI-SR goal score (SD)	14.33 (4.58)4–20	–	15.03 (4.26)4–20	–	14.18 (4.77)4–20	–	13.75 (4.64)4–20		F2,673 = 3.56; p = .03
Treatment ratings
Posttreatment credibility	23.17 (5.32)		23.25 (5.48)		23.48(5.21)		22.80 (5.26)		F2,673 = 0.64; p = .53
Satisfied/very satisfied overall	417	85.5	147	88.6	131	84.0	139	83.7	χ2(2, N=488) = 1.95; p = .38
Satisfied/very satisfied materials	437	89.6	152	91.6	138	88.5	147	88.6	χ2(2, N=488) = 1.16; p = .57
Satisfied/very satisfied phone calls	217	69.1	90	85.7	66	66.7	61	66.3	χ2(2, N=296) = 10.25; p = .006
Satisfied or very satisfied emails	402	82.4	142	82.5	127	81.4	133	80.1	χ2(2, N=488) = 1.83; p = .40
Increased/greatly increased confidence	439	90.0	150	90.4	138	88.5	151	91	χ2(2, N=488) = 0.92; p = .63
Increased/greatly increased motivation for other treatment	405	83	143	86.1	123	78.8	139	83.7	χ2(2, N=488) = 3.13; p = .21
Course was worth the time (%)	466	95.5	162	97.6	146	93.6	158	95.2	χ2(2, N=488) = 3.05; p = .22
Would recommend course to friend (%)	476	97.5	163	98.2	151	96.8	162	97.6	χ2(2, N=488) = 0.66; p = .72
Negative effects
Reported negative effects from treatment (%)	46	9.4	13	7.8	16	10.3	17	10.2	χ2(2, N=488) = 3.05; p = .22
Impact negative	0.66 (1.03)0–3	–	0.58 (1.00)0–3	–	0.68 (1.02)0–3	–	0.72 (1.06)0–3	–	F2,673 = 0.73; p = .48
Ongoing impact negative effects	0.36 (0.65)0–3	–	0.33 (0.65)0–3	–	0.35 (0.64)0–2	–	0.42 (0.67)0–3	–	F2,673 = 0.79; p = .46
Contact preferences
Prefer no email	10	2.0	6	3.6	1	0.6	3	1.8	χ2(2, N=488) = 3.62; p = .16
Prefer automated emails	40	8.2	6	3.6	17	10.9	17	10.2	χ2(2, N=488) = 7.07; p = .03
Prefer once-weekly email	327	67.0	124	74.7	97	62.2	106	63.9	χ2(2, N=488) = 6.83; p = .03
Prefer twice-weekly email	111	22.7	30	18.1	41	26.3	40	24.1	χ2(2, N=488) = 2.14; p = .34
Prefer no phone contact	16	3.3	7	4.2	4	2.6	5	3.0	χ2(2, N=488) = 0.69; p = .71
Prefer occasional phone contact self-directed	145	29.7	50	30.1	44	28.2	51	30.7	χ2(2, N=488) = 0.26; p = .88
Prefer occasional phone contact self- and therapist-directed	238	48.8	85	51.2	73	45.8	80	48.2	χ2(2, N=488) = 0.66; p = .72
Prefer regular weekly phone call	89	18.2	24	15.4	35	22.4	30	18.1	χ2(2, N=488) = 1.91; p = .38

Therapist time

On average, therapists from the specialized clinic spent 155 (SD: 75) minutes providing support to each patient in the 1W + 1BD group compared to 109 (SD: 53) minutes for each patient in the 1W group, which was a significant difference (F1,497 = 61.06, p < .001; See Table 6).

Table 6

Means and standard deviations for therapist time spent on patients per week by group.

	Specialized clinic				Statistical significance	Effect size (partial eta-squared)
	Once-weekly/one-business-day (n = 251)		Once-weekly (n = 248)
	Min–max	MinutesMean (SD)	Min–max	MinutesMean (SD)
Week 1	0–93	23 (15)	0–67	16 (9)	F1,497 = 36.77; p < .001	0.069
Week 2	0–74	19 (15)	0–81	13 (10)	F1,497 = 36.78; p < .001	0.069
Week 3	0–61	19 (13)	0–61	11 (8)	F1,497 = 50.92; p < .001	0.093
Week 4	0–62	17 (13)	0–56	12 (9)	F1,497 = 31.35; p < .001	0.059
Week 5	0–64	18 (14)	0–64	13 (9)	F1,497 = 19.78; p < .001	0.038
Week 6	0–62	17 (13)	0–63	12 (9)	F1,497 = 28.31; p < .001	0.054
Week 7	0–78	15 (10)	0–51	12 (9)	F1,497 = 15.40; p < .001	0.030
Week 8	0–77	15 (11)	0–64	12 (8)	F1,497 = 12.57; p < .001	0.025
Week 9	0–58	11 (10)	0–50	8 (10)	F1,497 = 12.44; p < .001	0.024
Overall total	0–413	155 (75)	0–344	109 (53)	F1,497 = 61.06; p < .001	0.109

Focus group

Impact on patients

Results of the focus group are summarized in Table 7. Therapists shared that the differences between 1W + 1BD and 1W were only apparent among patients who sent more than one email a week, otherwise the two conditions were perceived to be identical. For patients who tended to email their therapist more often, therapists perceived the increased frequency of emails in 1W + 1BD condition provided a greater opportunity for patients to build rapport with their therapist and to do so quickly. Similarly, for patients who tended to email their therapist more than 1W, therapists perceived 1W + 1BD support helped patients with skill development; they noted that a 1W + 1BD support provided the opportunity for back and forth discussions while the issue was current rather than a post-hoc discussion at a later date when the issue was no longer relevant.

Table 7

Therapist perceptions of benefits and challenges of once-weekly plus onebusiness0day (1W + 1BD) versus once weekly (1W) therapist support.

Theme	Subtheme	Description	Quote
Patient
Benefits	Greater and faster rapport with therapist	- Greater and faster rapport with therapist among patients who email multiple times each week	- I felt that I built a stronger rapport. My approach was the same, I just found there's more opportunity in 1BD. (Therapist #3)
	Greater and faster skill development	- Greater and faster skill development among patients who email multiple times each week	- I had a patient who, for graded exposure, she wanted to get out more because she stayed inside and isolated quite a bit. So throughout the course of a week, she worked on that and, we were having some back and forth about ‘Today I just put on my shoes, and walked around the house in my shoes.’ And two days later she was like, ‘Ok, I've been going outside and I'm standing outside.’ Only guessing, [but] I would think that she used the [graded exposure] skill more intensely and quicker because we were actually working through it in real time that week. (Therapist #2)
Challenges	Greater dependency	- Lower independence in learning strategies among those who email multiples each week	- I felt that sometimes I was almost building dependency. I noticed my higher anxiety patients were not self-soothing or using self-regulation kind of skills, they were so quick to reach out for help immediately … I didn't have the freedom to ask [patients] to take some space on their own to deal with it because part of our message was ‘I'll be happy to respond within 24 h’. (Therapist #1)
No impact	No impact on course completion	- No impact on course completion rates as both 1W + 1BD and 1W groups are phoned after not logging into system for one week	- I'm not seeing any signs of differences in course completion rates. (Therapist #4)
Therapist
Benefits	Greater therapist satisfaction	- A more natural/responsive email exchange between patient and therapist	- I feel like I'm helping. Whereas sometimes in standard [1W], I kind of feel like a robot. (Therapist #3)
		- More opportunity to build strong rapport, provide psychoeducation and facilitate understanding	- The people who are engaging, you know them better, you know their situation. They are giving you more specific examples, and you can make those connections for them to help bring that skill along, more intensely. (Therapist #4)
		- Greater connection with patient leading to greater excitement when patient has success	- If you're having regular contact with someone, when you see, or hear, or read about their accomplishments, that you're inherently more excited for them. The praise is more genuine. You know instead of just [writing] “Keep it up.” You get excited for what they're doing and seeing them really trying to integrate and use the skills, that they are feeling good about it. (Therapist #2)
		- Greater ease of addressing one issue at a time	- I wasn't typing up my responses … and making sure that I had all their points in my response; I could just reply to the message and hit ‘Reply’. It was easier in that sense that my messages weren't as scripted, less ‘templatey’; sometimes it's easier because you get pointed questions, and you can respond to that question. (Therapist #2)
Challenges	Greater therapist burden	- Days when many patients required 1BD support resulted in rushing and concerns over reduced quality emails	- There were times when it was really challenging to fit in answering 1BD questions and to meet their needs in my schedule … to make sure I was responding, and answering their questions … I'm sure I didn't completely put [in] as much thought and care because I was quickly trying to get through with the responding in the best way I could, in a quick way. (Therapist #3)
		- Increased cognitive burden due to heavy workload when they had many patients who required 1BD support	- [1W] is different because you set your time aside, you get organized and you are all set up, but 1BD is like ‘oh they emailed me’ (Therapist #4)
		- Challenge of managing repeated venting emails and helping patients focus on skill development	- I was satisfying a need on their part to express their frustration, then having a quick response from me just reinforced it… So it was hard to rein that in to a level that I felt worthwhile for the patient and effective… it's a little frustrating to try to juggle everything and not really feeling that there was benefit to it. To spend all that extra time, like I don't think that it was benefitting the patient. (Therapist #1)

In addition to advantages, therapists also described a distinct disadvantage of the 1W + 1BD approach. Specifically, they noted that this faster response from the therapist may have prevented some patients from independently developing skills taught in the course. Another observation made by therapists about a 1W + 1BD support is that the way the protocol was implemented they did not perceive the approach to impact course completion rates. Specifically, it was noted that in both 1W and 1W + 1BD, therapists followed a similar protocol for those who did not login for a week, namely waiting a full week before telephoning patients to encourage patient engagement with the course.

Impact on therapists

Therapists described increased satisfaction with delivering 1W + 1BD therapist support in a number of regards. Therapists highlighted that the communication felt more like a “natural exchange” and they enjoyed being able to communicate more frequently to build a strong rapport with patients, provide extra psychoeducation and facilitate understanding of strategies. Therapists also felt, at times, 1W + 1BD support was easier to provide as therapists often only had to address one issue in each email rather than preparing one weekly email that responded to multiple patient emails that were sent during the previous week.

Despite being satisfied with several aspects of 1W + 1BD support, therapists reported finding the approach burdensome in several ways. Therapists shared that responding to patient emails within 1W + 1BD often created a high volume of work, which meant they felt rushed and that the quality of their emails suffered. Adding to this burden of 1W + 1BD support, therapists described heavy workload days as cognitively demanding, highlighting that it was not as easy to effectively plan and manage completion of emails. Finally, therapists expressed increased burden related to having to navigate how to help some patients who emailed frequently to move from what was perceived to be non-beneficial “venting” to “skill development”. Therapists expressed concern that 1W + 1BD support required them to respond even when they felt that the response was non-beneficial for patients.

When weighing the benefits and challenges of offering therapist support, therapists expressed consensus that the challenges of providing 1W + 1BD therapist support (i.e., high workload resulting in rushing, which impacted quality, cognitive burden, managing patient venting) outweighed the benefits to therapists (e.g., natural exchanges, stronger relationships and ability to help, responding to one issue at a time) and patients (e.g., rapport, skill development). Therapists concluded that if outcomes were indeed improved among patients who received 1W + 1BD therapist support, then it would be necessary to overcome the above noted challenges of offering 1W + 1BD therapist support. In light of the pros and cons, therapists expressed a strong interest in emailing patients twice a week, perceiving this would overcome the challenges of 1W 11BD + 1BD approach while conferring benefits of 1W + 1BD therapist support.

Discussion

This study sought to compare the efficacy, acceptability, feasibility and resources required for iCBT delivered with 1W + 1BD therapist support versus 1W therapist support. Both patients and therapists have recommended that providing more responsive therapist support could improve patient engagement, treatment experiences and outcomes in iCBT. It was hypothesized that providing 1W + 1BD therapist support would require more therapist time, but would result in greater symptom improvement over time, greater engagement, and improved treatment experiences for both patients and therapists, compared to 1W therapist support. As this was a pragmatic trial, 1W + 1BD support was offered by therapists employed by a specialized iCBT clinic and compared to 1W support offered by these same specialized clinic therapists as well as 1W support offered by therapists employed by community mental health clinics, where the focus of services is typically on face-to-face care and only secondarily on iCBT.

Analyses confirmed that 1W + 1BD therapist support resulted in ~4 additional emails (13 versus 9 emails) sent to patients and required ~46 (155 versus 109 min) extra minutes of therapist time over the 8-week treatment period. Nevertheless, 1W + 1BD therapist support was not associated with clinically significant improved outcomes, noticeably greater treatment engagement or improved perceptions of treatment experiences than patients who received 1W therapist support. The only difference that emerged in terms of treatment engagement was that patients who received 1W + 1BD support sent 5 versus 4 emails to their therapist than those who received 1W support.

Across all groups, consistent with past research on transdiagnostic iCBT (e.g., Dear et al., 2015; Hadjistavropoulos et al., 2016, Hadjistavropoulos et al., 2019; Titov et al., 2015a), patients from pre-treatment to post-treatment and pre-treatment to 52-week follow-up demonstrated large reductions in symptoms of depression (within Cohen's d = 0.92–1.27; avg. % reduction ≥ 42%–55%; reliable improvement 39%–52%), generalized anxiety (within Cohen's d = 1.12–1.55; avg. % reduction ≥ 47%–58%; reliable improvement 64%–73%), and psychological distress (within Cohen's d = 0.86–1.24; avg. % reduction ≥ 23%–33%). In the case of depression, there appeared to be a difference between 1W + 1BD support offered by specialized therapists and 1W support offered by community clinic therapists. Specifically, examination of effect sizes, at 52-week follow-up, showed that patients who received 1W + 1BD support from therapists employed by the specialized clinic had significantly greater improvement than patients who received 1W support offered by therapists employed by community clinics. Nevertheless, the difference was not considered clinically meaningful (within Cohen's d 1.22 versus 0.96). When reliable change was examined on the PHQ-9 at post-treatment, some group differences were also observed at this point whereby patients in the 1W community group experienced lower rates of reliable improvement and higher rates of no change compared to other groups. No differences, however, were seen on reliable recovery or reliable deterioration on the PHQ-9 at post-treatment; furthermore, no group differences on any of the measures of reliable change were apparent at 52-week follow-up. Overall, given that differences were not large and were not maintained at 52-week follow-up, the significance of the findings was not considered clinically meaningful.

Also consistent with past research on transdiagnostic iCBT, patients in all groups, from pre-treatment to post-treatment showed significant small to moderate improvements in panic (within Cohen's d = 0.47–0.60; avg. % reduction ≥ 30%–44%), social anxiety (within Cohen's d = 0.29–0.41; avg. % reduction ≥ 20%–27%), and disability (within Cohen's d = 0.67–0.73; avg. % reduction ≥ 30%–34%). Of note, from pre-treatment to 52-week follow-up the improvements on panic, social anxiety and disability improved further and were considered large in magnitude (within Cohen's d = 0.79–1.14; avg. % reduction ≥ 46–57%). Adding to the literature on transdiagnostic iCBT, the findings suggested across all groups, symptoms of posttraumatic stress disorder also improved moderately from pre-treatment to 52-week follow-up (within Cohen's d = 0.61–68; avg. % reduction ≥ 27%–34%).

Based on a sensitivity analysis, no marked evidence was identified to suggest that 1W + 1BD therapist support resulted in better outcomes for patients with differing rates of symptoms prior to treatment. Rather, positive and comparable rates of symptom reductions were observed across the treatment groups, and within each of the symptom severity subgroups (e.g., with mild, moderate and severe symptoms at baseline). Similarly, the sensitivity analysis of patient email frequency demonstrated no clear evidence to suggest a relationship between the number of emails patients sent to their therapist and clinical outcomes. While these sensitivity analyses are limited in their power and therefore their ability to detect nuanced differences, these additional analyses suggest the overall findings are unlikely to be affected by participants' initial symptom severities or the frequency of patient emails in a marked or clinically meaningful way. When examining patient ratings of treatment, there were also very few differences between 1W + 1BD and 1W groups. In both 1W + 1BD therapist support and 1W therapist support, satisfaction with the therapeutic alliance (all dimensions) and various ratings of treatment satisfaction were consistently positive (e.g., 97.5% recommend iCBT to a friend) and unwanted negative effects of treatment were infrequent. The only difference to emerge was that more patients who received 1W + 1BD therapist support rated being satisfied/very satisfied with phone calls from therapist (85%) than patients who received 1W therapist support (66% in both 1W groups), which might be expected given that this group of therapists has greater familiarity with their patients.

We were also interested in exploring therapists' experiences of offering 1W + 1BD therapist support as it could be preferable to increase therapist contact if this is a highly preferred approach from a workforce perspective (Titov et al., 2019). Results of the therapist focus groups, however, suggested that while therapists perceived some benefits to 1W + 1BD therapist support (e.g., related to building rapport, skill development, therapist satisfaction) they also perceived some significant drawbacks and they expressed a clear preference for offering 1W therapist support. In terms of challenges, therapists described having high workload some days that resulted in them rushing and being concerned with the quality of their emails. They also perceived the 1BD therapist support as increasing the likelihood patients vented to their therapists and resulted in therapists having to help patients re-focus on skill development. Schueller et al. (2017) have also highlighted that there may be negative consequences associated with too much support, such as logistical challenges and reduced patient autonomy and learning. On a positive note, therapists perceived that a stronger rapport was built with patients who tended to email to a significant degree and received 1W + 1BD therapist support. Ultimately these observations did not translate into differences in ratings of therapists with one exception being that phone calls with therapists who provided 1W + 1BD therapist support were more positively rated than phone calls with therapists who provided 1W support.

Overall, the results of the current study suggest that providing 1W + 1BD support does not appear to garner benefits originally assumed by patients and therapists. Null findings in this case represent an important contribution to the literature on use of iCBT in routine care settings, wherein past patients and therapists have expressed a desire for increased therapist support. Strategies that increase therapist time need to be supported by evidence and or workforce benefits as increased therapist time significantly reduces the ability to disseminate iCBT in routine practice (Schueller et al., 2017).

Limitations

There are a number of limitations to this trial that need to be considered. We acknowledge that we were missing data from ~23% of patients at post-treatment and ~30% of patients at 52-week follow-up, which may limit the generalizability of our findings. Nevertheless, our completion rates are similar to other studies of iCBT delivered in routine practice (Ruwaard et al., 2012; Titov et al., 2015a). Second, diagnostic interviews were not conducted (because the iCBT program studied is not limited to those who meet diagnostic criteria) and thus we are not able to report on the impact of 1W + 1BD therapist support compared to 1W therapist support on DSM-5 diagnoses. However, previous studies on transdiagnostic iCBT have used a reduction of symptoms of at least 50% as an indicator of clinically significant remission of symptoms (e.g., Kayrouz et al., 2020), which was achieved for the primary outcomes in this study. Although we examined the extent to which baseline symptom severity and patient emails with the therapist moderated outcomes, we acknowledge that it is possible that other characteristics that we did not examine moderated outcomes, such as completion of homework. Thus, future research may identify variables that predict improved outcomes among those receiving 1W + 1BD support compared to 1W support. Finally, we acknowledge that results of the study may not generalize to other routine care clinics or therapists.

Strengths

The present study has a number of notable strengths. First, this study replicates past studies of the transdiagnostic iCBT (e.g., Dear et al., 2015; Hadjistavropoulos et al., 2016, Hadjistavropoulos et al., 2019; Titov et al., 2015a), and outcomes compare positively to other studies of iCBT for depression and anxiety (e.g., Andersson et al., 2019; Carlbring et al., 2018). Second, this study included 52-week follow-up, including assessing posttraumatic stress disorder, which has not been previously reported for this transdiagnostic iCBT course. Third, consistent with recommendations for implementation research (Hermes et al., 2019), the study was conducted as part of routine care and we captured therapist experiences with offering 1W + 1BD versus 1W therapist support through a focus group. Fourth, the findings provide useful information regarding how much extra time on average is needed to deliver 1W + 1BD support. Fifth, by randomly assigning patients to groups, we provided confirmation of past cohort comparisons (Hadjistavropoulos et al., 2016) that outcomes delivered by therapists in community mental health clinics are largely similar to outcomes for therapists employed by a specialized clinic; importantly, these findings suggest that results of iCBT can be obtained by therapists employed by specialized and nonspecialized clinics. The one area potentially worthy of further monitoring, however, is the treatment of depression where patients who received 1W support from therapists employed by the community clinics obtained somewhat lower rates of reliable improvement and somewhat higher rates of no change at post-treatment than patients treated by therapists employed by the specialized clinic offering either 1W or 1W + 1BD support. Sixth, the sample size was large, which gives confidence in the comparison of groups on primary and secondary outcomes. Finally, we not only assessed impact on symptoms but also treatment satisfaction, negative effects, and working alliance.

Future directions

Offering iCBT as part of routine care remains relatively uncommon and there continues to be room for improvement in patient outcomes. Based on the findings from the current study as well as past research (Klein et al., 2009), offering more therapist support is not the answer to improving outcomes. Other options for improving iCBT, therefore, need to be explored, such as improving patient expectations of iCBT as this has been found to predict outcomes in past research (Hedman et al., 2012). It is also possible that further attention could be given to exploring the benefits of offering motivational interviewing as part of iCBT (Beck et al., 2020) or examining methods that could improve the quality of therapist support, such as weekly questionnaires that would allow therapists to gain a better understanding of patient knowledge and use of skills (e.g., Kraepelien et al., 2019). These methods may prove more effective at enhancing therapist ability to respond to unique needs of the patient, which, in turn, could result in improved adherence and outcomes (Preference Collaborative Review Group, 2008). Experimental studies that examine the impact of different types of therapist behaviours (e.g., supportive versus skill-focused behaviours) rather than the amount of therapist support have also been called for as a method for improving iCBT outcomes (Hadjistavropoulos et al., 2018b; Schneider et al., 2016).

Conclusions

In summary, the present study contributes to the existing literature suggesting that outcomes are comparable with 1W + 1BD and 1W therapist support in terms of symptom improvement, engagement, and treatment experiences. The findings provide useful information to those delivering iCBT and suggest that supplementing 1W support with 1BD support, in essence increasing responsiveness, does not improve outcomes beyond those available with 1W therapist support. In addition to requiring more time, therapists described 1W + 1BD support as increasing burden in terms of number of patients to check-in on, decreasing quality of emails on days when they had many patients who required a 1W + 1BD therapist support and increasing the need to help patients move from venting to learning skills. Given the lack of benefits associated with a 1W + 1BD therapist support, other directions need to be explored for improving outcomes beyond increasing responsiveness to emails. The findings add to the existing trials highlighting the significant potential of iCBT in routine practice as a method of increasing access to evidence-based care by demonstrating the impact of transdiagnostic iCBT on symptoms of post-traumatic stress as well as showing that symptom improvements are maintained at 52-week follow-up in routine care.

The following is the supplementary data related to this article.

Supplementary Table 1

Means, confidence intervals, and test statistics by group, with adjustment for messages sent and baseline symptom severity.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that appear to have influenced the work reported in this paper.