My care manager, my computer therapy and me: The relationship triangle in computerized cognitive behavioural therapy.

Previous research has reported mixed findings regarding the relationship between therapeutic alliance, engagement and outcomes in e-mental health. This study aims to overcome some of the methodological limitations of previous research and extend our understanding of alliance-outcome relationships in e-mental health by exploring the nature of the relationship triangle between the patient, their care manager and their computerized cognitive behavioural therapy (CCBT) program, accessed with or without an Internet Support Group (ISG). Positive patient-rated alliance with both their care manager and the CCBT program itself was found and these were significantly associated with measures of engagement and clinical outcome. The magnitude of this association was moderate, and within the range of that reported for traditional face-to-face psychotherapies in recent meta-analyses. Limitations of the study, including the reliance on completer data and a cross-sectional design, and directions for future research are presented. Our findings suggest that both the training and supervision of support staff and the optimization of CCBT interventions themselves to enhance alliance and experience may lead to improved engagement and outcomes. TRIAL REGISTRATION: Clinicaltrials.gov Identifier: NCT01482806https://www.clinicaltrials.gov/ct2/show/NCT01482806?term=rollman&rank=4.

Introduction

There is growing evidence that guided computerized cognitive behavioural therapies (CCBT) can be an effective intervention for common mental health problems, including anxiety and depression (Andersson, 2016, Andersson et al., 2014, Andrews et al., 2010, Grist and Cavanagh, 2013, Newby et al., 2016, Richards and Richardson, 2012). Most of the studies to date have focused on evaluating the feasibility and clinical outcomes of such interventions, whilst limited research has begun to explore the change processes associated with their impact (Cavanagh and Millings, 2013).

Working alliance and guided CCBT

In traditional psychological therapies the quality of ‘common factors’, including the therapeutic relationship, are widely held to be important for patient engagement and clinical outcomes (Horvath et al., 2011, Lambert and Barley, 2001). These include the ‘working alliance’ – a collaborative relationship between the patient and professional consisting of three elements: agreement on the goals of treatment, agreement of the tasks of treatment, and a positive personal/emotional bond (Bordin, 1979). Guided CCBT presents a challenge to the importance of these factors as therapeutic interactions are typically limited, remote, and often asynchronous; for example, communication may occur only by text message or email. CCBT is commonly offered as a ‘predominantly self-guided therapy’ or ‘minimal contact therapy’ (Newman et al., 2011), with contact time ranging from < 1.5 h in total (Andersson, 2009, Titov, 2011) to more active involvement by the clinician, but to a lesser degree than in a traditional therapy for the target problem (Newman et al., 2011). Despite the belief amongst many clinicians that extended face-to-face contact is essential for a meaningful working alliance to be established or maintained (Berger, 2015, Lopez, 2015), where measured, the client-rated relationship appears fairly robust to distance and limited contact. Perhaps surprisingly, given the more limited nature of this contact, where compared, no significant differences in patient-rated alliance have been found between guided CCBT and face-to-face CBT (Kiropoulos et al., 2008).

However, the therapeutic relationship in guided CCBT may be less intimately associated with therapy outcomes than in traditional therapies (Berger, 2015, Cavanagh and Millings, 2013), and mixed findings have been reported on its relationship between the patient-rated therapeutic relationship and outcomes. Andersson et al. (2012) found that whilst working alliance ratings were high in the three samples of participants in their study (with depression (n = 49), generalized anxiety disorder (n = 35) and social anxiety disorder (n = 90)), these ratings were unrelated to self-reported measures of the primary outcome for each group. Similar findings have been reported elsewhere (e.g. Preschl et al., 2011, Richards and Richardson, 2012), whilst other studies have reported significant alliance-outcome relationships (e.g. Nordgren et al., 2013, Wagner et al., 2012). One explanation for this discrepancy in the literature may be that most studies in the field are not adequately designed to detect the relatively moderate correlation that might be expected between working alliance and clinical outcome. In a meta-analysis of > 200 studies of the alliance-outcome relationship in traditional face-to-face psychotherapies Horvath et al. (2011) identified an aggregate effect size of 0.275 (95% CI 0.25–0.30). A priori power calculations indicate that a sample size over 100 is required in order to be robustly designed to test this magnitude of association (G*power calculation, Power 0.9, Alpha 0.05, Faul et al., 2009) - a sample size well beyond most studies in this field. Indeed, Berger (2015) has noted that where the association between alliance and outcome is reported in published studies it is typically in a positive direction, if not statistically significant, supporting the notion that many studies reporting this relationship may simply have inadequate power to test such hypotheses.

The ‘relationship triangle’ in guided CCBT

Most research in this area has focused on testing the reach and relevance of the working alliance between the patient and their ‘CCBT guide’ (i.e. their therapist or coach) in relation to its impact on outcomes for patients (Berger, 2015). In addition to this patient-professional relationship, there is some evidence that CCBT programs themselves can be designed to elicit relational experiences in program users (e.g. Barazzone et al., 2012), and it may be that this patient-program ‘relationship’ also contributes to engagement and outcomes in guided CCBT (Cavanagh, 2010, Cavanagh and Millings, 2013). A minority of studies have adapted measures of the therapeutic relationship to consider the user's experience of relating to the CCBT program itself, and at least some outcomes support the idea of an association between the patient-rated ‘relationship’ with the CCBT program and therapy outcomes (e.g. Berger et al., 2014, Meyer et al., 2015, Ormrod et al., 2010). It has been argued that this patient-program relationship may in fact be more closely associated with patient outcomes than therapist-alliance ratings (Berger et al., 2014, Berger, 2015).

To date no studies have explored both elements of this ‘relationship triangle’ - that is both the nature of the patient-professional relationship and the patient-program relationship within one single study. One possibility is that relating experiences with both the guide and the CCBT program itself may contribute independently or interactively to the user's experience, engagement and outcomes associated with their care episode (Cavanagh, 2010). Furthermore, larger studies are needed to fully test the alliance-outcome relationship in CCBT and in order to draw conclusions about how it compares to the outcome-alliance relationship in traditional psychotherapies.

Aims of the study

In the parent study, the randomized controlled trial “Online Treatments for Mood and Anxiety Disorders in Primary Care” (Online Treatment Trial), participants with anxiety and/or depression were offered access to a guided CCBT program, with or without access to a moderated Internet Support Group (ISG), supported by a care manager in the context of a 6-month collaborative care strategy. Working alliance, engagement, and clinical outcomes were measured in the context of the parent study that showed that participants randomized to guided CCBT report significant 6-month improvements in mood and anxiety symptoms, but not in mental-health related quality of life, compared to those receiving their physician's usual care (Rollman et al., 2017). This study aimed to explore the nature of the relationship between the participant, their care manager and their CCBT program, in a large study well powered to detect alliance-outcome relationships.

Methods

Study setting

The Online Treatment Trial was conducted at 26 primary care offices that shared a common electronic medical record system (EMR; EpicCare, Madison, WI), implemented a protocol approved by the University of Pittsburgh's Institutional Review Board, and described in detail elsewhere (Rollman et al., 2017) (See Fig. 1). Briefly, primary care physicians (PCP) in participating practices received an EMR “Best Practice Alert” reminder about the trial study for all patients aged 18–75 whenever anxiety, generalized anxiety, panic, or depression was entered as an encounter diagnosis. If the patient agreed to a referral, then the PCP forwarded the patient's name and telephone number to a study recruiter who then telephoned the patient to confirm preliminary eligibility (Rollman et al., 2008).

Fig. 1

Flowchart of participants.

1Working Alliance Inventory, ‘Beating the Blues’ Sessions Rating Scale, and Overall Treatment Satisfaction Scale.

Abbreviations: CCBT, computerized cognitive behavioural therapy; GAD-7, 7-item Generalized Anxiety Disorder Scale; ISG, Internet support group; MH, mental health; PCP, primary care physician; PHQ-9, 9-item Patient Health Questionnaire; PROMIS Anxiety, Patient-Reported Outcomes Measurement Information System for Anxiety (fixed length, short form); PROMIS Depression, Patient-Reported Outcomes Measurement Information System for Depression (fixed length, short form), SF-12 MCS, Medical Outcomes Study Short Form Mental Component Scale.

Study recruiters screened referred patients for the presence of an anxiety disorder and/or depression using the Generalized Anxiety Disorder scale (GAD-7; Spitzer et al., 2006) and Patient Health Questionnaire (PHQ-9; Kroenke et al., 2001) respectively. If a patient scored ≥ 10 on either the PHQ-9 or GAD-7, signifying an at least moderately severe symptom level, met all other eligibility criteria (e.g., reliable Internet access, no severe mental health conditions, medically stable), and consented to the trial, recruiters administered the PRIME-MD Anxiety and Mood Modules (Spitzer et al., 1994) to establish a diagnosis and collected information on patients' self-reported race, gender, marital status, employment, and use of pharmacotherapy, followed by the clinical assessment battery.

Protocol-eligible participants were randomized in a 3:3:1 ratio to either: (1) 24/7 access to a CCBT program provided under the guidance of a care manager (CCBT-alone; N = 301); (2) CCBT-alone plus access to a password-protected and moderated Internet Support Group (CCBT + ISG; N = 302); or (3) their PCP's “usual care” (UC; N = 101).

Interventions

Computerized cognitive behavioural therapy

All participants randomized to either of the intervention groups received username and password for “Beating the Blues,” an evidence-based CCBT program (Proudfoot et al., 2004). It consisted of eight 50-min long interactive sessions and “homework” to complete between weekly sessions. Each session included simple and easily understood text, audio, and audiovisual clips to educate and teach basic CBT techniques (e.g., automatic thoughts, thinking errors, attributional styles) for up to three problems that the participant could choose during the first session. At the start of each session participants entered their mood symptoms using the PHQ-9 and their anxiety symptoms with the GAD-7, as well as their perceived burden by their problems.

Internet support group

Participants randomized to the CCBT + ISG study arm, received also access to a password-protected and moderated study ISG and a unique username to preserve confidentiality. The ISG featured a variety of discussion boards on which participants posted and a collection of links to various resources (e.g., local $4 generic pharmacy programs, crisis hotlines, brief YouTube videos on such topics as stress management, sleep hygiene, and exercises). It also had a direct link to the CCBT program to further encourage participants to complete all sessions in a timely manner. Engagement in the ISG was promoted via email notifications of new ISG activities, invited member-guest moderators, and various contests and personalized home pages to encourage log-ins and posts.

Care manager support

College graduates with mental health research experience and trained in a basic understanding of mood and anxiety disorders and the trial's interventions served as care managers. Following randomization, one of the care managers emailed the participant a web-link to the CCBT program, and, if applicable, the ISG, and telephoned the participant for an introductory telephone call to review the program(s) and establish rapport. They encouraged participants to complete a new session every 1–2 weeks, thus ideally finishing the program during the first three months of our interventions.

Later, care managers monitored the progress of their participants using the reports generated by the “Beating the Blues” program (e.g., sessions completed, symptom ratings, stress ratings). They emailed participants a positive feedback after each completed session. If a participant did not complete a new session within two weeks, care managers sent an email reminder, and this was repeated once a week for one month, and then they telephoned participants to motivate them to complete the program or assist with technical problems. If a participant reported a worsening of symptoms or suicidal ideation, the care manager emailed or telephoned the at-risk individual to provide support, reinforce use of the CBT skills, and/or launch our suicide risk management protocol (Herbeck Belnap et al., 2015).

Case review and follow-up

Each care manager presented their participants to the study PCP, psychiatrist, and psychologist-study coordinator (Clinical Team) in separate, weekly 60-min case review sessions using the trial's electronic patient registry that could sort patients by randomization date, last contact, and highest self-rated PHQ-9 or GAD-7 score. In addition to conveying general lifestyle adjustments including exercise and social engagement, the Clinical Team recommended antidepressant/anxiolytic pharmacotherapy based on participants' treatment preferences and response to CCBT; and referrals to mental health specialists (MHS) when they did not improve or had complex psychosocial issues.

Given the trial's collaborative care framework, the care managers notified PCPs of their patients' progress and our treatment recommendations, as they prescribed and approved all adjustments to their patients' pharmacotherapy and the trial never dispersed medication to any participant (Rollman et al., 2003).

Depending upon the participant's treatment choices, symptoms, and motivation, the care manager sent customized emails or telephoned the participant bi-weekly for the first two months of the intervention. These contacts lasted approximately 15 and 30 min, respectively. Afterwards, the care manager contacted the participant approximately monthly until the end of our 6-month intervention. We recorded number of emails, telephone calls, and total contacts to each participant.

Study participants

Participants of the present study were those randomized to one of the intervention study arms as part of the parent trial and completed the ‘end of treatment experience questionnaires.’ Participants assigned to the UC condition are not reported on in this study, as they were not exposed to the CCBT program.

Assessments

Research staff blinded as to the participant's randomization status administered via telephone the clinical outcome assessment battery at baseline, 3-, and 6-months. The battery included the 12-Item Short Form Health Survey (SF-12) to determine health-related quality of life (HRQoL) (Ware et al., 1995) and the fixed length Patient-Reported Outcomes Measurement Information System (PROMIS) measures to assess mood and anxiety symptoms (Pilkonis et al., 2011).

At the end of the 6-month intervention, we emailed all participants from both intervention arms a link to complete measures of the end-of-intervention experience questionnaires. Alternatively, participants could ask to report their ratings via telephone at the conclusion of their assessment to a blinded research assistant who would directly enter the data into an electronic data system.

Working alliance

Participants completed the 12-item client version of the short form of the Working Alliance Inventory (Horvath and Greenberg, 1989, Tracey and Kokotovic, 1989). This scale includes three 4-item subscales measuring i) bond (e.g. I believe that < name > liked me), ii) task (e.g. < name > and I agreed about the things that I should do to help my situation) and iii) goal (e.g. < name > did not understand what I was trying to accomplish in the interventions (reverse scored item)). The name of the participant's care manager was used to tailor these items (Table 4). Responses were rated on a 7 point scale from 0 = ‘never’ to 6 = ‘always’, with questions 4 and 10 reverse coded. Mean item scores are reported. Higher scores reflect more positive ratings. Reported internal consistency of subscales and total scores ranges from 0.90–0.98 (Hanson et al., 2002, Tracey and Kokotovic, 1989), strong convergent validity with the Agnew Relationship Measure has been reported (Stiles et al., 2002, Duncan et al., 2003). For this study, subscale reliability measured by Cronbach's alpha ranged from 0.78–0.89, and total score reliability was 0.90.

Table 4

Effects of treatment measures on 6-month changes in CCBT engagement and clinical outcomes.

Engagement measure: 6-month change in number of BtB sessions completed
Model	Effecta	Estimate (SE)	(95% CI)	P-value
(1)	WAI total	0.45 (0.11)	(0.24, 0.66)	< 0.001
(2)	BTB session total	0.55 (0.07)	(0.41, 0.70)	< 0.001
(3)	BTB session total	0.53 (0.08)	(0.36, 0.69)	< 0.001
(3)	WAI total	0.08 (0.12)	(− 0.16, 0.32)	0.523
(4)	BTB session total	0.01 (0.32)	(− 0.61, 0.63)	0.976
(4)	WAI total	− 0.32 (0.27)	(− 0.85, 0.20)	0.224
(4)	BTB * WAI interaction	0.11 (0.06)	(− 0.02, 0.24)	0.090
(5)	Overall treatment satisfaction	0.36 (0.08)	(0.20, 0.53)	< 0.001
Outcome: 6-month change in SF-12 MCS
(1)	WAI total	2.22 (0.54)	(1.15, 3.28)	< 0.001
(2)	BTB session total	1.69 (0.37)	(0.96, 2.42)	< 0.001
(3)	BTB session total	1.14 (0.43)	(0.31, 1.98)	0.007
(3)	WAI total	1.64 (0.63)	(0.40, 2.87)	0.009
(4)	BTB session total	0.46 (1.61)	(− 2.70, 3.62)	0.776
(4)	WAI total	1.09 (1.37)	(− 1.60, 3.79)	0.427
(4)	BTB * WAI interaction	0.15 (0.33)	(− 0.50, 0.79)	0.659
(5)	Overall treatment satisfaction	1.84 (0.42)	(1.02, 2.66)	< 0.001
Outcome: 6-month change in PROMIS depression
(1)	WAI Total	− 2.08 (0.38)	(− 2.81, − 1.34)	< 0.001
(2)	BTB session total	− 1.49 (0.26)	(− 2.00, − 0.98)	< 0.001
(3)	BTB session total	− 0.99 (0.30)	(− 1.57, − 0.41)	0.001
(3)	WAI total	− 1.49 (0.44)	(− 2.35, − 0.64)	0.001
(4)	BTB session total	0.02 (1.11)	(− 2.17, 2.20)	0.987
(4)	WAI total	− 0.71 (0.95)	(− 2.57, 1.16)	0.459
(4)	BTB * WAI interaction	− 0.21 (0.23)	(− 0.66, 0.23)	0.350
(5)	Overall Treatment Satisfaction	− 1.53 (0.29)	(− 2.10, − 0.95)	< 0.001
Outcome: 6-month change in PROMIS anxiety
(1)	WAI total	− 2.22 (0.41)	(− 3.02, − 1.42)	< 0.001
(2)	BTB session total	− 1.28 (0.28)	(− 1.83, − 0.72)	< 0.001
(3)	BTB session total	− 0.65 (0.32)	(− 1.28, − 0.02)	0.043
(3)	WAI total	− 1.86 (0.47)	(− 2.79, − 0.93)	< 0.001
(4)	BTB session total	− 0.34 (1.21)	(− 2.72, 2.04)	0.781
(4)	WAI Total	− 1.61 (1.04)	(− 3.65, 0.42)	0.120
(4)	BTB * WAI interaction	− 0.07 (0.25)	(− 0.55, 0.42)	0.789
(5)	Overall treatment satisfaction	− 1.60 (0.32)	(− 2.21, − 0.98)	< 0.001

Abbreviations: BtB, Beating the Blues; PROMIS Anxiety, Patient-Reported Outcomes Measurement Information System for Anxiety (fixed length, short form); PROMIS Depression, Patient-Reported Outcomes Measurement Information System for Depression (fixed length, short form); SF-12 MCS, Medical Outcomes Study Short Form Mental Component Scale; WAI, Working Alliance Inventory.

All estimates are derived from 6-month contrasts in the interaction between the listed Effect and Time.

Patient-program alliance

Patient-program alliance was measured using an adapted version of the Session Rating Scale (Duncan et al., 2000). This 4-item measure invites the participant to rate their therapeutic experience in terms of the i) relationship (bond), ii) goals and topics (goals), iii) approaches or methods (tasks), and iv) overall. For each item a positive anchor statement was presented with a 7-point response scale running from 0 = ‘absolutely agree’ to 6 = ‘absolutely disagree.’ The adapted version asked participants to rate their experience of the ‘Beating the Blues’ CCBT program (e.g., The computer therapy program worked on and talked about what I wanted to work on or talk about, Table 3). Internal consistency of the original Session Rating Scale has been reported as 0.88 (Duncan, Miller, Sparks & Johnson, 2003). Moderate concurrent validity for this measure with the Helping Alliance Questionnaire II (r = 0.48, p < 0.01) has been reported (Duncan et al., 2003). For this study, reliability measured by Cronbach's alpha was 0.92 for the “Beating the Blues” version.

Table 3

Treatment Experience Ratings.

	Survey completers (N = 394)
Work Alliance Inventorya
a) Task scale, mean (SD)	4.8 (1.2)
b) Bond scale, mean (SD)	5.0 (1.1)
c) Goal scale, mean (SD)	5.0 (1.1)
d) Total working alliance scale, mean (SD)	4.9 (1.0)
BtB Sessions Rating and Overall Satisfactionb
a) I felt heard, understood and respected by the computer therapy program, mean (SD)	4.6 (1.5); n = 385
b) The computer program worked on and talked about what I wanted, mean (SD)	4.5 (1.4); n = 385
c) The computer therapy approach is a good fit for me, mean (SD)	4.0 (1.9); n = 385
d) Overall, the computer therapy program was right for me, mean (SD)	4.0 (1.9); n = 384
Sessions Rating BtB Total score (a-d), mean (SD)	17.1 (5.9); n = 384c
e) I felt heard, understood and respected by the study intervention, mean (SD)	5.0 (1.2); n = 385
f) We worked on and talked about what I wanted, mean (SD)	4.8 (1.3); n = 384
g) The therapy approach is a good fit for me, mean (SD)	4.4 (1.7); n = 385
h) Overall, the study intervention was right for me, mean (SD)	4.4 (1.7); n = 385
Overall treatment satisfaction score (e-h), mean (SD)	18.6 (5.3); n = 384c

Abbreviation: BtB, Beating the Blues.

All Working Alliance questions are coded here as: 0 = Never … 6 = Always.

All Sessions Rating questions are coded here as: 0 = Absolutely Disagree … 6 = Absolutely Agree.

When comparing the mean session ratings survey completers to a midpoint score of 12 using a one-sample t-test, the p-value < 0.0001 (for both a-d total score and e-h total score).

Overall treatment satisfaction

Overall treatment satisfaction was measured using an adapted version of the Session Rating Scale (Duncan et al., 2000; see above). This adapted version asked the patient to rate their overall experience of treatment received through the study intervention (Table 3). For this study, reliability measured by Cronbach's alpha was 0.92 for the Treatment Overall version.

Treatment engagement

Treatment engagement was measured by number of log-ins to the “Beating the Blues” program, number of sessions completed, total number of care manager contacts (by email and telephone), and total number of care manager contacts pertaining to “Beating the Blues.”

Statistical analyses

The reliability of Working Alliance Scale, Session Rating Scale for Beating the Blues, and Session Rating Scale for Overall Experience were assessed using Cronbach's Alpha. This was calculated overall as well as for the subscales of the Working Alliance Scale (i.e. Task, Bond, Goal).

Based on the results of the parent study (Rollman et al., 2017) we combined the two intervention groups CCBT-alone and CCBT + ISG, as they did not differ on their baseline characteristics, nor did we find any significant differences in clinical outcomes at 6-months between the 2 groups. All further analyses were conducted using treatment measure completers of both intervention groups.

Univariate associations between 6-month changes in clinical outcomes and the Beating the Blues Sessions Rating, Overall Treatment Satisfaction, and Working Alliance Inventory scales and subscales were assessed using Spearman correlation coefficients (rs).

Survey completers were compared to non-completers on baseline characteristics, levels of CCBT engagement, care manager contacts at 6-months. Two-sample t-test and chi-square tests (or their nonparametric counterparts) were used to determine statistical significance for between-group comparisons of continuous and categorical measures, respectively.

Linear mixed models were used to assess the 6-month changes in outcomes (SF-12 MCS, PROMIS Mood, PROMIS Anxiety) between survey completers and non-completers. We included predictors for age group and referral site size, as they were fixed by experimental design, as well as a random effect for subject. Additionally, we adjusted for gender, PHQ-9, and prior treatment due to between-group imbalances.

Associations between 6-month changes in clinical outcomes and each of the WAI and Beating the Blues Sessions Rating scale total scores were assessed using linear mixed models with fixed effects for month, Beating the Blues Session Ratings, WAI, and all 2- and 3-way interactions. Of interest was whether there were additive and/or multiplicative effects of both rating scales on outcomes as well as on ‘Beating the Blues’ engagement (# of sessions completed). In addition, associations between 6-month changes in clinical outcomes and CCBT engagement and the Overall Treatment Satisfaction scale total score were examined using linear mixed models with fixed effects for month, Overall Treatment Satisfaction, and their 2-way interaction. All reported estimates, derived from linear combinations of model coefficients, are unstandardized.

All analyses were conducted using SAS version 9.4 (SAS Institute, Cary, NC, USA) with a type I error rate of 5% with no adjustment for multiplicity.

Results

Description of treatment measures completers vs. non-completers

Of the 603 participants randomized to the CCBT arms of the parent study 65.3% (394) completed the end of treatment questionnaires (treatment measures). They were 77.2% female, 82.0% white, and had a mean age of 43.0 years (SD 14.4 years).

As shown in Table 1, participants who completed the treatment measures did not differ in age, level of education, diagnostic status GAD-7 score, and pharmacotherapy use from non-completers at baseline, but they were more likely to be male, less likely to have received treatment for depression in the past year and had a significantly lower baseline PHQ-9 score than the non-completers. During the 6-months intervention period, non-completers withdrew from the study significantly more often than completers (29% vs. 1%).

Table 1

Baseline characteristics and engagement of completers and non-completers of the treatment measures.

	Completers(N = 394)	Non-completers(N = 209)	P
Baseline characteristics
Age, mean (SD)	43.0 (14.4)	42.5 (13.8)	0.74
Male, n (%)	90 (22.8)	33 (15.8)	0.04
White, n (%)	323 (82.0)	176 (84.2)	0.49
Education
≥ High school, n (%)	380 (96.5)	202 (96.7)	0.90
≥ 4 yr. college degree, n (%)	189 (48.0)	92 (44.0)	0.35
PHQ-9, mean (SD)	12.9 (4.9)	14.0 (5.0)	0.01
GAD-7, mean (SD)	12.9 (4.2)	12.7 (4.7)	0.60
Dx on PRIME-MD
Anxiety only, n (%)	33 (8.4)	12 (5.7)	0.24
Depression only, n (%)	71 (18.0)	39 (18.7)	0.85
Depression and anxiety, n (%)	272 (69.0)	154 (73.7)	0.23
Treatment in past 1 year, n (%)	269 (68.3)	161 (77.0)	0.02
Pharmacotherapy use, n (%)	304 (77.2)	168 (80.4)	0.36
Withdrawn, n (%)	2 (0.5)	61 (29.2)	< 0.001
Treatment engagement
CCBT program
Logged in, n (%)	363 (92.1)	158 (75.6)	< 0.001
Completed ≥ 1 session, n (%)	356 (90.4)	148 (70.1)	< 0.001
Completed all 8 sessions, n (%)	192 (48.7)	29 (13.9)	< 0.001
Care management
Total contacts, median [IQR]	15 [13,19]	12 [9,16]	< 0.001a
Email contacts, median [IQR]	11 [8,14]	9 [5,12]	< 0.001a
Telephone contacts, median [IQR]	4 [3,6]	3 [2,5]	< 0.001a
Contacts pertaining to CCBT program, median [IQR]	7 [4,8]	2 [3,6]	< 0.001a

Abbreviations: CCBT, computerized cognitive behavioural therapy; GAD-7, 7-item Generalized Anxiety Disorder Scale; IQR, interquartile range; PRIME-MD, Primary Care Evaluation of Mental Disorders; PHQ-9, 9-item Patient Health Questionnaire.

Note: p-values for Contacts were obtained using Wilcoxon rank-sum tests.

Compared to those who did not complete the treatment measures, participants who completed them were significantly more likely to have logged on to the CCBT program (92.1%), completed at least one session (90.4%), and were more likely to have completed all 8 sessions within the 6-months intervention period (48.7%). Completers had more care manager contacts (median 15, IQR 13–19), and more pertaining to the CCBT program (7, IQR 4–8) (Table 1).

Participants who completed the treatment measures also reported significantly higher improvements in all clinical outcomes at the end of the 6-months intervention compared to those who did not complete the surveys (Table 2). Full analysis of clinical outcomes is reported in Rollman et al., 2017.

Table 2

Improvement of clinical outcomes of treatment measures completers vs. non-completers.

Estimated 6-month change	Completers (N = 394)	Non-completers (N = 209)	Completers vs. non-completers mean difference [95% CI]	P
SF-12 MCS, mean (SE)	12.90 (0.55)	9.81 (0.91)	3.09 [1.00, 5.18]	0.004
PROMIS Mood, mean (SE)	− 9.14 (0.38)	− 7.22 (0.53)	− 1.92 [− 3.35, − 0.48]	0.009
PROMIS Anxiety, mean (SE)	− 9.76 (0.41)	− 6.64 (0.67)	− 3.12 [− 4.68, − 1.57]	< 0.001

Abbreviations: PROMIS Anxiety, Patient-Reported Outcomes Measurement Information System for Anxiety (fixed length, short form); PROMIS Depression, Patient-Reported Outcomes Measurement Information System for Depression (fixed length, short form), SF-12 MCS, Medical Outcomes Study Short Form Mental Component Scale;

Treatment measures, patient engagement and clinical outcomes

All subsequent analyses in this paper refer to participants who completed the treatment measures only.

Survey completers reported a positive working alliance with their care manager on all subscales and overall (mean = 4.9, SD 1.0). Ratings of both the CCBT program and overall treatment satisfaction also indicated positive experiences in survey completers (Table 3).

Treatment experience and engagement with CCBT program

The number of ‘Beating the Blues’ sessions completed was significantly associated with the Working Alliance Inventory (WAI) total score (rs = 0.18 p ≤ 0.001), as well as all WAI subscale scores (Bond rs = 0.11, p = 0.03; Task rs = 0.17 p = 0.001; Goal rs = 0.18, p < 0.001). Number of sessions completed was also significantly associated with Beating the Blues Sessions Rating total score (rs = 0.32, p < 0.0001), and the Overall Treatment Satisfaction score (rs = 0.23, p < 0.0001).

As presented in Table 4, increases in WAI total score were associated with a 0.45 (CI 0.24, 0.66; p < 0.001) point increase in sessions completed. Similarly, Beating the Blues Sessions Rating and Overall Treatment Satisfaction ratings showed increases in sessions completed, 0.55 (CI 0.41, 0.70; p < 0.001) and 0.36 (CI 0.20, 0.53; p < 0.001), respectively. However, in the main effects model of working alliance and program alliance, only the Beating the Blues Sessions Rating was significantly associated with CCBT sessions completed (Table 4, Model 3). Furthermore, no significant overall interaction effect was found (Table 4, Model 4).

Treatment experience and clinical outcomes

Working Alliance Inventory total scores were significantly associated with symptom improvements at 6 months (depression, rs = 0.25, p < 0.0001; anxiety, rs = 0.25, p < 0.0001; SF-12 MCS, rs = 0.19, p < 0.001). Beating the Blues Session Ratings were significantly associated with symptom improvements at 6 months (depression, rs = 0.28, p < 0.0001; anxiety, rs = 0.26, p < 0.0001; SF-12 MCS, rs = 0.22, p < 0.001). Overall Treatment Satisfaction ratings were significantly associated with symptom improvements at 6 months (depression, rs = 0.25, p < 0.0001; anxiety, rs = 0.26, p < 0.0001; SF-12 MCS, rs = 0.21, p < 0.001).

Increases in WAI total score were associated with a 2.22 (CI 1.15, 3.28; p < 0.001) point increase in SF-12 MCS improvement, and with a significant point decrease in mood (− 2.08; CI − 2.80, − 1.34; p < 0.001), and anxiety symptoms (− 2.22; CI − 3.02, − 1.42; p < 0.001). Similarly, increases in the Beating the Blues Sessions Ratings and Overall Treatment Satisfaction score were related with a significant SF-12 MCS improvement (1.69; and 1.84 p < 0.001), and a mood (− 1.49; and − 1.53; p < 0.001) and anxiety symptom (− 1.28; and − 1.60; p < 0.001) decease. For all three clinical outcomes measures, there were significant main effects of WAI and Beating the Blues Sessions Rating scores (Table 4, Model 3). We did not find any significant interaction effects for any of the clinical outcomes (Table 4, Model 4).

Discussion

This study appears to be the first to directly measure the patient-rated therapeutic alliance, patient-program alliance and overall treatment satisfaction for study participants with anxiety and/or depression accessing a CCBT program, in the context of a 6-month collaborative care strategy. With > 600 participants randomized to receive CCBT and around two-thirds completing the end-of-intervention treatment experience measures it is also sufficiently powered to test our hypotheses about the association of treatment experience with participant engagement and clinical outcomes.

Participants who completed the treatment experience measures were more likely to be male, have received a recent mental health treatment, and reported fewer depression symptoms at baseline. Not surprisingly, participants who did not complete the measures showed less engagement with the CCBT program or even withdrew from the study, had fewer interactions with the care managers, and reported worse clinical outcomes at 6-months. Interestingly, survey non-completers showed similar modest improvements as participants randomized to the ‘usual care’ group in the parent trial, although the majority of survey non-completers engaged in at least one CCBT session and they had numerous contacts with their care managers (Rollman et al., 2017). Based on our results we cannot discern whether participants who do not engage with a guided CCBT program may have disengaged because they did not agree with the treatment package or they felt that they had received enough care.

Relationship triangle

Participants' ratings of their working alliance with the care manager, who supported their use of the CCBT program, indicated a positive experience of alliance. These findings echo previous research supporting the idea that a positive working alliance can be established despite the rather limited contact associated with supporting a CCBT program (e.g. Andersson et al., 2012, Kiropoulos et al., 2008). Ratings across task, bond and goal sub-scales were similar, contrasting with Berger's (2015) hypothesis that bond subscale ratings are necessarily lower for interventions using CCBT.

In general, participants also rated positively the relationship, goals, tasks and overall experience with the CCBT program itself. They felt heard, understood and respected by the CCBT program, that the program ‘talked about’ what they wanted to, and was a good fit to their needs. This accords with previous research reporting positive patient-program relationships in the context of CCBT (e.g. Nordgren et al., 2013, Ormrod et al., 2010). Participants rated the overall treatment approach somewhat higher than their experience with the CCBT program alone, (18.6 vs. 17.1, Table 3) suggesting that interactions with the care-manager and additional components of the collaborative care intervention enhanced participants experience of their treatment overall.

Engagement

We found statistically significant relationships between the number of CCBT sessions completed and measures of working alliance, patient-program alliance and overall treatment experience. This dose-response effect echoes previous research suggesting that patient-provider relationship is associated with mental health service engagement outcomes more broadly (Marsh et al., 2012, Thompson and McCabe, 2012). Furthermore, the patient-program alliance may be more closely associated with CCBT engagement than the patient-provider alliance. This supports previous commentary (Berger, 2015). However, the cross-sectional nature of this data means that it is not possible to make causal inferences about the direction of this relationship. Longitudinal research is needed to unpick the sequence of events.

Clinical outcomes

We found statistically significant relationships between each treatment measure and improvements on our clinical outcome measures (SF-12 MCS, PROMIS depression and anxiety). Correlations between alliance and outcome measures were in the small-medium range and similar to those found for the alliance-outcome relationship in traditional psychotherapy studies (mean rs = 0.275, 95% CI 0.25–0.30; Horvath et al., 2011). This supports the idea that both a positive patient-provider and patient-program alliance may be typical of CCBT interventions and contribute to clinical improvements, but remain undetected in some smaller studies (Berger, 2015).

Linear mixed model analysis indicated that patient-provider and patient-program alliance ratings may contribute independently and additively rather than interactively to clinical outcomes and program engagement.

Study limitations

Only two thirds of participants randomized to the CCBT arms of the parent trial completed the treatment measures reported in this manuscript, had lower depression baseline scores, and were more likely to be generally more engaged with the trial's interventions than those who did not complete the survey. Given these differences between completers and non-completers caution in the interpretation of all further analysis is advised. In particular, it is possible that non-completers felt a poorer alliance and were less satisfied with the care package than completers. Second, treatment measures were collected at a single time-point at the end of the 6-months intervention which may inflate the correlation between the alliance and outcomes and limit confidence in our interpretation of these variables as predictors of engagement or outcome. Further research using a longitudinal design with multiple measurement points for treatment experience is needed to test the limits of our findings. Third, the study relied on adaptations of measures developed to capture relational aspects of ‘psychotherapy’ (WAI and SRS). As the nature of the relationship between a patient and computer program or remote care manager may be expected to differ from that of a traditional therapeutic relationship, research is needed to further understand and measure these constructs. As with any study measuring the therapeutic relationship in the context of an effective psychological intervention, a halo effect on measurement is possible. Finally, we combined the intervention group that had access to both the CCBT program and the ISG with the group who had access to the CCBT program alone. Although these two groups did not differ in baseline characteristics, program engagement, and improvements in clinical outcomes (Rollman et al., 2017), the availability of the ISG may have influenced the participants' working alliance with the care manager that may affect the generalizibility of our findings.

Future research implications

This study highlights the need for future studies to be designed to adequately test questions relating to patient-professional and patient-program alliance. Like alliance-outcome relationships in traditional face-to-face psychotherapies, the association may be consistent but moderate in magnitude. Small N studies are unlikely to be sufficiently robust to test questions in this domain.

Longitudinal research is required to untangle the relationship between patient-provider and patient-program relationships, engagement and outcomes. It may be the case that fuller engagement with the program leads to an improved therapeutic experience, or that a better therapeutic experience leads to greater engagement with the program or an interaction of these two effects best explains the patient journey.

Finally, adding provider perspectives on the therapeutic/working alliance could augment future studies.

Clinical implications

As patient-program alliance appears to be at least moderately associated with engagement and clinical outcomes for guided CCBT programs, ensuring that these programs are optimized to elicit a positive relational experience by tailoring them to a patient's specific needs (e.g., sociodemographic characteristics, medical conditions, cultural and linguistic preferences) that maximizes felt collaboration in terms of goals, tasks and bond may help to ensure the greatest patient benefit from these interventions.

Furthermore, since the patient-provider alliance appears to be similar to face-to-face psychotherapy, the training and supervision of support staff to enhance alliance and experience may lead to improved engagement and outcomes.

Conclusions

This study aimed to overcome some of the methodological limitations of previous research and extend our understanding of alliance-outcome relationships in e-mental health by exploring the nature of the relationship triangle between the patient, their care manager and their CCBT program. We found a positive participant-rated alliance with both their care manager and the CCBT program itself and these were each significantly associated with measures of engagement and clinical outcome. The magnitude of this association was moderate, and within the range of that reported for traditional face-to-face psychotherapies. These findings add importantly to the knowledge base in this research domain, and set the ground work for future research and practice. Further research is needed to ensure optimal service provision in e-mental health that acknowledges the role of both patient-professional but also patient-program alliance.

Funding

This work was supported by NIH grant R01 MH093501 from the National Institute of Mental Health.

Declaration of interest

The first author (KC) has acted as a research consultant to Ultrasis plc, the company that owned and marketed the British version of Beating the Blues at the time this study commenced.

All work described was supported by a grant from the National Institute of Mental Health (R01 MH093501). The funding source had no role in the design, conduct, or reporting of our study or in the preparation, review, or decision to submit this manuscript for publication.