Explaining variability in therapist adherence and patient depressive symptom improvement: The role of therapist interpersonal skills and patient engagement.

Understanding why therapists deviate from a treatment manual is crucial to interpret the mixed findings on the adherence-outcome association. The current study aims to examine whether therapists' interpersonal behaviours and patients' active engagement predict treatment outcome and therapist adherence in cognitive behaviour therapy (CBT) and mindfulness-based cognitive therapy (MBCT) for depressive symptoms. In addition, the study explores rater's explanations for therapist nonadherence at sessions in which therapist adherence was low. Study participants were 61 patients with diabetes and depressive symptoms who were randomized to either CBT or MBCT. Depressive symptoms were assessed by the Beck Depression Inventory-II. Therapist adherence, therapist interpersonal skills (i.e., empathy, warmth, and involvement), patients' active engagement, and reasons for nonadherence were assessed by two independent raters (based on digital video recordings). Therapist adherence, therapists' interpersonal skills, and patients' active engagement did not predict posttreatment depressive symptom reduction. Patients' active engagement was positively associated with therapist adherence in CBT and in MBCT. This indicates that adherence may be hampered when patients are not actively engaged in treatment. Observed reasons for nonadherence mostly covered responses to patient's in-session behaviour. The variety of reasons for therapist nonadherence might explain why therapist adherence was not associated with outcomes of CBT and MBCT.

RCT Entities:   Population Interventions Outcomes

Therapist adherence was not associated with posttreatment depressive symptom improvement after CBT and MBCT

Patient engagement was positively associated with therapist adherence to CBT and MBCT

A broad variety of patient‐related reasons for therapist nonadherence were observed, of which some may not result in poorer treatment outcomes and may rather reflect therapist flexibility.

INTRODUCTION

Adherence by therapists to treatment manuals is recommended to ensure that the intervention is carried out as intended and produces the aimed therapy effects (Moncher & Prinz, 1991; Perepletchikova, Treat, & Kazdin, 2007). Although several studies showed that the extent to which therapists adhere to a treatment manual is predictive of subsequent symptom change in cognitive therapy for depression (DeRubeis & Feeley, 1990; Feeley, DeRubeis, & Gelfand, 1999; Strunk, Brotman, & DeRubeis, 2010; Strunk, Cooper, Ryan, DeRubeis, & Hollon, 2012), a meta‐analysis found no association between therapist adherence and treatment outcome (Webb, DeRubeis, & Barber, 2010). The role of therapist adherence in psychological treatments may in fact be more complex (Owen & Hilsenroth, 2014). Different sources of variability in adherence may be differently related to treatment outcomes. Understanding why therapists deviate from a treatment manual is therefore crucial to be able to interpret the mixed findings on the adherence–outcome association.

Research on predictors of variability in therapist adherence is scarce. Boswell et al. (2013) showed that patient self‐reported interpersonal aggression predicted poorer therapist adherence in cognitive behavioural therapy (CBT) for panic disorder. In a study on process variables in CBT for patients with binge‐eating disorder (Brauhardt et al., 2014), patient and therapist characteristics did not predict therapist adherence to CBT. Other studies showed that a more positive working alliance as perceived by patients is associated with higher therapist adherence (Owen & Hilsenroth, 2011; Tschuschke et al., 2015; Weck, Grikscheit, Jakob, Höfling, & Stangier, 2015), whereas the direction of the interaction between adherence and the therapeutic alliance in predicting treatment outcomes seems to be inconsistent (Barber et al., 2006; Weck et al., 2015).

Yet little is known about how therapists' and patients' in‐session behaviour relates to therapist adherence and treatment outcome. Assessing actual therapist and patient behaviour during treatment sessions and relating these behaviours to adherence and treatment outcome may provide more insight in why adherence is not a consistent predictor of treatment effectiveness.

One reason adherence may not predict treatment outcome is that poor adherence may reflect therapists' flexibility in employing other techniques to responsively meet the needs of each individual patient (Owen & Hilsenroth, 2014). This might also be understood as a sign of therapeutic competence. Therapists may not be inclined to follow a treatment manual when they believe it negatively affects the therapeutic alliance between them and a particular patient (Addis & Krasnow, 2000). For example, if a patient experienced a stressful event during the past week, the therapist may focus more on conveying understanding, empathy, and warmth, than on employing prescribed treatment techniques. Displaying such interpersonal skills—also referred to as Rogerian conditions (Zuroff, Shahar, Blatt, Kelly, & Leybman, 2016), facilitative interpersonal skills, or non‐specific factors—may be important for treatment efficacy, as the extent to which the patient experiences that the therapist conveys empathy, genuineness, and positive regard are associated with positive treatment outcomes (Elliott, Bohart, Watson, & Greenberg, 2011; Keijsers, Schaap, & Hoogduin, 2000; Zuroff & Blatt, 2006). Thus, if therapists spend time on conveying empathy, warmth, or involvement instead of employing prescribed techniques (i.e., low adherence), patients may still benefit from psychological treatment, despite low levels of therapist adherence.

Alternatively, even when therapists intend to deliver the treatment as intended, their ability to do so effectively might be hampered by the extent to which patients' are actively engaged during treatment sessions. Patients who engage less actively during treatment have been shown to benefit less form treatment (Gaston, Thompson, Gallagher, Cournoyer, & Gagnon, 2010; Gomes‐Schwartz, 1978; Zuroff et al., 2000). When patients are not actively engaged in treatment, they might respond poorly to the therapist or refuse to cooperate with certain procedures of the treatment, eventually resulting in lower therapist adherence. A recent study indeed indicated that in‐session resistant behaviour of patients interfered with therapist adherence to CBT for panic disorder (Zickgraf et al., 2015). Furthermore, patient contributions to the therapeutic alliance, which includes patients' active engagement in treatment, was associated with therapist adherence in CBT and interpersonal psychotherapy for bulimia nervosa (Loeb et al., 2005). Studies including ratings of patients' in‐session behaviour are needed to examine whether patients' engagement accounts for variability in therapist adherence on the one hand and outcomes of psychological treatments for depressive symptoms on the other hand.

The current study aims to examine in‐session behaviour of therapists and patients, including therapists' interpersonal skills and patients' active engagement, and their association with therapist adherence and outcome of two psychological treatments for depressive symptoms. The study is embedded in a randomized controlled trial on the efficacy of individually delivered CBT and mindfulness‐based cognitive therapy (MBCT) for diabetic patients with co‐morbid depressive symptoms. Previously, we found that both interventions were efficacious in reducing depressive symptoms (Tovote et al., 2014). The effect sizes of CBT and MBCT were comparable (Tovote et al., 2014), and the beneficial effects were sustained at 9‐month follow‐up (Tovote et al., 2015).

First, it is examined whether therapist adherence, therapists' interpersonal skills (i.e., conveying empathy, warmth, and involvement), and patients' active engagement during treatment session are associated with a pre‐post change in depressive symptoms. Second, we examine whether therapists' interpersonal skills and patients' active engagement are associated with therapist adherence. In addition, to gain more detailed insight in why therapists deviate from treatments manuals, we explore rater's explanations for therapist nonadherence at sessions in which therapist adherence was low. These research aims were examined in CBT and MBCT separately because the adherence–outcome association and reasons for nonadherence might be therapy specific.

METHODS

Procedures and participants

The current study is part of a multicenter, randomized controlled trial comparing the efficacy of CBT and MBCT for depressive symptoms to a waiting‐list control condition. The study was registered on ClinicalTrials.gov (NCT01630512), and the study design was described by Tovote et al. (2013). The study protocol was approved by the Medical Ethical Committee of the University Medical Center Groningen. All study procedures were in accordance with the ethical standards of the Medical Ethical Committee of the UMCG and with the Declaration of Helsinki of 1975, as revised in 2000. Informed consent was obtained from all study participants included in the study.

An elaborate description of the flow of participants and procedures can be found elsewhere (Tovote et al., 2014). In short, patients with diabetes (type I or type II) and co‐morbid depressive symptoms (Beck Depression Inventory‐II [BDI‐II] ≥ 14) participated in a randomized controlled trial consisting of three arms: CBT, MBCT, and a 3‐month waiting‐list control group. After 3 months, patients allocated to the waiting‐list control group were randomized to CBT or MBCT. Exclusion criteria were age < 18 and >70, inability to read or write, pregnancy, severe psychiatric co‐morbidity, acute suicidal ideations, having received psychological treatment within 2 months prior to inclusion, and unstable use of antidepressants within 2 months prior to inclusion.

In the initial study, 94 patients were randomized, but only 68 participants gave written informed consent on recording their treatment sessions. Participants who provided informed consent were on average older (t = 2.3, df = 92, p = 0.03) and were diagnosed more often with diabetes type II (χ 2 = 5.1, p = 0.02) than participants who did not provide consent for recording their treatment sessions. Of these 68 participants, two did no longer have elevated BDI‐II scores (>14) after the waiting period and were therefore excluded from the current study. Of the remaining 66 participants, four participants received one or no treatment sessions, and the videotapes of one participant were missing. In total, the sample in the current study consisted of 61 participants (CBT: N = 32, MBCT: N = 29).

Treatments

Participants received either CBT based on the treatment manual of Beck, Rush, Shaw, and Emery (1979) or MBCT (Schroevers et al., 2015) based on the standardized group MBCT manual developed by Segal, Williams, and Teasdale (2002). Both treatments were individually delivered and consisted of eight weekly sessions of 45 to 60 min. The structured CBT and MBCT treatment manuals prescribed the themes, exercises, and homework assignments for each session. CBT included activity monitoring, scheduling and performing pleasant or functional activities, identifying and challenging dysfunctional thoughts, and relapse prevention. CBT homework included one to two exercises a day (approximately half an hour a day).

MBCT included formal mindfulness exercises (i.e., guided meditation/yoga such as the body scan and mindful stretching), informal exercises (e.g., 3‐min breathing space and mindfulness of a routine activity) and relapse prevention assignments. For this individual format of MBCT, the in‐session practice length of the original exercises was reduced to about 20 min as well as the duration of the inquiry (for a detailed description of the individual MBCT manual, see Schroevers et al., 2015, and Tovote et al., 2014). Furthermore, the original cognitive exercise in Session 2 and the relapse prevention within Session 7 were not included. Homework included both formal exercises and informal exercises for approximately 30–45 min a day.

Therapists and training

CBT was delivered by 12 therapists, and MBCT was delivered by nine therapists. To enhance treatment differentiation, therapists who delivered CBT did not deliver MBCT, and the other way around. All therapists had received clinical training. The MBCT therapists all had participated in a mindfulness‐based treatment as a participant and practiced mindfulness in daily life. Therapists received a structured treatment manual including specific instructions on exercises, inquiry, and homework assignments per session. The CBT manual was based on the treatment manual of Beck et al. (1979), and the individualized MBCT manual (Schroevers et al., 2015) was based on the manual of Segal, Williams, et al. (2002). Therapists with fewer than 3 years of experience in CBT or MBCT received 2 days of training including mainly role playing. All therapists received supervision once every 3 weeks. The CBT training and supervision was provided by a licensed clinical psychologist and CBT therapist (fourth author) with more than 35 years of experience in providing CBT supervision. The MBCT training and supervision was provided by a mental health psychologist (second author) who received extensive training in MBSR/MBCT and has provided more than 25 mindfulness programmes in the past 7 years. Therapists provided treatment to a minimum of two patients and a maximum of eight patients, with a median of four treated patients per therapist.

Digital video recordings

All treatment sessions were recorded with camcorders to reduce differential adherence (Perepletchikova & Kazdin, 2005). For each patient, an early treatment session (i.e., Session 2) and a late treatment session (i.e., Session 6) were selected, as sampling from one session may be unrepresentative (Moncher & Prinz, 1991). When a session was not available, the recording of the next session was selected (Session 3: N = 3, Session 7: N = 3). In total, digital recordings were available of 58 early treatment sessions (CBT: N = 31, MBCT: N = 27) and 47 late treatment sessions (CBT: N = 24, MBCT: N = 23).

Coding

The recordings were rated independently by three students pursuing a master's degree in Clinical Psychology who had finished their clinical training. Each student took part in a 2‐day training for raters by the first author, which mainly included discussing the coding manual and reviewing segments of recorded therapy sessions that illustrated the rating scales. Furthermore, the students coded four trial sessions and discussed the ratings until sufficient concordance between the ratings was reached. For each item, the manual provided a general definition of the described behaviour, examples of the behaviour to be coded, and examples of high and low ratings. During the coding period, the raters met biweekly with the first author to review discrepant ratings and to discuss questions to promote higher interrater agreement of the videos still to be coded.

Each rater coded two third of the available recorded therapy sessions, so that each therapy session was coded by two raters. A reduced Latin‐square design was used to divide the sessions across the raters, with a balanced distribution across the type of treatment, therapists, and session number. The raters were blinded with respect to the clinical experience of the therapists. The raters watched each therapy session twice, a first time to code treatment adherence as well as exploration of other topics than provided in the manual, and a second time to code therapist's interpersonal skills and patients' active engagement.

Measures

Treatment adherence

Treatment adherence was measured by rating the occurrence (1) or nonoccurrence (0) of techniques prescribed in the treatment manual, in line with recommendations of Waltz, Addis, Koerner, and Jacobson (1993). We did not use the MBCT Adherence Scale as developed by Segal, Teasdale, Williams, and Gemar (2002), because some of the items are not applicable for evaluating individual MBCT and because the scale is not session specific. Session specific checklists were developed for CBT and MBCT separately, based on the used CBT and MBCT treatment manuals. These checklists were shown to have high interrater reliability in previously performed unpublished pilot studies. The CBT and MBCT adherence scales consisted of seven items for Session 2 and 11 items for Session 6. The items covered performance of exercises, inquiry of exercises, reviewing homework, psycho‐education, and assigning homework. An example of a CBT item is “The therapist asked the patient to perform one or more activities that may bring pleasure or satisfaction and asked the client to formulate an action plan.” An example of an MBCT item is “The therapist enquired about patients' experiences and reactions to experiences during the performed exercise.” The items were scored on a scale including “no” (0) and “yes” (1). The overall agreement between the raters on adherence to all the prescribed techniques in CBT was 83.5%. The overall agreement between the raters on adherence to prescribed techniques in MBCT was 94.2%.

Therapists' interpersonal skills

We focus on three interpersonal skills of therapists: conveying involvement, warmth, and empathic understanding. Three items were selected from the Sheffield Psychotherapy Rating Scale (Shapiro & Startup, 1992) to measure these interpersonal skills: “How involved was the therapist?” to measure involvement, “Did the therapist convey warmth?” to measure warmth, and “Did the therapist convey an understanding of the client's experiences and feelings?” to measure empathy. These items were rated on a 5‐point Likert scale ranging from 1 (Not at all) till 5 (Extensively). Based on the Sheffield Psychotherapy Rating Scale (Shapiro & Startup, 1992), verbal and non‐verbal behaviours were specified that reflect these three techniques. For involvement, verbal behaviours included providing encouraging phrases as well as responsive answers. Non‐verbal behaviours included providing attention by nodding, eye contact, gesticulation, and a calm body posture. For warmth, verbal behaviours included providing compassionate responses as well as the absence of judgmental responses. Non‐verbal behaviours included mirroring as well as a tender voice and facial expression. For empathy, verbal behaviours included paraphrasing, validating, and emotional reflection.

The consistency between raters was examined by calculating the intraclass correlation coefficient (ICC). The average ICC for the three raters across sessions was 0.77, indicating acceptable interrater reliability. The mean of the three items was computed (based on the average of the provided ratings) as a measure of therapists' interpersonal skills. The internal consistency (Cronbach's alpha) of this scale was satisfactory for Session 2 (α = 0.75) and very good for Session 6 (α = 0.88).

Patients' active engagement

The extent to which patients were actively engaged in treatment was measured with the item “The patient worked actively with the therapist's comments” (Godfrey, Chalder, Ridsdale, Seed, & Ogden, 2007). This item referred to the following verbal behaviours by the patient: responding to the therapist, providing responsive answers, asking for clarification, reflecting on experiences, and not departing from the discussed topic. Raters coded the extent to which these verbal behaviours occurred during the treatment session on a 5‐point Likert scale ranging from 1 (not) till 5 (constantly). The interrater reliability was acceptable for the measure of patients' active engagement; the average ICC for the three raters across sessions was 0.79.

Observed explanations of nonadherence

An open question was used to ask the coders to report any peculiarities during treatment session that might be related to low adherence.

Depressive symptoms

The BDI‐II (Beck, Steer, & Brown, 1996) is a 21‐item questionnaire measuring the severity of depression. The total score ranges from 0 (no depressive symptoms) to 63 (severe depressive symptoms). Assessments of the BDI‐II at pretreatment and posttreatment were analysed. The internal consistency of the BDI‐II in the current sample was good (α ranging between 0.81 and 0.91).

Statistical analyses

To examine predictors of treatment outcome, multilevel analyses were performed, with patients (Level 1) nested within therapists (Level 2), to adjust for therapist effects. All analyses were performed for CBT and MBCT separately using STATA 14.2. Sensitivity analyses were performed in the total sample (CBT and MBCT combined). Posttreatment depressive symptoms were regressed on treatment adherence (mean of Sessions 2 and 6), therapists' interpersonal skills (mean of Sessions 2 and 6), or patients' engagement (mean of Session 2 and 6), while controlling for pretreatment depressive symptoms (BDI‐II). First, univariable multilevel analyses were performed including only one of the predictor variables and pretreatment BDI‐II scores as independent variables. Second, the three predictors variables of interest and pretreatment BDI‐II scores were entered simultaneously in a multivariable multilevel analysis.

Predictors of therapist adherence were examined in multilevel analyses, with sessions (Level 1) nested within patients (Level 2), and patients nested within therapists (Level 3). Therapist adherence was regressed on therapists' interpersonal skills or patients' engagement, while controlling for pretreatment depressive symptoms and time (Session 2 or Session 6). These predictors were examined both separately in univariable multilevel analyses and simultaneously in a multivariable multilevel analysis.

The mean scores of the two raters were used in all analyses. All models included a random intercept. Random slopes were only added if they improved model fit. As none of the patient characteristics (e.g., age, gender, or history of depression) were significantly related to both the outcome and one of the predictor variables, the analyses were not controlled for these characteristics. Missing BDI‐II scores at pretreatment (N = 1) and posttreatment (N = 4) BDI‐II scores were imputed by means of multiple imputations (N = 5; Tovote et al., 2014).

When adherence was lower than 75% (based on the average of both raters), the observed explanations for nonadherence were investigated. Only those explanations that were provided by both raters were included in the analyses. Thereafter, the observations were categorized based on the type of peculiarity occurring during the session, including “verbosity of the patient,” “patient did not perform homework,” “patient considers to quit treatment,” “patient had a life event in the past week,” “patient has to leave during the session,” “patient becomes distressed,” “patient refuses to do exercise,” “patient does not report symptoms,” “patient is already active,” “therapist does not structure the session,” and “therapist combines two sessions.”

RESULTS

Sample characteristics

The mean levels of the study variables are presented in Table 1. Therapist adherence was high on average. The mean level of therapist adherence during Session 6 was lower than in Session 2, both in CBT and in MBCT (see Table 1). Mean levels of interpersonal skills conveyed by therapists and patients' active engagement were relatively high. Variation around these mean levels was relatively low.

Table 1

Means and standard deviations of the variables under study

	MBCT M (SD)	CBT M (SD)
BDI‐II
Pretreatment	23.4 (7.0)	25.1 (7.5)
Posttreatment	16.6 (9.4)	17.7 (9.8)
Percentage adherence
Session 2	90.4 (15.2)	86.2 (14.9)
Session 6	81.2 (20.7)	70.8 (30.2)
Interpersonal skills
Session 2	3.95 (0.42)	3.66 (0.70)
Session 6	3.77 (0.72)	3.75 (0.85)
Patient engagement
Session 2	3.77 (0.70)	3.30 (0.63)
Session 6	3.72 (0.61)	3.34 (0.81)

Note. BDI‐II: Beck Depression Inventory‐II; CBT: cognitive behaviour therapy; MBCT: mindfulness‐based cognitive therapy; M: Mean; SD: standard deviation

Predicting posttreatment depressive symptom reduction

Depressive symptom reduction was not predicted by therapist adherence, neither in CBT nor in MBCT (see Table 2). Therapists' interpersonal skills also did not predict reduction in depressive symptoms in CBT or MBCT either. Finally, patients' active engagement was not predictive of depressive symptom reduction in CBT or MBCT. Similar results were found in both the univariable models and the multivariable model. Thus, none of the examined in‐session therapist or patient behaviours predicted outcomes of CBT and MBCT. Sensitivity analyses in the total sample (i.e., CBT and MBCT combined) also showed that posttreatment depressive symptom reduction was not significantly associated with therapist adherence, therapists' interpersonal skills, or patients' active engagement during treatment (see Table A1).

Table 2

Predictors of depressive symptoms (BDI‐II scores) at post treatment

Fixed effects	Mindfulness‐based cognitive therapy		Cognitive behavioural therapy
	Coef.	95% CI	Coef.	95% CI
Univariable model
BDI‐II pretreatment	0.95	[0.61, 1.28]**	0.81	[0.47, 1.17]**
Therapist adherence	−0.11	[−0.25, 0.04]	0.01	[−0.11, 0.13]
Univariable model
BDI‐II pretreatment	1.07	[0.75, 1.40]**	0.81	[0.45, 1.16]**
Interpersonal skills	−0.59	[−5.12, 3.94]	0.70	[−2.87, 4.27]
Univariable model
BDI‐II pretreatment	1.06	[0.75, 1.37]**	0.83	[0.47, 1.19]**
Patient engagement	−1.17	[−3.78, 1.44]	0.60	[−2.73, 3.92]
Multivariable model
BDI‐II pretreatment	0.96	[0.62, 1.29]**	0.82	[0.44, 1.19]**
Therapist adherence	−0.08	[−0.26, 0.11]	0.01	[−0.12, 0.14]
Interpersonal skills	−0.22	[−5.42, 4.97]	0.60	[−3.45, 4.64]
Patient engagement	−1.42	[−5.28, 2.44]	0.27	[−3.63, 4.17]
N	29		32

Note. Estimates are based on multilevel models, with patients nested within therapists. Coef.: unstandardized B coefficient; CI: confidence interval; BDI‐II: Beck Depression Inventory‐II; N: number of participants.

p < 0.05.

p < 0.01.

Table A1

Predictors of depressive symptoms (BDI‐II scores) at post treatment in the total sample

Fixed effects	CBT and MBCT
	Coef.	95% CI
Univariable model
BDI‐II pretreatment	0.89	[0.64, 1.13]**
Therapist adherence	−0.02	[−0.12, 0.07]
Univariable model
BDI‐II pretreatment	0.91	[0.67, 1.15]**
Interpersonal skills	0.45	[−2.28, 3.18]
Univariable model
BDI‐II pretreatment	0.92	[0.67, 1.16]**
Patient engagement	−0.05	[−2.11, 2.01]
Multivariable model
BDI‐II pretreatment	0.82	[0.44, 1.19]**
Therapist adherence	0.02	[−0.12, 0.14]
Interpersonal skills	0.60	[−3.45, 4.64]
Patient engagement	0.27	[−3.63, 4.17]
N	61

Note. Estimates are based on multilevel models, with patients nested within therapists. Coef.: unstandardized B coefficient; CI: confidence interval; BDI‐II: Beck Depression Inventory‐II; N: number of participants.

p < 0.05.

p < 0.01.

Therapist interpersonal skills and patient engagement in association with adherence

In the univariable model, patients' engagement during the treatment session was positively associated with therapist adherence, both in CBT and in MBCT (see Table 3). This positive association between patients' engagement and therapist adherence to CBT and MBCT was also found in the multivariable model. The multivariable model shows that if ratings of patients' engagement were 1 point higher on a scale from 1 till 5, the percentage of therapist adherence was 11% higher in MBCT and 14% higher in CBT. When the analyses were repeated in the total sample, similar results were found: If ratings of patients' engagement were 1 point higher on a scale from 1 till 5, the percentage of therapist adherence was 12% in the total sample (see Table A2).

Table 3

Predictors of percentage of therapist adherence

Fixed effects	Mindfulness‐based cognitive therapy		Cognitive behavioural therapy
	Coef.	95% CI	Coef.	95% CI
Univariable model
BDI‐II pretreatment	−0.55	[−1.29, 0.20]	−0.05	[−1.00, 0.90]
Time	−10.80	[−19.60, −1.99]*	−15.63	[−25.13, −6.14]*
Interpersonal skills	−2.60	[−11.26, 6.07]	−4.42	[−12.70, 3.85]
Univariable model
BDI‐II pretreatment	−0.52	[−1.13, 0.10]	0.14	[−0.74, 1.01]
Time	−8.95	[−17.35, −0.55]*	−15.91	[−25.41, −6.40]**
Patient engagement	8.53	[3.69, 13.36]**	11.02	[4.09, 17.95]**
Multivariable model
BDI‐II pretreatment	−0.34	[−0.94, 0.27]	0.37	[−0.53, 1.27]
Time	−10.17	[−18.40, −1.94]*	−15.80	[−23.93, −7.67]**
Interpersonal skills	−8.70	[−16.57, −0.84]*	−8.76	[−16.49, −1.04]*
Patient engagement	10.50	[5.57, 15.42]**	13.89	[6.67, 21.12]**
N/Observations	29/55		32/50

Note. Estimates are based on multilevel models, with observations (at Sessions 2 and 6) nested within patients, and patients nested within therapists. Coef.: unstandardized B coefficient; CI: confidence interval; BDI‐II: Beck Depression Inventory‐II; N = number of participants.

p < 0.05.

p < 0.01.

Table A2

Predictors of percentage of therapist adherence in the total sample

Fixed effects	CBT and MBCT
	Coef.	95% CI
Univariable model
BDI‐II pretreatment	−0.39	[−1.02, 0.23]
Time	−13.34	[−20.03, −6.65]*
Interpersonal skills	−1.96	[−8.06, 4.15]
Univariable model	−0.21
BDI‐II pretreatment		[−0.75, 0.34]
Time	−12.60	[−19.11, −6.09]**
Patient engagement	9.51	[5.43, 13.59]**
Multivariable model
BDI‐II pretreatment	−0.07	[−0.61, 0.48]
Time	−12.89	[−19.13, −6.64]**
Interpersonal skills	−7.46	[−13.21, −1.70]*
Patient engagement	11.60	[7.30, 15.89]**
N/Observations	61/105

Note. Estimates are based on multilevel models, with observations (at Sessions 2 and 6) nested within patients, and patients nested within therapists. Coef.: unstandardized B coefficient; CI: confidence interval; BDI‐II: Beck Depression Inventory‐II; N: number of participants. CBT: cognitive behaviour therapy; MBCT: mindfulness‐based cognitive therapy.

p < 0.05.

p < 0.01.

Whereas the univariable model showed that the extent to which therapists conveyed interpersonal skills was not associated with therapist adherence, the multivariable model showed that therapists' interpersonal skills were significantly associated with therapist adherence, both in CBT and in MBCT. Thus, one unit increase in interpersonal skills (on a scale from 1 to 5) was associated with about 9% decrease in therapist adherence in CBT and MBCT, but only when controlling for patients' engagement. The sensitivity analyses showed similar results: An increase of 1 point in interpersonal skills was associated with a 7% decrease in therapist adherence in the total sample, but only when controlling for patients' engagement (see Table A2).

Observers' explanations of nonadherence

In 16 out of 56 CBT sessions (Session 2: N = 5, Session 6: N = 11) and 11 out of 52 MBCT sessions (Session 2: N = 5, Session 6: N = 6), adherence was lower than 75%. A broad range of explanations for therapist nonadherence were reported by the observers (see Table 4). For several sessions, more than one reason for nonadherence was provided. Most observations concerned patients' in‐session behaviour; therapists' in‐session behaviour was reported less often as a reason for nonadherence. In both CBT and MBCT, a prominent reason for nonadherence was verbosity of the patient, sometimes in combination with failure by the therapist to structure the session. Several of the other observed peculiarities seem to have a one‐on‐one association with therapist adherence and involve therapists' flexibility of responding to the needs of a particular patient, such as that the patient considers to quit treatment, the patient has to leave during the session, or the patient experienced life events during the past week.

Table 4

Qualitative explanations of nonadherence

	CBT		MBCT
	Session 2	Session 6	Session 2	Session 6
Patient
Is verbose	1	5	1	3
Did not perform homework		3	2
Considers to quit treatment		1	2
Life events in past week		1	1
Has to leave during session		1		1
Becomes distressed			1
Refuses to do exercise	2	2
Does not report symptoms		2
Is already active	1
No reason	1			1
Therapist
Does not structure session		3		2
Combines two sessions				1

Note. The Arabic numbers represent the number of sessions in which this reason for nonadherence was reported by both raters. CBT: cognitive behaviour therapy; MBCT: mindfulness‐based cognitive therapy.

Some of the reasons for nonadherence were specific to CBT and would not interfere with the ability to follow the treatment manual in MBCT. These reasons specific to CBT included that the patient was already active and it was therefore not necessary to plan activities, that the patient did not report symptoms (and therefore also no negative automatic cognitions, which could be challenged), and that the patient did not cooperate in performing an exercise such as planning activities or reporting negative thoughts.

DISCUSSION

The aim of this study was to gain a deeper understanding of the sources of variability in adherence to treatment manuals and treatment outcome in CBT and MBCT for depressive symptoms in individuals with diabetes. We investigated both patients' active engagement in treatment and therapists' interpersonal skills (i.e., conveying empathy, warmth, and involvement). Posttreatment depressive symptom reduction was not predicted by therapist adherence, patients' active engagement, or therapists' interpersonal skills, neither in CBT nor in MBCT. Therapist adherence was predicted by patients' active engagement during treatment. Therapists' interpersonal skills only predicted therapist adherence when controlling for patients' active engagement. Furthermore, the raters reported a broad range of reasons for low therapist adherence.

The finding that lower adherence was not predictive of poorer outcome in CBT and MBCT is consistent with a meta‐analysis finding no significant mean weighted association between therapist adherence and outcome of psychological treatments (Webb et al., 2010). It also corresponds with a recent trial showing that competence in delivering MBCT (including conveying course themes, guiding practice, embodiment of mindfulness, and relational skills) was not associated with posttreatment changes in depressive symptoms or with relapse and recurrence rates (Huijbers et al., 2017).

In contrast with earlier studies (Gaston et al., 2010; Gomes‐Schwartz, 1978; Zuroff et al., 2000), we did not find that patients' engagement was associated with posttreatment outcome. A possible explanation for this finding is that patients with low levels of engagements may not have been engaged in treatment for reasons that are differently related to treatment outcome. Some patients may not have been actively engaged because of early symptom gains, as also indicated by our qualitative analyses, which is generally associated with better outcomes of psychological treatments (Aderka, Nickerson, Bøe, & Hofmann, 2012). Others may not have been motivated to engage in treatment because they had low expectations of treatment, which is associated with poorer outcomes of CBT and MBCT (Snippe et al., 2015).

Patients did not benefit more from CBT and MBCT when therapists conveyed more empathy, warmth, and involvement. Although numerous studies have shown that patient evaluations of these interpersonal skills do predict treatment outcome (Barnicot, Wampold, & Priebe, 2014; Elliott et al., 2011; Keijsers et al., 2000; Zuroff & Blatt, 2006), the current study shows that rater's observations do not. This should be examined further as, to our knowledge, only a series of criticized studies performed between the 1940s and 1970s examined rater's observations of empathy and warmth, showing no association with treatment outcomes (Lambert, DeJulio, & Stein, 1978). Future studies may also investigate the interaction between competent delivery of treatment techniques and therapists' interpersonal skills, as higher quality of a therapist may in fact be characterized by both therapists' delivery of specific techniques and their interpersonal skills (DeRubeis, Gelfand, German, Fournier, & Forand, 2014). Furthermore, the timing and responsiveness of conveying empathy, warmth, and involvement could be explored in future studies.

An explanation for the fact that neither therapist adherence nor therapists' interpersonal skills or patient's engagement predicted treatment outcome is that patients might have received sufficient “active ingredients” for the treatments to be effective (Huijbers et al., 2017). Both CBT and MBCT were provided according to standardized treatment manuals and included weekly homework assignments. As therapist adherence to these manuals was high on average and almost all patients performed at least one homework assignment a week (Snippe et al., 2015), participants might have received a sufficient dose of treatment to reduce depressive symptoms, as we found in the original trial (Tovote et al., 2014). The standardization of the treatment manuals and training of therapists might also explain the limited variance in therapist adherence, therapists' interpersonal skills, and patients' engagement, which might as well have played a role in not finding significant associations.

A second aim of the study was to understand why therapists deviate from a treatment manual. Our study indicates that there may be different sources of variability in therapist adherence. Therapist adherence can be partly explained by the extent to which patients work actively with the therapists' comments. Patients' active engagement during Sessions 2 and 6 were positively associated therapist adherence during the same treatment sessions. This aligns with a recent study showing that in‐session resistant behaviour of patients interfered with therapist adherence to CBT for panic disorder (Zickgraf et al., 2015).

Furthermore, we found that therapist adherence was lower when therapists employ more interpersonal skills, including conveying empathy, warmth, and involvement, when controlling for patients' engagement. In the univariable models, the negative association between therapists' interpersonal skills and therapist adherence were much smaller and not significant, neither in CBT nor in MBCT. As there may be various reasons for this inconsistent finding, replication in future research is warranted before conclusions can be drawn.

Different sources of therapist variability in adherence to the treatment manual were reported by the observers, of which some might not be indicative of a treatment not going well. Most reasons reported by raters were related to in‐session behaviour of patients. Although our number of sessions with low adherence was too small to empirically test this, it can be argued that some of the reported reasons may not result in poorer treatment outcome and rather reflect therapist flexibility or competence of the therapist. For example, treatment efficacy may not be negatively affected when a therapist does not perform activity scheduling because a patient already engages in pleasurable activities. The idea that therapist flexibility or therapist responsiveness (i.e., being responsive to the emerging context of a therapy session; Stiles, Honos‐Webb, & Surko, 1998) may lead to more optimal treatment effects was supported by a study showing that within‐patient variability in adherence was associated with better outcomes of psychotherapy (Owen & Hilsenroth, 2014). Furthermore, some of the reasons for nonadherence might only have occurred during one session (e.g., the patient had to leave during the session or the therapist combined two sessions). These reasons may not be representative of overall treatment adherence and may thus not be associated with poorer treatment outcomes.

For clinical training of therapists, it seems useful to discuss when treatment nonadherence is responsive to the patients' needs and when it is less appropriate. For example, spending more time on voicing meanings in a patient's experience instead of performing an exercise does not seem efficacious because spending more time on conveying empathy, warmth, or involvement did not predict positive treatment outcomes. In order to train ability to adhere to a treatment manual, the current study indicates that it may be valuable to train therapists how to motivate patients to work actively with the therapists' comments and how to handle patients who are verbose.

Strengths of the study include the assessment of both treatment adherence and in‐session behaviour of patients and therapists, ratings by independent trained observers, and controlling for therapist effects in the multilevel models. A limitation of the study is the restricted range and reduced variability in therapist adherence, therapists' interpersonal skills, and patients' engagement. In addition, patients' engagement was measured with only one item, which might partially explain the limited variance in this measure. Another limitation of the study is that only the second and sixth treatment session were coded. Assessment of more therapy sessions would have provided more accurate data on therapist adherence across the whole treatment. Because of the high costs in terms of time and money associated with assessing adherence (Perepletchikova, Hilt, Chereji, & Kazdin, 2009), it was not feasible to rate the recordings of all therapy sessions. Furthermore, we cannot exclude the possibility that the association between therapist adherence, therapists' interpersonal skills, and patients' engagement is attributable to the same raters rating these variables. Finally, our study did not include a measure of therapist competence, which might have been insightful given that some of the reasons for nonadherence might in fact reflect higher levels of therapist competence.

To conclude, this study reveals that therapists deviate more from treatment manuals when treating patients who are less actively engaged during treatment sessions. Furthermore, a broad range of other reasons for nonadherence were observed, such as verbosity of the patient, absence of symptoms, and life events during the past week. As some of these reasons may not be associated with poorer treatment outcomes, the heterogeneity in reasons for nonadherence might explain the mixed findings on the adherence–outcome association. Future studies should reveal under which circumstances nonadherence is acceptable or even responsive and when it may negatively affect treatment efficacy.

CONFLICT OF INTEREST

The authors declare that they have no conflict of interest.