Cognitive mediators of treatment for social anxiety disorder: comparing acceptance and commitment therapy and cognitive-behavioral therapy.

OBJECTIVE: To assess the relationship between session-by-session mediators and treatment outcomes in traditional cognitive-behavioral therapy (CBT) and acceptance and commitment therapy (ACT) for social anxiety disorder.
METHOD: Session-by-session changes in negative cognitions (a theorized mediator of CBT) and experiential avoidance (a theorized mediator of ACT) were assessed in 50 adult outpatients randomized to CBT (n=25) or ACT (n=25) for DSM-IV social anxiety disorder.
RESULTS: Multilevel modeling analyses revealed significant nonlinear decreases in the proposed mediators in both treatments, with ACT showing steeper decline than CBT at the beginning of treatment and CBT showing steeper decline than ACT at the end of treatment. Curvature (or the nonlinear effect) of experiential avoidance during treatment significantly mediated posttreatment social anxiety symptoms and anhedonic depression in ACT, but not in CBT, with steeper decline of the Acceptance and Action Questionnaire at the beginning of treatment predicting fewer symptoms in ACT only. Curvature of negative cognitions during both treatments predicted outcome, with steeper decline of negative cognitions at the beginning of treatment predicting lower posttreatment social anxiety and depressive symptoms.
CONCLUSIONS: Rate of change in negative cognitions at the beginning of treatment is an important predictor of change across both ACT and CBT, whereas rate of change in experiential avoidance at the beginning of treatment is a mechanism specific to ACT.

RCT Entities:   Population Interventions Outcomes

Social anxiety disorder is among the most common psychological disorders, affecting approximately 13% of individuals at some point in their lives (Kessler, Petukhova, Sampson, Zaslavsky, & Wittchen, 2012). Cognitive-behavioral therapy (CBT) is an effective treatment for social anxiety disorder (Heimberg, 2002; Rodebaugh, Holaway, & Heimberg, 2004). However, a significant number of individuals do not benefit from CBT (Arch & Craske, 2009; Clark et al., 2006; Davidson et al., 2004). Recently, new behavioral treatments such as acceptance and commitment therapy (ACT; Hayes, Strosahl, & Wilson, 1999) have emerged that draw from Eastern mindfulness meditation practice, and preliminary evidence supports their effectiveness for anxiety disorders (Eifert & Forsyth, 2005). Understanding the mechanisms that drive treatment response is essential for optimizing their delivery and improving outcomes (Kazdin, 2007). The goal of the current study was to examine possible mediators of treatment outcome in two treatments for social anxiety disorder—CBT and ACT—to better understand why these treatments work.

Testing mediators in randomized controlled trials can tell us why and how treatments are effective (Kraemer, Wilson, Fairburn, & Agras, 2002), and comparison of mechanisms across two active treatments may ultimately help us tailor treatment approaches based on an individual’s presentation. For a rigorous test of treatment mediation, the mediators must be tested during treatment and preferably at multiple time points. Doing so ensures that the mediator temporally precedes the outcome (Kraemer et al., 2002), and assessing mediators at multiple time points throughout treatment allows assessment of change in mediators over time. Multilevel modeling is optimal for nested designs where repeated measures are collected within individuals (Kenny, Kashy, & Bolger, 1998). In addition, multilevel modeling handles missing data effectively. As reviewed below, few studies have examined treatment mediators using this rigorous approach.

Evidence for Treatment Mediators in CBT and ACT

The cognitive model for social anxiety disorder posits that reductions in negative cognitions in relation to social situations explain subsequent symptom reduction following CBT (Craske, 2010; Craske et al., 2008). In support of this purported mediator, Hofmann (2004) found that reduction in social cost ratings (patient ratings of “How bad would it be?” if a feared social outcome occurred) from pre- to post-treatment predicted symptom reduction. However, since social cost ratings were not measured during treatment, rigorous testing of the mediator as a temporal precedent to the outcome was not conducted. In another study, session-by-session ratings of the probability of a negative social outcome predicted subsequent fear reduction (Smits, Rosenfield, McDonald, & Telch, 2006). The mediator was measured during treatment and prior to the outcome, and the authors used multilevel modeling to model change in the mediator over time. However, to fully understand whether a mediator is specific to CBT (as opposed to common treatment processes), it is necessary to compare CBT mediators with those of another active treatment (see Arch & Craske, 2008; Kraemer et al., 2002).

ACT (Hayes et al., 1999) has been shown to be effective for anxiety disorders (Arch, Eifert, et al., 2012), and in one randomized controlled trial, ACT was effective for social anxiety disorder in particular (Dalrymple & Herbert, 2007). ACT aims to promote mindfulness, acceptance, and cognitive defusion (learning to detach from thoughts and observe them more dispassionately) with the ultimate goal of increasing psychological flexibility and promoting behavior change that aligns with one’s life values (Hayes et al., 1999). Decreased experiential avoidance, or becoming more willing to experience uncomfortable physical sensations and emotions, has been proposed as a possible mechanism of change (Hayes et al., 2004).

In a study of ACT for social anxiety disorder, Dalrymple and Herbert (2007) found that greater increases in acceptance and cognitive defusion by midtreatment predicted better outcomes posttreatment, whereas greater perceived control over anxiety (a more CBT consistent mediator measure) did not. However, the meaningfulness of these results was limited by the fact that the mediator was assessed only once midtreatment, and difference scores were calculated to assess the effect of the mediator on treatment outcomes. Repeated measurement of the mediator at multiple time points throughout treatment and subsequent analysis using growth curve modeling would allow for a more fine-grained assessment of how the mediator changes throughout treatment.

To our knowledge, only one study has compared treatment mediators in CBT and ACT. Arch, Wolitzky-Taylor, Eifert, and Craske (2012) examined treatment mediators in CBT and ACT for individuals with a variety of anxiety disorders. Participants with panic disorder, social anxiety disorder, generalized anxiety disorder, and specific phobia were randomized to 12 sessions of either CBT or ACT. Participants completed measures of purported treatment mediators (negative beliefs in CBT and cognitive defusion in ACT) repeatedly throughout treatment. Using multilevel modeling, both purported mediators were found to change significantly in both treatments, with experiential avoidance and negative cognitions decreasing more in ACT than in CBT. Also, change in both purported mediators significantly predicted symptom reduction and increased quality of life in both treatments, suggesting similarity in the change mechanisms in ACT and CBT.

CURRENT STUDY

The current study included analysis of session-by-session data from a treatment study in which ACT and CBT were compared for the treatment of social anxiety disorder. Patients in both treatment groups demonstrated significant symptom reduction following completion of treatment, and the two groups did not significantly differ posttreatment, or at 6- or 12-month follow-ups (Craske et al., 2014).

We assessed the following research questions: First, do CBT and ACT affect negative cognitions and experiential avoidance, which have been posited as mediators of treatment outcome in CBT and ACT, respectively? In accordance with the theoretical models underlying each treatment approach, we hypothesized that negative cognitions would decrease more in CBT than in ACT, and that experiential avoidance would decrease more in ACT than in CBT. Even though previous research showed that experiential avoidance and negative cognitions decreased to a greater extent in ACT than in CBT, we linked our hypotheses to theory in the absence of replicated empirical data to the contrary.

Second, we addressed whether negative cognitions and experiential avoidance mediated treatment outcomes. The Baron and Kenny (1986) approach to mediation requires that the independent variable X is related to the outcome Y. Although no differences were found between treatment groups (X) on treatment outcome (Y; Craske et al., 2014), more recent approaches to testing mediation no longer require a significant relationship between X and Y (MacKinnon, Lockwood, Hoffman, West, & Sheets, 2002). Fritz and MacKinnon (2007) suggest that the original Baron and Kenny (1986) approach is underpowered and increases the likelihood of Type II error. In addition, Arch and Craske (2008) argue that even in the absence of significant differences in treatment outcome, the examination of mediators can address important questions about similarities and differences in how these two treatments produce change. Thus, we proceeded with testing mediation in this trial despite no group differences in treatment outcome.

In line with the respective theoretical models, change in negative cognitions should mediate outcomes in CBT, whereas change in experiential avoidance should mediate outcomes in ACT. Therefore, we hypothesized that negative cognitions would predict greater improvement in CBT than in ACT, whereas experiential avoidance would predict greater improvement in ACT than in CBT. Again, this hypothesis was not supported in previous research, in which reductions in negative cognitions and experiential avoidance similarly predicted treatment outcome across ACT and CBT (Arch, Wolitzky-Taylor, et al., 2012), but in the absence of replicated empirical data, we deemed it more logical to hypothesize based on theorized mechanisms.

Method

PARTICIPANTS

Seventy-one participants who met DSM-IV criteria for a principal or co-principal diagnosis of social anxiety disorder, generalized type, were randomized to ACT (n = 34) or CBT (n = 37). Analyses included only participants who completed treatment (n = 27 ACT, n = 25 CBT) because we were interested in examining treatment mediators for participants completing a full course of treatment. Two participants were excluded from analyses due to large amounts of missing data (>50%) on the session-by-session treatment measures. The final sample analyzed included 50 participants (n = 25 ACT, n = 25 CBT). See Craske et al. (2014) for participant flow of the full sample. Participants were recruited from the Los Angeles area in response to local flyers, Internet and local newspaper advertisements, and referrals. The study took place at the Anxiety Disorders Research Center at the University of California, Los Angeles (UCLA), Department of Psychology starting September 2008 and ending March 2013 upon completing collection of the desired sample. Forty-three percent of the sample was female. In terms of ethnicity, 13% identified as Latino/Hispanic American, 15% as Asian American, 59% as Caucasian, and 13% did not respond or indicated “other.” The mean age of participants was 28.4 years (6.5 SD, range 18–42) with 15.5 years of education (1.9 SD, range 12–19 years), and 7% were married, 83% were single, 4% were cohabitating, and 6% were separated or divorced. Twenty percent of participants had comorbid anxiety disorders and 20% met criteria for major depressive disorder or dysthymia.

Participants were either medication free or stabilized on psychotropic medications for a minimum length of time (1 month for benzodiazepines and beta blockers, 3 months for SSRIs/SNRIs, heterocyclics, and MAO inhibitors). Also, participants were psychotherapy free or stabilized on alternative psychotherapies (other than cognitive or behavioral therapies) that were not focused on their anxiety disorder for at least 6 months prior to study entry. Participants were encouraged not to change their medication or alternative psychotherapy during the course of the study. Exclusion criteria included active suicidal ideation, severe depression (clinical severity rating > 6, see below), or a history of bipolar disorder, psychosis, mental retardation, or organic brain damage. Participants with substance abuse or dependence within the last 6 months, or with respiratory, cardiovascular, pulmonary, neurological, muscular-skeletal diseases, or pregnancy were excluded. Patients with asthma, high blood pressure, or thyroid diseases were included only if they were currently receiving treatment and were stabilized for these conditions. Because our study included neuroimaging (results to be reported elsewhere) additional exclusion criteria were left-handedness, metal implants, claustrophobia, and over 45 years of age.

Participants received 12 weekly sessions of reduced-cost, sliding-scale treatment and were financially compensated for post and follow-up assessments. The study was fully approved by the UCLA Human Subjects Protection Committee; full informed consent was obtained from all participants, including for video and audio recordings.

DESIGN

Participants were assessed at four time points: pretreatment (pre), posttreatment (post), and 6 months (6MFU) and 12 months (12MFU) after pre (although only pre and post measures are used in the current study). Assessments included a diagnostic interview and self-report questionnaires. Participants were randomly assigned to ACT, CBT, or a wait-list condition using a random number generator. Because no treatment measures were collected from wait-list participants, they were not included in the mediation analysis. Participants were stratified by age and gender in CBT and ACT to ensure equal distribution across groups; study personnel did not inform patients of their treatment condition using the terms CBT or ACT, but rather informed patients they were receiving behavioral treatment with cognitive strategies (CBT) or behavioral treatment with acceptance and mindfulness strategies (ACT).

DIAGNOSTIC ASSESSMENT

Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV)

The ADIS-IV (Brown, Di Nardo, & Barlow, 1994) is a semistructured interview that assesses for anxiety disorders, mood disorders, psychotic disorders, and alcohol and substance abuse and dependence. With the exception of dysthymia, diagnosis of psychological disorders using the ADIS-IV evidenced good to excellent interrater reliability (κs range from .67 to .86; Brown, Di Nardo, Lehman, & Campbell, 2001). All interviews were audio recorded and a subset was randomly selected (n = 22) for blind rating by a second interviewer.[1] Interrater reliability on the principal diagnosis (n = 22) was 100%.

After completing the ADIS-IV, interviewers rated the severity of all diagnoses in the past month using a 0 to 8 clinician severity rating (CSR) scale. Scores of 1 and 2 indicate that at least some symptoms have been present in the past month but severity, impairment, and distress are subclinical. A score of 3 indicates that symptoms may be clinically significant. A score of 4 or above indicates moderately severe symptoms associated with clinically significant distress or impairment. Participants were eligible for the study if they received a CSR rating of 4 or higher. The CSR rating has demonstrated good to excellent interrater reliability for anxiety disorders (Brown et al., 2001; Craske et al., 2007).

TREATMENTS

Participants in CBT or ACT received 12 weekly, 1-hour, individual therapy sessions based on detailed treatment manuals.[2] ACT and CBT were matched on number of sessions devoted to exposure but differed in framing of the intent of exposure. A subsample of therapy sessions were reviewed for independent assessment of therapist adherence and competency, and therapists adhered strongly to their assigned treatment approach (for further details, see Craske et al., 2014). Following the 12 sessions, therapists conducted follow-up booster phone calls (20, 35 mins) once per month for 6 months to reinforce progress consistent with the assigned therapy condition.

Cognitive-Behavioral Therapy

CBT for social anxiety disorder was derived largely from standard CBT protocols (e.g., Hope, Heimberg, Juster & Turk, 2000), but differed in its inclusion of interoceptive exposure. We have shown this particular CBT to be effective for individuals with social anxiety disorder (Craske et al., 2011). Session 1 focused on assessment, self-monitoring, and psychoeducation. Sessions 2–4 emphasized cognitive restructuring errors of overestimation and catastrophizing regarding negative evaluation, combined with hypothesis testing (i.e., conducting behavioral experiments with the purpose of disconfirming negative thoughts), self-monitoring, and breathing retraining. Exposure to feared social cues (including in vivo, imaginal, and interoceptive exposure combined with in vivo exposure) was introduced in Session 5, and emphasized strongly in Sessions 6–11. Session 12 focused on relapse prevention.

Acceptance and Commitment Therapy

ACT for anxiety disorders largely followed a manual authored by Eifert and Forsyth (2005).[3] Session 1 focused on psychoeducation, experiential exercises, and discussion of acceptance and valued action. Sessions 2 and 3 explored creative hopelessness or explored whether efforts to manage and control anxiety had “worked” and how such efforts had led to the reduction or elimination of valued life activities, and encouraged acceptance. Sessions 4 and 5 emphasized mindfulness, acceptance, and cognitive defusion, or the process of experiencing anxiety-related language (e.g., thoughts, self-talk) as part of the broader, ongoing stream of present experience rather than getting stuck in responding to its literal meaning. Sessions 6–11 continued to hone acceptance, mindfulness, and defusion, and added values exploration and clarification with the goal of increasing willingness to pursue valued life activities. Behavioral exposures (e.g., interoceptive, in vivo, imaginal) were employed to provide opportunities to practice mindfully observing and accepting anxiety, and to practice engaging in valued activities while experiencing anxiety. Session 12 reviewed what worked and how to continue moving forward.

THERAPISTS

Study therapists were advanced clinical psychology doctoral students and recent Ph.D.s at UCLA, all of whom had at least 2 years of supervised training in delivering psychological treatments and at least 1 year training in CBT or ACT. In addition, therapists completed intensive in-person 2-day workshops for CBT or ACT, prior to treating participants. Therapists were assigned to ACT, CBT, or both (i.e., treated in both CBT and ACT, though never at the same time), depending on need.[4] There were 28 therapists; 13 therapists worked exclusively in CBT, 12 worked exclusively in ACT, and 3 treated both ACT and CBT participants. There were no differences among therapists who provided CBT, ACT, or both in terms of gender, age, or years since entering graduate school (ps > .39). Generally, therapists treated one to two patients at a time and two to five therapists worked within each treatment condition at a time. The mean number of patients treated by CBT-only therapists was M = 2.38, SD = 1.56 (range 1–6, total = 31 participants), by ACT-only therapists was M = 2.67, SD = 1.30 (range 1–5, total = 32 participants), and by therapists who treated both ACT and CBT was M = 5.67, SD = 2.52 (range 3–8, total = 17 participants).

Weekly, 90-minute group supervision meetings were held separately for CBT and ACT. For CBT, the supervision was led by professors and postdoctoral fellows at UCLA, and was held in person. For ACT, supervision was led by advanced therapists from the University of Nevada, Reno, where ACT was originally developed, via Skype.[5]

SESSION-BY-SESSION MEASURES

To assess change over the course of treatment, two measures were administered at even numbered treatment sessions (2, 4, 6, 8, and 10). The measures were the 16-item Acceptance and Action Questionnaire (AAQ; Hayes, Luoma, Bond, Masuda, & Lillis, 2006) and a modified version of the Self-Statements During Public Speaking Questionnaire (SSPS; Hofmann & DiBartolo, 2000). Clients completed the measures before each designated treatment session and returned them in a sealed envelope to the clinic staff (not the therapist). The AAQ was conceptualized as the ACT-specific treatment mediator measure and the SSPS was the CBT-specific measure. Correlations between the AAQ and SSPS ranged from .47 to .71 (M = . 62) across the five treatment sessions at which these measures were taken.

Acceptance and Action Questionnaire–16-Item Version

The AAQ (Hayes et al., 2004) was developed specifically to measure the proposed mechanisms of change in ACT, namely experiential avoidance. Only psychometric properties for the 9-item version have been published, but Hayes and colleagues tested 7-, 9-, and 16-item versions, and state that the three scales perform “nearly identically” (Hayes et al., 2004). The authors also state that the 16-item version may be more useful to detect small changes (e.g., over the course of therapy) than the 9-item version because it contains more items (Hayes et al., 2004). The scale measures experiential avoidance and control, negative evaluation of internal experience, psychological acceptance, and the tendency to act despite emotional distress as opposed to avoiding due to distress. Items are rated on a 7-point Likert scale (1 = never true, 7 = always true). Higher scores indicate more experiential avoidance. Sample items include “I try to suppress thoughts and feelings that I do not like by just not thinking about them” and “It’s okay to feel depressed or anxious” (reverse scored). Test–retest reliability for the 9-item version was .64 over a 4-month period and internal consistency (Cronbach’s alpha) is .70 (Hayes et al., 2004). For the current sample, alpha was .86 across all treatment sessions.

Self-Statements During Public Speaking Questionnaire

The SSPS (Hofmann & DiBartolo, 2000) is a 10-item scale that measures negative and positive self-statements in the context of public speaking. The scale was developed to assess the cognitive component of fear of public speaking. For purposes of the current study, the scale was modified so that participants rated their most anxiety-provoking social situation rather than public speaking specifically. The instructions read as follows: “Imagine your most anxiety provoking social situation and consider the thoughts that might occur to you if you were to enter this situation right now. Read each statement and rate the degree of agreement.” The scale consists of two subscales: one that assesses positive cognitions (SSPS-P), and one that assesses negative cognitions (SSPS-N). Although the SSPS-N subscale has shown the greatest sensitivity to change (Hofmann & DiBartolo, 2000), to maximize the number of scale items, all 10 items were used and positive scale items were reverse coded. Cronbach’s alpha for the positive subscale was .80 and for the negative subscale was .86, and test–retest reliability was acceptable for both subscales (Hofmann & DiBartolo, 2000). In the current sample, alpha was .89 across all treatment sessions for the full scale with positive items reverse coded. Alpha for the positive subscale was .86 and for the negative subscale was .89.

OUTCOME MEASURES

We assessed whether changes in SSPS and AAQ during treatment mediated outcomes at posttreatment. Given the emphasis on symptom reduction and symptom mastery/control in CBT, and the emphasis on acceptance, valued action, and living a meaningful life in ACT, we assessed social anxiety symptom-specific outcome measures as well as nonspecific or broader outcomes across both treatments. Questionnaires to assess social anxiety symptoms comprised the symptom-specific measures, and measures of quality of life and depression comprised the nonspecific, broader outcomes. Each outcome questionnaire is described briefly.

Social Anxiety Symptom Scales

We selected three widely used and well-validated self-report measures of social anxiety symptoms. The Liebowitz Social Anxiety Scale–Self-Report (LSAS-SR; Fresco et al., 2001) is a 24-item measure that assesses fear and avoidance of social interactional and performance situations. Each item is rated on a scale from 0 (no fear/never avoid) to 3 (severe fear/ usually avoid). Scores were calculated as the sum of fear and avoidance ratings across social and performance situations. In the current sample, Cronbach’s αs = .97 (pre) and .94 (post). The Social Interaction Anxiety Scale (SIAS; Mattick & Clarke, 1998) is a 20-item measure of cognitive, affective, or behavioral reactions to social interaction in dyads or groups. Participants respond on a Likert scale from 0 (not at all characteristic or true of me) to 4 (extremely characteristic or true of me). In the current sample, Cronbach’s αs = .96 (pre) and .95 (post). The Social Phobia Scale (SPS; Mattick & Clarke, 1998) is a 20-item measure describing situations or themes related to being observed by others. Participants rate the extent to which each item is characteristic of them on a 0 to 4 scale. In the current sample, αs = .93 (pre) and .90 (post).

Composite Symptom Scale

A composite was created from the LSAS, SIAS, and SPS to enhance the assessment of each construct and reduce the number of analyses. Z scores were calculated for each measure at pre and standardization was based on pre means and standard deviations for the follow-up assessment. The composite score represented averages of the three measures and was used as our assessment of social anxiety symptoms.

Quality of life

The Quality of Life Inventory (QOLI; Frisch, 1994) assesses values and life satisfaction across 16 broad life domains and has good test–retest reliability and internal validity (Frisch et al., 2005). In the current study, we calculated a weighted score that accounts for both satisfaction and importance ratings for each life domain. In the current sample, Cronbach’s αs = .85 (pre) and .84 (post).

Depression

The Anhedonic Depression Scale of the Mood and Anxiety Symptom Questionnaire (Watson & Clark, 1991) is a 22-item measure of depressive symptoms. Participants indicate the extent to which they have experienced each of the 22 symptoms on a 1 to 5 Likert scale where 1 = not at all and 5 = extremely. The scale demonstrates good convergent and discriminant validity in community samples (Watson, Clark, et al., 1995; Watson, Weber, et al., 1995). Current αs were .93 (pre) and .94 (post).

STATISTICAL ANALYSES

Treatment of Missing Data

For the session-by-session measures (AAQ and SSPS), therapists administered the questionnaires at even-numbered treatment sessions. Thirty-eight participants completed measures at all five sessions, 11 completed four measures, 1 completed three measures, and 2 completed two measures. The 2 participants who completed less than 50% of the session measures were excluded from analyses. For the outcome measures (symptoms, quality of life, and anhedonic depression), 4 participants were missing data at baseline with 2 participants missing only the quality-of-life measure and 2 missing all three measures. At posttreatment, 7 participants were missing data: 3 missing data only on the quality-of-life measure and 3 missing data on all three measures.

To account for missing data in analyses, multilevel modeling was used to analyze change in session-by-session measures over time (see details below), and multiple imputation using a multivariate normal method with 20 imputations was used to estimate missing data on outcome measures for mediation analyses. Consequently, 50 participants were included in data analyses.

Change in experiential avoidance (AAQ) and cognitions (SSPS) during treatment

To test whether there was a significant decline in AAQ and SSPS during treatment and whether the rate of change in these measures differed by treatment group, we used multilevel modeling in Stata 12. The ACT group was coded as 1 and the CBT group as 0. To estimate change in AAQ and SSPS over time, we modeled time at Level 1, and individual at Level 2. The Level 1 predictor was session, and the Level 2 predictor was group. To test whether change in each mediator differed by group, we included the Session × Group interaction. We modeled session as a continuous variable and examined the quadratic effect. Therefore, the model included the following predictors: Session, group, Session × Group, Session 2, and Group × Session2. Random effects of intercept, linear slope, and the covariance among them were included (i.e., unstructured Level 2 variance/covariance structure). The random effect of the session2 term was not included because its inclusion did not improve the model fit. For Level 1 residuals, within-group errors were modeled using an autoregressive structure.

Experiential avoidance (AAQ) and cognitions (SSPS) as mediators of treatment outcome

Mediation was tested using the MacArthur guidelines as outlined by Kraemer et al. (2002). Using this method, mediators of treatment outcome must (a) occur during treatment, (b) correlate with treatment condition, and (c) either relate directly to treatment outcome or interact with treatment group in relation to the outcome. For all analyses using AAQ and SSPS as possible mediators, criterion (a) was met given that these variables were assessed during treatment. Criteria (b) and (c) were tested using seemingly unrelated regression (SUR) in Stata 12. SUR allows for simultaneous estimation of multiple equations. This method calculates parameter estimates and standard errors accounting for the correlation among residual variances among models. This method produces more efficient parameter estimates than ordinary least squares (OLS) regression, and allows for calculation of the indirect effect for multiply, imputed data. For details on how to implement this method, see the UCLA Stata Frequently Asked Questions page (UCLA: Statistical Consulting Group, 2013)

Outcome variables included symptoms, quality of life, and anhedonic depression. Because quadratic trends were found in the AAQ and SSPS (see the Results section and Fig. 2), using Stata 12, estimates of the quadratic effect for each participant were obtained by extracting random effects of session2 for each participant and adding those random effects to the fixed effect of session2. These estimates represent the direction and steepness of curvature for each participant. Positive curvature indicates that the curve is convex, or that change occurs more quickly at the beginning of treatment and levels off toward the end. Negative curvature indicates that the curve is concave, or that change occurs more slowly at the beginning of treatment and then speeds up toward the end. The estimates of curvature were examined as mediators using SUR. The independent variable was treatment condition with ACT coded as 1 and CBT coded as 0.

FIGURE 2

Change in (A) Acceptance and Action Questionnaire and (B) Negative Self-Statements over sessions 2 through 10.

For each mediator and outcome, we first tested moderated mediation, or whether treatment group interacted with the mediator to affect outcome (see Fig. 1A). For moderated mediation, the two regression equations included in each SUR were as follows: The first equation included the mediator variable regressed on the outcome variable at pre (covariate) and treatment condition (independent variable). The second equation included the outcome variable at post regressed on the outcome variable at pre (covariate), the mediator, treatment condition (independent variable), and mediator × treatment condition. If moderated mediation was not found, mediation was tested (see Fig. 1B). For mediation, the two regression equations included in each SUR were as follows: The first equation included the mediator variable regressed on the outcome variable at baseline (covariate) and treatment condition (independent variable). The second equation included the outcome variable at post regressed on the outcome variable at pre (covariate), the mediator, and treatment condition (independent variable). Finally, if mediation was not found, we tested the mediators as predictors of treatment outcome using OLS regression (see Fig. 1C): the equation included the outcome variable at post regressed on the outcome variable at pre (covariate), the predictor, and treatment condition (as a covariate).

FIGURE 1

Moderated mediation (A), mediation (B), and prediction (C) models tested.

Results

References to AAQ and SSPS refer to scores on the mediation measures taken every other session from Session 2 through Session 10 (5 total data points).

CHANGE IN EXPERIENTIAL AVOIDANCE (AAQ) AND COGNITIONS (SSPS) DURING TREATMENT

Acceptance and Action Questionnaire

Results for the AAQ are displayed in Fig. 2A. The Session2 × Group interaction was significant (b = .35, 95% confidence interval (CI) = .17 to .54, p = .001). AAQ scores declined in both groups. The quadratic term was significant within ACT (b = .20, z = 2.76, p = .006) and CBT (b = −.16, z = −2.60, p = .009). Because the beta coefficient associated with the quadratic term was positive in ACT, the curve was convex, meaning that the decline in ACT was steeper at the beginning of treatment and leveled off toward the end of treatment. In CBT, because the beta coefficient associated with the quadratic term was negative, the curve was concave, meaning that decline was level at the beginning of treatment and became steeper toward the end of treatment. AAQ scores were significantly lower in ACT than CBT at Sessions 4, 6, 8, and 10 (ps < .012), but did not significantly differ at Session 2 (p = .051).

Results for the SSPS are displayed in Fig. 2B. The Session2 × Group interaction was significant (b = .21, CI = .03 to .40, p = .025). SSPS scores declined in both groups. The quadratic term was significant in ACT (b = .19, z = 2.33, p = .020), and the linear term, but not the quadratic term, was significant in CBT (b = −1.56, z = −8.60, p < .001). The curve in ACT was convex, indicating that the decline in ACT was steeper at the beginning of treatment and leveled off toward the end of treatment. The rate of decline in CBT remained constant throughout treatment. SSPS scores were significantly lower in ACT than in CBT at Sessions 4, 6, and 8 (ps < .039), but did not significantly differ at Sessions 2 or 10 (ps > .327).

EXPERIENTIAL AVOIDANCE (AAQ) AND COGNITIONS (SSPS) AS MEDIATORS OF TREATMENT OUTCOME

For all analyses using AAQ and SSPS as possible mediators, criterion (a) was met given that these variables were assessed during treatment. In this section, we tested simultaneously whether criteria (b) and (c) were met using SUR. For these analyses, the mediators were the curvature (coefficient associated with Session2) of AAQ and SSPS for each individual participant. For each mediator, we first tested whether treatment group interacted with the mediator to affect outcome (moderated mediation). In the absence of a significant interaction, we tested the mediated effect regardless of treatment group. Finally, if no mediation was found, we tested the mediators as predictors of treatment outcome (see Fig. 1 for models).

Social Anxiety Symptoms

Results are displayed in Table 1. For AAQ curvature, the moderated mediation model was significant. Group was a significant predictor of AAQ curvature (b = .23, p < .001) such that participants in ACT had significantly more positive curvature (more convex curve) than participants in CBT. Group significantly interacted with AAQ curvature to predict social anxiety symptoms (b = −2.72, p = .024, η2 = .16). Tests of simple effects revealed a significant indirect effect (a × b) for ACT (b = −.57, p = .006), but not CBT (b = .05, p = .813), such that more positive AAQ curvature (more convex curve) was associated with greater symptom reduction in ACT, but not in CBT. Therefore, criterion (c) was met and AAQ curvature mediated symptom reduction in ACT, but not in CBT.

Table 1

Significant moderated mediation and prediction models (see Fig. 1 for models)

Moderated Mediation (mediator interacts with treatment group)
Dependent Variable	Mediator	X → M (a path)	X* M → Y (b2 path)	M → Y (b1 path)	indirect effect ACT (a*(b2 + b1))	indirect effect CBT (a* b1)
Social Anxiety Symptoms	AAQ curvature	.23*** (.12 to .34)	−2.7* (−5.1 − .36)	.23 (−1.7 to 2.2)	−.57** (−.97 to −.16)	.05 (−.39 to .50)
Depressive Symptoms	AAQ curvature	.23*** (.12 to .33)	−75** (−118 to −32)	39 (.89 to 77)	−8.2* (−16 to −.80)	8.9 (−.76 to 18)
Prediction (non-significant mediators)
Dependent Variable			Predictor			P → Y (b path)
Social Anxiety Symptoms			SSPS curvature			−2.0** (−3.2 to −.76)
Depressive Symptoms			SSPS curvature			−30* (−54 to −5.8)

Note.

p < .05,

p < .01,

p < .001;

X = treatment condition (0 = CBT and 1 = ACT); M = mediating variable; Y = outcome at Post; P = predictor variable; AAQ curvature = non-linear effect of Acceptance and Action Questionnaire; SSPS curvature = non-linear effect of Self Statements Questionnaire.

For SSPS curvature, neither the moderated mediation model nor the mediation model was significant (ps > .451). For the prediction model, SSPS curvature significantly predicted social anxiety symptoms (b = −1.97, p = .002, η2 = .14) such that more positive SSPS curvature (more convex curve) was associated with fewer social anxiety symptoms at post across both groups. Criterion (c) was not met and SSPS curvature did not significantly mediate symptom outcome. However, SSPS curvature did significantly predict symptom outcome.

Quality of Life

For AAQ and SSPS curvature, neither the moderated mediation model, nor the mediation model, nor the predictor model was significant (ps > .106). Therefore, criterion (c) was not met and neither AAQ nor SSPS curvature mediated quality of life.

Anhedonic Depression

Results are displayed in Table 1. For AAQ curvature, the moderated mediation model was significant. Group was a significant predictor of AAQ (b = .23, p < .001) such that participants in ACT had significantly higher AAQ curvature than participants in CBT. Group significantly interacted with AAQ curvature to predict anhedonic depression (b = −74.76, p = .001, η2 = .21). Tests of simple effects revealed a significant indirect effect (a × b) for ACT (b = −8.19, p = .030), but not CBT (b = 8.85, p = .071) such that more positive AAQ curvature (more convex curve) was associated with greater anhedonic depression reduction in ACT, but not in CBT. Therefore, criterion (c) was met and AAQ curvature mediated anhedonic depression in ACT, but not in CBT.

For SSPS curvature, neither the moderated mediation model nor the mediation model was significant (ps > .453). For the prediction model, SSPS curvature significantly predicted anhedonic depression (b = −29.74, p = .017, η2 = .15) such that more positive curvature of SSPS (more convex curve) was associated with fewer social anxiety symptoms at post across both groups. Criterion (c) was not met and SSPS curvature did not significantly mediate anhedonic depression. However, SSPS curvature did significantly predict anhedonic depression.

Discussion

The first goal of the current study was to examine how theorized mediators in ACT and CBT changed over the course of treatment. In particular, we were interested in whether avoidance of uncomfortable internal experiences (experiential avoidance) decreased to a greater extent in ACT than in CBT, and whether negative cognitions in social situations decreased to a greater extent in CBT than in ACT. Experiential avoidance showed a nonlinear decline in both treatments, with ACT showing a convex curve (steeper at the beginning of treatment) and CBT showing a concave curve (steeper at the end of treatment). As hypothesized, experiential avoidance decreased to a greater extent in ACT than in CBT. This finding is not surprising given that ACT directly targets experiential avoidance in the first five sessions. However, during the second half of treatment in which exposure to reduce fearfulness is emphasized in CBT and exposure to increase living a valued life is emphasized in ACT, the rate of decline in experiential avoidance increased in CBT and slowed in ACT. This suggests that the cognitive defusion approach used in ACT and the behavioral exposure approach used in CBT may have the most pronounced effect on willingness to experience uncomfortable internal experiences, although this cannot be directly tested in the current study. This finding replicates previous research, which showed greater decreases in experiential avoidance in ACT than in CBT (Arch, Wolitzky-Taylor, et al., 2012).

Negative cognitions also decreased significantly in both treatments. The decrease was nonlinear and convex (steeper at the beginning of treatment) in ACT, and was linear in CBT. Contrary to hypotheses, at the beginning of treatment, negative cognitions decreased more quickly in ACT than in CBT. However, by the end of treatment, both groups had shown the same reduction in negative cognitions as the decline slowed in ACT but remained constant in CBT. This finding replicates previous research, which showed greater decreases in negative cognitions in ACT than in CBT (Arch, Wolitzky-Taylor, et al., 2012). Again, although not directly tested, the cognitive defusion approach used in ACT and the behavioral exposure approach in CBT seem to have caused the greatest reduction in negative cognitions. This finding is surprising given that ACT does not directly aim to change thinking patterns while CBT does. However, cognitive restructuring has been compared with thought suppression (Hayes et al., 1999) because it labels certain cognitions as “faulty,” which may encourage suppression of those thoughts. Suppression is thought to be counterproductive, according to the ACT framework, as it can result in increased intrusion of suppressed thoughts (Wegner, 1994). In addition, Eifert and Forsyth (2005) have argued that focusing attention on the content of negative thoughts maintains the ruminative cycle. These findings lend support to the idea that mindfulness and acceptance may be a more effective way of reducing negative thoughts than cognitive restructuring, while behavioral exposure to feared situations in CBT may prove to be more effective at reducing negative thoughts than the values-driven approach emphasized in ACT. In other words, despite similarities between the behavioral approaches in ACT and CBT, the differences in framing and structure appear to yield differential effects on cognitive change.

The second goal of the study was to examine experiential avoidance and negative cognitions as mediators of treatment outcome. We tested whether the curvature (or nonlinear effect) of experiential avoidance and negative cognitions mediated outcome differentially in ACT and CBT (moderated mediation), and if there were not group differences, whether the mediators were shared between the two treatments. If neither mediation model was significant, we tested a prediction model. Using the MacArthur guidelines as outlined by Kraemer et al. (2002), the curvature in experiential avoidance emerged as a significant mediator of social anxiety symptom reduction and depressive symptom reduction in ACT and not in CBT such that participants with greater reduction in experiential avoidance at the beginning of treatment had better outcomes only within ACT. Consistent with hypotheses and contrary to previous results examining treatment mediation (Arch, Wolitzky-Taylor, et al., 2012), experiential avoidance was a mechanism of treatment outcome only within ACT. Although curvature in negative cognitions did not differ between the two treatments and therefore was not a significant mediator, it was a predictor of improvement regardless of group such that greater reduction in negative cognitions at the beginning of treatment predicted better outcome. These findings indicate that how quickly negative cognitions change in treatment is an important factor in improvement across both treatments.

It is important to note that no significant differences in treatment outcome emerged between ACT and CBT, which makes interpretation of mediation results more complex. For experiential avoidance, greater reductions were found in ACT than in CBT, and experiential avoidance was related to better treatment outcome more so in ACT than in CBT. These findings imply better outcomes in ACT than CBT, but we did not find such differences. There are at least two factors that may explain this contradiction. The first is an additional mediator that decreases to a greater extent in CBT than in ACT may not have been measured in the current study. Possibilities include control over negative thoughts, self-efficacy, behavioral avoidance, or adherence to behavioral exposures. The second possibility is that ACT did, in fact, work better than CBT, but that we did not have sufficient power to detect this difference. As indicated in Craske et al. (2014), we had enough power to detect only a large effect size difference between ACT and CBT. That being said, differences between ACT and CBT did not even approach significance in the complete sample, and for some outcome measures, participants in CBT showed marginally greater symptom reduction than did those in ACT (Niles, Mesri, Burklund, Lieberman, & Craske, 2013). In another study comparing ACT and CBT across multiple anxiety disorders with a larger sample size, no group differences were found (Arch, Eifert, et al., 2012). Therefore, it is unlikely that ACT would have outperformed CBT had our sample been larger. It is more likely that the key mechanism of change for CBT was not measured in the current study.

These findings provide evidence that the rate of change in experiential avoidance and negative cognitions, particularly at the beginning of treatment, does, in fact, explain a significant portion of the variance in treatment outcome (approximately 15–20%), albeit differentially in ACT and CBT. This is the first study to show that change mechanisms may differ in ACT and CBT. Given that experiential avoidance is the primary target of change in ACT, it is not surprising that experiential avoidance was a significant mediator only for ACT. However, the results conflict with an earlier study with a mixed anxiety disorder sample (Arch, Wolitzky-Taylor, et al., 2012), where differential mediation in ACT and CBT was not established. One possible explanation of the contradictory findings is that treatment mechanisms differ for a social anxiety sample and a mixed anxiety sample. The studies also differed in the way the mediators were measured and treated in analyses. For the ACT-specific mediator, the current study used the AAQ, which is more widely validated than the Believability of Anxious Feelings and Thoughts Questionnaire used in the previous study. The CBT-specific mediator for the current study was a cognitive measure specific to social situations, whereas the measure in the previous study, the Anxiety Sensitivity Index, was intended to capture cognitive mechanisms across anxiety disorders. Finally, the current study examined nonlinear change of the mediators and used simultaneous regression models to test mediation, while the previous study assessed linear change for the mediators and tested mediation using two separate models. Therefore, although both approaches can be used to assess mediation, the different methods used in the two studies could explain the difference in findings.

Several study limitations should be noted. First, we were only able to test one hypothesized mediator of each treatment model given that measurements were taken repeatedly throughout treatment and concerns about participant burden limited the number of mediators that could be assessed. Future studies may aim to assess additional mediators, including controllability of negative thinking, self-efficacy, or the content of anxiety-related thoughts for CBT, and acceptance, cognitive defusion, behavioral commitments, or values clarification for ACT. Second, ACT and CBT were matched on the number of sessions spent on exposure exercises, which likely minimized group differences in mediation pathways. Comparing cognitive therapy to a mindfulness-based approach to anxiety treatment would likely result in greater group differences in mediation. Third, our assessed mediators and outcomes were based on self-report questionnaires. Future studies would benefit from integrating behavioral and brain-based measurements of mediators and outcomes. Fourth, the SSPS was developed for use in a public speaking context and reflects cognitions that individuals with social phobia report following public speaking. We therefore cannot be certain that the negative cognitions captured with this measure are the same as those experienced in other social situations. Fifth, the test–retest reliability of the AAQ-9 was only .64, which indicates that the measure may not be sufficiently stable over time. The 16-item version, however, demonstrated a correlation of .86 between Sessions 8 and 10 of treatment, indicating that within our sample, the measure was quite stable. Sixth, the AAQ and SSPS were significantly correlated (r ranged from .47 to .71), indicating that these measures assess similar constructs. Despite significant overlap, however, it is notable that mediation results differed between the two measures. Finally, our sample was relatively young, well educated, and more likely to be single compared with the population. Therefore, it is possible that our results would not generalize to a more diverse sample.

In conclusion, these findings shed light on how cognitive processes change in ACT and CBT for social anxiety disorder and how these processes relate to treatment outcome. For both treatment approaches, negative thoughts decreased and willingness to have uncomfortable internal experiences increased, indicating that these treatment approaches overlap significantly in terms of their effect on cognitive change. Arch and Craske (2008) reviewed the similarities between these two treatment approaches, and the current project adds to existing research on these similarities (Arch, Wolitzky-Taylor, et al., 2012). Patients whose negative thinking decreased more quickly at the beginning of treatment demonstrated the greatest symptom reduction across both treatment approaches, and those with the fastest increase in willingness to experience uncomfortable sensations had greater symptom reduction in ACT but not in CBT. In other words, faster reduction in negative cognitions at the beginning of treatment predicted overall improvement, and faster reduction in experiential avoidance predicted improvement only for ACT.

These findings have a number of important implications for treatment. First, change in the core mechanisms of psychopathology early in treatment may enable subsequent symptom change to occur in the time remaining before the end of treatment. By extension, the role of early change may differ if symptom change was examined over a lengthier interval. Second, these findings could indicate that change in negative beliefs is central to social anxiety psychopathology independent of treatment approach, whereas change in acceptance is less relevant to psychopathology in general, but more relevant to the effects of a specific treatment strategy. Third, early change in core mechanisms may allow other critical mechanisms, such as engagement in exposure and values-driven activities, to take place, which may contribute to better symptom outcome. Finally, these findings highlight the importance of strategies specifically designed to reduce negative cognitions and experiential avoidance early in treatment, such as emphasizing cognitive defusion and exposure within the first few sessions of treatment. Overall, these findings add to a growing body of literature on treatment mediation for anxiety disorders and begin to shed light on differences in mechanisms between ACT and CBT for social anxiety disorder.