The efficacy of manualized Cognitive Behavior Therapy conducted by student-therapists treating Danish youths with anxiety using a benchmark comparison.

BACKGROUND: Several systematic reviews have demonstrated the efficacy of cognitive behavioral therapy (CBT) treating anxiety disorders in children and adolescents (hereafter referred to as youths). Treatment of anxiety disorders conducted by student therapists (ST) has been found to be an effective alternative, to treatment conducted by psychologists.
OBJECTIVE: The primary aim of the study is to investigate the effects of ST treating youths, using a group-based CBT program. Second, the study aims to compare these results with outcomes achieved by professional-therapists (PT).
METHOD: The study investigate in an open trial design, the treatment outcome from a manualized CBT program (Cool Kids) in a group of youths (n = 54) treated by ST. Results are benchmarked against the outcomes of a group of youths (n = 56) treated by PT using the same program, derived from a separate randomized controlled trial.
RESULTS: There was a significant reduction of both self-reported and clinician rated measures of youth anxiety over time in the ST group, with small to large effect sizes. No significant differences of improvements in self-report measures were found between the ST and the PT groups. There was no significant difference in remission rates for participants' primary anxiety disorder between the ST (50.0%) and the PT (66.1%) at post-treatment or at three-month follow-up (ST: 74.1%, PT: 76.8%). There was a significant difference regarding number of youths free of all anxiety disorder between the ST (14 [25.9%]) and PT (27 [48.2%]) group at post-treatment. This difference was not significant at three-month follow-up (ST: 25 [46.3%], PT: 33 [58.9%]).
CONCLUSION: The findings support previous findings, suggesting that student-therapists, receiving training and supervision, can successfully treat youths with anxiety disorders using a manualized CBT program. The outcomes following CBT treatment conducted by ST are comparable to outcomes achieved by PT.

Introduction

Anxiety disorders are some of the most common psychiatric disorders among children and adolescents (hereafter referred to as youths) (1,2). In an epidemiological meta-analysis, the prevalence for any anxiety disorder was found to be 12.3% among children (age 6-12 years old) and 11.0% for adolescents (age 13-18 years old) (1). In Denmark, the yearly number of youths diagnosed with depression or an anxiety disorder has tripled from 2006 to 2016 (3). Although many anxiety disorders remit by themselves without intervention within three to four years (4), there are studies showing that many adult cases of anxiety disorders may have their onset in childhood (5). Furthermore, findings suggest that youth anxiety may represent a significant risk of developing other disorders, like other anxiety disorder, depression, and substance abuse (4,6). Youths suffering from anxiety disorders have a number of psychosocial impairments compared to non-anxious youths. Among them are impairment in peer-relations, attention, school performance, and social behavior (7). There are also a considerable societal costs related to youth anxiety (8). There exists several effective psychological treatments aimed at youths with anxiety disorders, which have been tested in randomized controlled trials (RCT), yet many youths do not receive effective treatment (9–11). Researchers describe this discrepancy between the effective treatments of youth anxiety used in research and the lack of effective treatment in clinical practice, as a dissemination problem. They highlight a need for spreading knowledge of effective treatments aimed at youths with anxiety disorders and training therapists in these treatments, thus making treatment more available and the therapists conducting the treatment more qualified (12).

The efficacy of cognitive behavioral therapy (CBT) treating anxiety disorders among youths have been thoroughly tested in several randomized controlled trials (RCT). A systematic review of 41 RCT studies using cognitive CBT to treat youth anxiety disorders found a remission rate for any anxiety diagnosis, of 58.9% for CBT versus 16 % for control groups (9). The effectiveness of CBT treating youths with anxiety disorders has also been tested in more ‘realworld’ conditions like schools or community-clinics, showing effects comparable to previous efficacy trials (13,14).

Intervention studies regarding anxiety disorders are often conducted in university settings, where psychology students are utilized as therapists, and practice psychotherapy in university clinics as an integrated part of their training and education toward becoming a psychologist (15–18). In a large effectiveness study, investigating the effects of student therapists (ST) conducting CBT (15), they reported the treatment outcomes of 591 adult patients treated by ST in a university clinic in Stockholm, Sweden. Patients mainly had an anxiety disorder or depression/dysthymia as their primary diagnosis, and were treated with individual CBT for a mean of 18 sessions. The ST received basic CBT training at the university, and weekly supervision during treatment. Patients improved significantly on both measures for anxiety and depression. The study also used a benchmarking strategy to compare the recovery rates on anxiety and depression with a study using professional therapists (PT). The comparison showed that the ST achieved twice the recovery rates on anxiety, and almost twice the recovery rates for depression compared to the PT (15). However, there are some limitations to these findings as the Swedish study (15) and the English study used as a benchmark (19) used samples from different countries and in different treatment settings. The benchmark study also lacked information regarding the treatment given in the study and the training the PT received (19). Thus, limiting the generalizability of the comparison of the ST and PT (15). However, the findings are in line with the results from other studies, comparing the effects of ST with PT treating adults with anxiety disorders using CBT (18,20–22). Indicating that relatively inexperienced therapists can provide sufficient treatment for anxiety disorders in adults with appropriate training and supervision. In a study conducted by Higa-McMillan et al. (2015), they reviewed clinical research regarding treatment of youth anxiety. One of their findings indicated that ST at a bachelor-, master, and PhD degree levels could provide effective highly manualized treatment for anxious youths (23). These studies did not compare the effects of ST with professional therapists (PT) (e.g. psychologists or psychiatrists), and lacked an overview of therapists training and supervision.

There exists a vast number of effective evidence-based treatments (EBT) treating youth anxiety (9). Yet, there is still a need for more therapists qualified to provide these treatments. Psychology students conducting anxiety treatment, as part of their clinical training or education, could possibly alleviate a significant proportion of youths diagnosed with anxiety disorders each year. However, integrated evidence-based clinical training as part of the education of psychologists is not common practice in Denmark. This is the first study to investigate the effects of student therapists (ST) conducting CBT, treating youths with anxiety disorders in a university clinic in Denmark. The study is also the first study to compare the effects of ST with PT treating youths with anxiety disorders.

Aims of the study

The primary aim of the current study was to evaluate the treatment effects achieved by ST treating youths with anxiety disorders in a university setting, using a manualized CBT program (Cool Kids). Second, the study aims to compare these results with outcomes achieved by PT. Based on previous studies, it was hypothesized that CBT delivered by ST would produce improvements, comparable to the improvement achieved by PT.

Method

Participants

Participants in both the ST and PT group consisted of Danish youths and their parents, seeking help regarding youth anxiety. Assessment and treatment were conducted at Centre for the Psychological Treatment of Children and Adolescents (CEBU), Department of Psychology and Behavioral Science, Aarhus University, Denmark. Procedures for assessment and inclusion were identical for both groups. Inclusion criteria were that participants (7-16 years) met the criteria for an anxiety disorder according to DSM-IV as the primary diagnosis. Exclusion criteria were psychosis, untreated attention deficit hyperactivity disorder (ADHD), intellectual disability, severe behavior disorders, eating disorders, and substance abuse. Participants were encouraged not to engage in other forms of treatment or change psychopharmacological medication during the treatment period. All families signed written consent forms regarding the use of their data in scientific publications, audio-visual recording, and information exchange (e.g. physicians and teachers).

Participants in the PT group were assessed for eligibility from January 2011 to April 2012, and consisted of 56 youths aged 7 to 16 years and their parents, who were enrolled in a randomized waitlist controlled trial (24). The randomization procedure was done in three blocks of six groups per block stratified to age groups (7-9, 10-12, 13-16), resulting in three treatment groups for each age group in both the treatment and waitlist condition (24). All participants were treated by PT. In total, 56 mothers (100%) and 55 fathers (98.2%) participated in the treatment.

Participants in the ST group were assessed for eligibility from August 2014 to September 2015 and consisted of 54 youths aged 7 to 12 years and their parents. All enrolled participants received treatment in nine separate groups, with six participants in each group. Participants were allocated in groups corresponding to their age, four groups with youths aged 7 to 9 years, and five groups with youths aged 10 to 12 years. All participants were treated by ST. In total, 54 mothers (100%) and 49 fathers (90.7%) participated in the treatment.

The attrition rates were low in both groups for the clinical assessment. In the ST group, two (3.7%) and four (7.4%) participants were lost at post and follow-up (FU) clinical assessment, respectively. In the PT group, no participants were lost to post or FU clinical assessment. Missing responses on the questionnaires were generally low for the youths, mothers, and fathers at post (n: 4 youths (7.4%); 2 mothers (3.7%); 1 father (2.0%) and FU (n: 5 youths (9.2%); 3 mothers (5.5%); 8 fathers (16.3%)) in the ST group. Missing response rates were also low in the PT group among youths, mothers, and fathers at post (n: 1 youth (1.7%) and FU (n: 4 youths (7.1%); 1 mother (1.7%); 4 fathers (7.2%). There were no significant differences between the groups relating to attrition or missing responses.

Measures

Primary outcome measure

The Anxiety Disorder Interview Schedule for DSM-IV, Parent and Child Versions (ADIS-C/P) (25) was utilized as the primary measure. The ADIS-C/P is a semi-structured diagnostic interview assessing youth anxiety disorders based on the criterions from the DSM-IV. Information is gathered by separate interviews with the youths (ADIS-C) and their parents (ADIS-P). A Clinician Severity Rating (CSR) is given, ranging from 0 (no interference) to 8 (extreme interference). A CSR of 4 or above indicates a disorder, while a CSR of less than 4 is considered subclinical. The most impairing ADIS diagnosis was considered the primary diagnosis. Diagnostic interviews were conducted by psychologists or graduate students trained in the use of ADIS-C/P. The training consisted of a two day workshop, watching two gold standard ADIS interviews of parents and children, observing one live interview conducted by an experienced interviewer, and conducting one ADIS interview observed by an experienced ADIS interviewer.

The ADIS-C/P has previously proven to be a reliable instrument for assessing anxiety disorder symptoms and diagnoses based on the DSM-IV, with good to excellent test-retest reliability (26). In the RCT study used as the PT group benchmark, an interrater reliability check of the Danish version of ADIS-C/P was conducted by two trained assessors watching and rating 22 (20.2%) of the video-recorded baseline interviews (24). The interrater reliability (Cohen’s κ) for the primary anxiety diagnosis was 0.77. The intraclass coefficient for the CSR of the primary anxiety diagnosis was 0.69. The education, procedure, and supervision of the assessors in the ST group were identical to the assessors in the PT study. No interrater reliability check was conducted for the ST group (24).

Secondary outcome measures

The Spence Children’s Anxiety Scale (SCAS) is a self-report rating scale assessing youth anxiety symptoms (27). The scale consists of 44 items (including six positive filler items), rated from 0 (never) to 3 (always). Within the scale are six subscales based on the anxiety disorders from DSM-IV: social phobia (SoP), panic disorder (PD) and agoraphobia (AP), separation anxiety disorder (SAD), generalized anxiety disorder (GAD), obsession compulsive disorder (OCD), and specific phobia (called fear of physical injury [SP]). The Danish version of the SCAS has demonstrated excellent internal consistency for the total scale in a clinical sample of youths with anxiety disorders, good test-retest reliability after two weeks and three months and good convergent and divergent validity (28). For the current study, the internal consistency for the total SCAS was good, and within the range of previous findings (α = 0.88). The SCAS parent version (SCAS-P) includes the same items as the SCAS, without the six filler items, and is administered and scored like the SCAS (29). The Danish version of SCAS-P has shown good internal consistency for both mothers (α = 0.87) and fathers (α = 0.87) in a clinical sample (Arendt et al., 2014). The current study showed good internal consistency for both mothers (α = 0.90) and fathers (α = 0.88).

The Child Anxiety Life interference Scale (CALIS) measures life interference and impairment associated with anxiety (e.g. in areas like school, at home or with friends) from the youth and parents perspective (30). The CALIS consists of 10 items where the youth reports the interference of their own life. The parent version of the CALIS (CALIS-P) consists of two subscales examining the interference of their child’s life (9 items) and of their own life (9 items). The scale is scored from 0 (not at all) to 4 (a great deal). CALIS and CALIS-P has previously demonstrated acceptable internal consistency for youths and parents, moderate-to-high test retest reliability, significant interrater reliability, and good convergent and divergent validity in a Australian population (30). The internal consistency of CALIS and CALIS-P for the current study was good for youths (α=0.85), and excellent for mothers (α = 0.91) and for fathers (α = 0.92).

The Experience of Service Questionnaire (ESQ) is a measure used to assess the youth’s and parents’ satisfaction with the treatment (31). There are separate versions for youths, with seven items, and parents, with 10 items, rated 0 (not true), 1 (partly true), or 2 (true).

Treatment

The Cool Kids program is a manualized generic group CBT program for youth with anxiety disorders, involving both youths and their parents (32). The treatment aims to reduce levels of anxiety by focusing on teaching youths to recognize their emotions, restructure negative automatic thinking, and gradually confront feared situations (i.e. exposure). A systematic review and meta-analysis of 16 studies exploring the effects of the Cool Kids program, showed a moderate pre-post effect size for all the studies combined (g = 0.65). The study also included 11 studies comparing the Cool Kids program with a control group, which yielded a moderate between groups effect size (g = 0.54) (33). The efficacy and effectiveness of the Danish version of the Cool Kids program has also been found effective in treating anxiety disorders among youths (13,24).

The treatment in the current study consisted of 10 2-hr weekly group sessions, over approximately 12 weeks. The treatment sessions were typically divided so that the youths and parents were introduced to new techniques and worked with different exercises separately with assistance from the therapists. In addition to learning and assisting with the strategies taught to their children, the parents also received psycho-education in parent management strategies (34,35).

All sessions took place at CEBU, except for the in vivo session (Session 8) which took place at a local shopping center where the families and therapists practiced exposure relevant to the youths’ anxiety. Three months after the end of treatment, participants were offered a 1-hr booster group session.

The treatment was identical for both the PT and ST group. Treatment in the PT group was conducted by a psychologist, and assisted by three graduate psychology students (helped with practical tasks, and assisted families during in-session assignments). Treatments in the ST group were conducted solely by three graduate students.

Therapists

Student Therapists: 24 graduate psychology students functioned as therapists in the ST group. They were in their first or second year of their master program. All students had received training in the Cool Kids program (10-hr workshop) and had assisted in one or two treatment groups using the Cool Kids program where they had been assigned to one family, and assisted in treatment assignments and exposure exercises. The ST had no other prior experience with the Cool Kids program. They received weekly (2-hr) group supervision throughout the treatment. The group supervision included a maximum of six ST, led by a psychologist with extensive experience with the Cool Kids program. During each supervision session, three ST presented their treatment case, their assigned families, their progress and possible challenges in treatment. The cases were discussed within the group, and the future work with the youth and their families were planned (e.g. planning gradual exposure or homework). Each ST presented their case, a minimum of two times during the treatment course.

Professional Therapists: two psychologists functioned as therapist in the PT group. They were both trained and supervised in the use of the Cool Kids program by an authorized psychotherapy specialist who had either developed or received training in the Cool Kids program at Macquarie University. The psychologists had five, and one year prior clinical experience, respectively. In each of the treatment groups, three graduate psychology students assisted the psychologist in treatment. Differences in therapist effects were explored for the primary treatment outcomes, and no significant differences were found between the two therapists.

Design and procedure

Families had prior to assessment, referred themselves to CEBU, in response to recommendations from other health institutions, friends, advertisements, and the clinic’s webpage. The families where required to send a description of the youth’s anxiety symptoms, and relevant families were called in for assessment using the diagnostic ADIS-C/P interview. Families who were offered and accepted treatment completed electronically administered rating scales prior to treatment start.

Participants were assessed, at pre-treatment (pre), post-treatment (post), and at a three-month follow-up (FU), using the ADIS-C/P interview and electronically administered rating scales. Assessors were masked to youth’s prior diagnosis at post- and three-month follow-up assessments.

Statistical analyses

The analyses included all participants with pre-treatment data. Missing data were controlled for by using Little’s Missing Completely at Random test. Results indicated that the missing data were missing completely at random. Missing data at post or FU were accounted for by using the last observation carried forward approach, bringing forward the participant’s last observation (i.e. pre or post measures) replacing the missing values at post or FU. All analyses and results were conducted based on the intention to treat sample (ITT).

Participant’s baseline demographics, clinical characteristics and scores on outcome measures were compared between the two groups using independent samples t-tests and Fisher’s exact test as appropriate (i.e. gender).

The magnitude of change (effect size [ES]) within groups was estimated using Hedges’ g, to control for the small sample size (36). Effect sizes were calculated for changes from pre to post and pre to FU on all measures of continuous anxiety outcomes.

Pre to post and pre to FU comparisons between the two groups were based on degree of change on continuous outcome measures following treatment. Repeated-measure, mixed-model analyses of variance (ANOVA) were conducted with group (ST vs PT) as the between-subjects variable and time (pre- vs post-treatment/and pre- vs three-month follow-up) as the within subjects variable. The magnitude of change between groups was calculated using partial η2.

The proportion of youths meeting criteria for reliable change (RC) and clinical significant change (CSC) on the primary measure of youth anxiety (i.e. SCAS and SCAS-P) was calculated, in accordance to Jacobs and Truax criteria, using age and gender-specific Danish norms to calculate individual cut-off scores (28,37). The RC represents the amount of change required to make the change statistical significant. The CSC represents a change that is both statistically significant (RC) and clinically significant. Clinically significant change requires a change in means crossing a cut-off score between a clinical and non-clinical population, according to Jacobson and Truax (1991). Clinical cut-off scores were defined as the mid-point between clinical and non-clinical mean-scores from norms reported in the study of Arendt et al. (2014). Clinical significant change (CSC) was defined as the proportion of those scoring above the clinical cut-off before treatment, achieved RC, and scored below clinical cut-off after treatment (37). Reliable change and CSC calculations were compared between the two groups using Fischer’s exact test.

The proportion of participants that were free of their primary anxiety diagnosis and all anxiety diagnosis at either post-treatment or at the three-month follow-up were compared across the two groups using Fisher’s exact test.

All statistical analyses were carried out using Statistical Product and Service Solutions (SPSS) version 22.0.0.0 (38), and all ES were calculated using Comprehensive Meta-Analysis Version 3 (CMA) (39).

Results

Sample characteristics

Tests of normality were conducted using the Shapiro–Wilk test of normality, as well as a visual inspection of the data, to assess the normal distribution for the primary and secondary measures in both groups. All scores were approximately normally distributed in both the ST and PT group, with the exception of the mothers SCAS-P score in the PT group (W = 0.96, p = 0.04) with a skewness of 0.70 (SE = 0.32) and kurtosis of 0.47 (SE = 0.63). By visual inspection, one score was detected as an outlier. By removing the outlier, the mothers SCAS-P scores in the PT were normally distributed (W = 0.96, p = 0.08) with a skewness of 0.51 (SE = 0.32) and kurtosis of 0.08 (SE = 0.63). The analysis was conducted with and without this outlier, resulting in no significant changes in outcome. The outlier was therefore dropped, and analysis conducted without the outlier.

Baseline comparison

Baseline demographic and diagnostic characteristics for the current study at pre-assessment are presented in Table 1. There were no significant differences between groups regarding gender distribution, use of medication, type of primary anxiety disorder, or CSR for primary anxiety disorder. However, there was a significant difference in youth mean age, between the ST (M = 10.18, SD = 1.46) and PT group (M = 11.82, SD = 2.49; t(89) = 4.24, p < 0.005, 95% CI [0.87, 2.41]) where participants in the PT group were significantly older. There was also a significant difference in CSR for all anxiety disorders between the ST (M = 4.60, SD = 1.33) and PT group (M = 5.32, SD = 0.95; t(95) = 3.27, p < 0.005, 95% CI [1.16, 0.28] ) where the CSR was significantly higher for participants in the PT group compared to the ST group.

TABLE 1.

Demographic and diagnostic characteristics for youths in student therapist and professional therapist group

	Youths in ST group (n = 54)	Youths in PT group (n = 56)
Mean age in years (SD)	10.18 (1.46)	11.82 (2.49)
Males (%)	32 (59.3)	25 (44.6)
On psychopharmacological medication	3 (6.7)	2 (4.5)
Primary diagnosis (%)
SAD	24 (44.4)	15 (26.8)
GAD	15 (27.8)	14 (25.0)
SoP	7 (13.0)	7 (12.5)
SP	6 (11.1)	12 (21.4)
OCD	1 (1.9)	4 (7.1)
AP without PD	1 (1.9)	4 (7.1)
CSR mean for primary diagnosis	6.33 (0.75)	6.09 (1.07)
CSR mean for all diagnosis	4.60 (1.33)	5.32 (0.95)
Comorbid diagnoses (%)
Anxiety disorders	50 (92.6)	47 (83.9)
Externalizing disorders	6 (11.2)	6 (10.7)
Mood disorder	1 (1.9)	4 (7.1)
Other	4[a] (7.4)	3[a] (5.4)
No comorbidity	4 (7.4)	9 (16.1)
Number of anxiety disorders per youth (M)	2.36	2.10

Notes: ST, student therapist; PT, professional therapist; SAD, separation anxiety disorder; GAD, generalized anxiety disorder; SoP, social phobia; PD, panic disorder; OCD, obsessive compulsive disorder; SP, specific phobia; AP, agoraphobia; CSR, clinician severity rating (ADIS C/P-IV)

4 enuresis

1 sleep terror disorder, 1 selective mutism, 1 enuresis

No significant differences in means between the groups were found for the SCAS or SCAS-P. Youth reported CALIS was significantly higher in the ST group (M = 15.10, SD = 8.09) compared to the PT group (M = 11.90, SD=7.35; t(108) = 2.17, p = 0.03, 95% CI [0.28, 6.12]). There was also a significant difference for fathers score on CALIS-P (self-reported interference of parents life) between the means of ST (M = 10.16, SD = 5.56) and PT (M = 7.85, SD = 6.09; t(102) = 2.00, p = 0.05, 95% CI [0.14, 9.39]) were the fathers ratings were higher for the ST group. No other significant differences were found for CALIS and CALIS-P (Table 2).

TABLE 2.

Continuous measure outcomes at pre- and post-treatment for student therapist and professional therapist groups, the time-by-group effect and ES

	Pre –M (SD)	Post –M (SD)	Time-by-group effect	Pre-post ES
ADIS CSR primary diagnosis
ST	6.33 (0.75)	2.85 (2.41)	F1,108 = 0.18, p = 0.72, ηp2 = 0.01	p < 0.001, g = 1.36
PT	6.09 (1.07)	2.16 (2.59)		p < 0.001, g = 1.51
ADIS CSR all diagnoses
ST	4.60 (1.32)	2.13 (1.36)	F1,108 = 0.52, p = 0.47, ηp2 = 0.01	p < 0.001, g = 1.82
PT	5.32 (0.95)	1.69 (1.53)		p < 0.001, g = 2.52
SCAS youth
ST	41.57 (16.93)	24.13 (14.36)	F1,108 = 0.80, p = 0.37, ηp2 = 0.00	p < 0.001, g = 1.08
PT	39.16 (18.06)	21.57 (14.42)		p < 0.001, g = 1.03
SCAS-P mother
ST	43.72 (14.98)	23.76 (11.90)	F1,108 = 1.94, p = 0.27 ηp2 = 0.02	p < 0.001, g = 1.41
PT[a]	38.91 (15.79)	22.25 (12.59)		p < 0.001, g = 1.12
SCAS-P father
ST	38.18 (15.33)	22.55 (12.57)	F1,102 = 0.00, p = 0.99, ηp2 = 0.00	p < 0.001, g = 1.07
PT	37.15 (13.80)	23.56 (13.87)		p < 0.001, g = 0.97
CALIS youth
ST	15.10 (8.09)	7.57 (6.40)	F1,108 = 1.87, p = 0.17, ηp2 = 0.01	p < 0.001, g = 0.99
PT	11.90 (7.35)	7.55 (6.46)		p < 0.001, g = 0.61
CALIS mother
ST	19.65 (6.80)	12.06 (7.28)	F1,108 = 1.89, p = 0.17, ηp2 = 0.01	p < 0.001, g = 1.06
PT	17.78 (7.20)	10.61 (7.28)		p < 0.001, g = 0.98
CALIS father
ST	18.51 (6.93)	11.90 (6.29)	F1,102 = 1.96, p = 0.17, ηp2 = 0.02	p < 0.001, g = 0.98
PT	16.05 (6.84)	10.96 (7.72)		p < 0.001, g = 0.68
CALIS-P mother
ST	13.13 (5.72)	7.26 (5.66)	F1,108 = 1.39, p = 0.24, ηp2 = 0.01	p < 0.001, g = 1.02
PT	11.07 (6.91)	6.82 (6.28)		p < 0.001, g = 0.63
CALIS-P father
ST	10.16 (5.65)	6.86 (4.63)	F1,102 = 3.17, p = 0.08, ηp2 = 0.03	p < 0.001, g = 0.62
PT	7.85 (6.09)	5.71 (5.46)		p < 0.001, g = 0.32

Notes. ST, student therapist; PT, professional therapist; ES, effect-size; ADIS, Anxiety Disorder Interview Schedule for DSM-IV; CSR, Clinician Severity Rating; SCAS, Spence Children’s Anxiety Scale; SCAS-P, Spence Children’s Anxiety Scale Parent version; CALIS, Child Anxiety Life Interference Scale; CALIS-P, Child Anxiety Life Interference Scale – Interference on parents’ life

Outlier excluded

To avoid the risk of creating biased estimates of the baseline data, no baseline adjustment or analysis controlling for these differences were conducted. The differences were taken into account in the interpretation and discussion of the results (40,41).

Pre- to post-treatment measure outcomes

Changes in CSR for the primary anxiety diagnosis presented significant and large ES for both the ST (g = 1.36, p < 0.01) and PT (g = 1.51, p < 0.01) group. The within group ES were also significant and large for changes in CSR for all anxiety diagnoses for the ST (g = 1.82) and (g = 2.52) PT group. Both groups also showed a significant reduction in all the self- and parent-reported symptoms of anxiety, with most ES ranging from medium to large (ES range: g = 0.61-2.52). One exception was the CALIS-P (fathers self-reported interference of parents life) in the PT group were the ES was small (g = 0.32), but still significant.

The pre- to post-time-by-group comparison of the CSR revealed no interaction effect between the two groups for the primary anxiety diagnosis (CSR primary: F 1,108 = 0.18, p = 0.72, η2 = 0.01), or for all anxiety diagnosis (CSR all: F 1,108 = 0.52, p = 0.47, = 0.01). Furthermore, the time-by-group comparison of youth and parent self-reported-ratings showed no interaction effect between the two groups, for any of the self-reported outcome measures (Table 2).

Treatment maintenance: three-month follow-up

The within group ES for changes in CSR for primary anxiety diagnosis remained significant from pre to FU in both treatment groups (ST: g = 1.88, PT: g = 1.81) group, both showing a higher ES than for pre to post. The CSR for all anxiety diagnosis also remained significant for both groups (ST: g = 2.51, PT: g = 3.27), and showed a higher ES in change at three-month follow-up than at post-treatment. All self-reported ES remained significant at three-month follow-up and there was an increase in the ES for most of the self-reported measures in both groups.

The pre to follow-up time-by-group comparison found no significant interaction effect between the two groups, regarding the CSR for primary anxiety diagnosis (CSR primary: F 1,108 = 1.53, p = 0.22, η2 = 0.01) or CSR for all anxiety diagnosis (CSR all: F 1,108 = 0.13, p = 0.72, = 0.00) (Table 3). No significant interaction effects were found for any of the self-reported anxiety measures (Table 3).

TABLE 3.

Continuous measure outcomes at pre- and FU-treatment for the student therapist and professional therapist groups, the time-by-group effect and ES

	Pre –M (SD)	FU - M (SD)	Time-by-group effect	Pre-FU ES
ADIS CSR primary diagnosis
ST	6.33 (0.75)	1.74 (2.31)	F1, 108 = 1.53, p = 0.22, ηp2 = 0.01	p < 0.01, g = 1.88
PT	6.09 (1.07)	1.59 (2.49)		p < 0.01, g = 1.81
ADIS CSR all diagnoses
ST	4.60 (1.32)	1.37 (1.22)	F1, 108 = 0.13, p = 0.72, ηp2 = 0.00	p < 0.01, g = 2.51
PT	5.32 (0.95)	1.28 (1.32)		p < 0.01, g = 3.27
SCAS youth
ST	41.57 (16.93)	20.91 (12.95)	F1, 108 = 1.47, p = 0.23, ηp2 = 0.01	p < 0.01, g = 1.30
PT	39.16 (18.06)	16.70 (13.23)		p < 0.01, g = 1.33
SCAS-P mother
ST	43.72 (14.98)	21.72 (11.53)	F1, 108 = 1.49, p = 0.22, ηp2 = 0.01	p < 0.01, g = 1.56
PT[a]	38.91 (15.79)	19.79 (13.61)		p < 0.01, g = 1.26
SCAS-P father
ST	38.18 (15.33)	18.70 (12.55)	F1, 102 = 0.00, p = 0.99, ηp2 = 0.00	p < 0.01, g = 1.34
PT	37.15 (13.80)	18.72 (10.63)		p < 0.01, g = 1.42
CALIS youth
ST	15.10 (8.09)	8.50 (8.21)	F1, 108 = 3.30, p = 0.07, ηp2 = 0.03	p < 0.01, g = 0.80
PT	11.90 (7.35)	5.79 (6.07)		p < 0.01, g = 0.88
CALIS mother
ST	19.65 (6.80)	10.13 (7.09)	F1, 108 = 1.70, p = 0.20, ηp2 = 0.02	p < 0.01, g = 1.35
PT	17.78 (7.20)	8.80 (7.75)		p < 0.01, g = 1.18
CALIS father
ST	18.51 (6.93)	8.83 (6.67)	F1, 102 = 0.41, p = 0.53, ηp2 = 0.00	p < 0.01, g = 1.40
PT	16.05 (6.84)	10.00 (7.85)		p < 0.01, g = 0.80
CALIS-P mother
ST	13.13 (5.72)	6.39 (5.12)	F1, 108 = 1.10, p = 0.30, ηp2 = 0.01	p < 0.01, g = 1.22
PT	11.07 (6.91)	5.68 (6.83)		p < 0.01, g = 0.77
CALIS-P father
ST	10.16 (5.65)	5.27 (5.10)	F1,102 = 2.25, p = 0.14, ηp2 = 0.02	p < 0.01, g = 0.89
PT	7.85 (6.09)	4.65 (4.77)		p < 0.01, g = 0.56

Notes. ST, student therapist; PT, professional therapist; ES, effect-size; ADIS, Anxiety Disorder Interview Schedule for DSM-IV; CSR, Clinician Severity Rating; SCAS, Spence Children’s Anxiety Scale; SCAS-P, Spence Children’s Anxiety Scale Parent version; CALIS,Child Anxiety Life Interference Scale; CALIS-P, Child Anxiety Life Interference Scale - interference on parents life

Outlier excluded

Only CALIS youth for the ST group showed a significantly lower ES at three-month follow-up (g = 0.81), when compared to the ES from post-treatment (g = 0.99) (Table 3).

Diagnostic status at post-treatment and at three-month follow-up

Post-treatment comparison revealed no significant difference regarding number of youths free of primary diagnosis between the ST (27 [50.0%]) and the PT (37 [66.1%]) group (χ2 = 2.92, p = 0.12). However, there was a significant difference regarding number of youths free of all anxiety disorders between the ST (14 [25.9%]) and PT (27 [48.2%]) group (χ2 = 5.84, p = 0.02) (Figure 1).

FIGURE 1

Mean scores on SCAS, SCAS-P and CSR primary diagnosis for both treatment groups, at pre-, post-treatment and 3-month follow-up

Note. CSR = Clinician Severity Rating, SCAS = Spence Children’s Anxiety Scale

At three-month follow-up, there was an increase in number of participants free of their primary diagnosis for both the ST (40 [74.1%]) and PT (43 [76.8%]) group, and there was no significant difference between groups in number of participants free of primary anxiety diagnosis (χ2 = 0.11, p = 0.83). There was also an increase in number of participants free of all anxiety diagnosis at three-month follow-up for both the ST (25 [46.3%]) and PT (33 [58.9%]) group, and there were no difference between the groups in number of participants free off all anxiety diagnosis at three-month follow-up (χ2 = 1.76, p = 0.25).

Reliable and clinical significant change

At post-treatment, a significantly larger proportion of youths in the ST group achieved RC but not CSC on the mothers ratings (SCAS-P Mother χ2 = 6.79, p = 0.01). There were no other significant differences between the two groups in the proportion of participants achieving RC or CSC.

At three-month follow-up, there was a significant larger proportion of the youths in the PT group who achieved RC but not CSC (SCAS Youth χ2 = 6.70, p = 0.02). Consequently there was also a significant larger proportion of the youths in the ST group who achieved CSC compared to the PT group (χ2 = 6.11, p = 0.02). These differences between the ST and PT group, on CSC, was also significant for the mothers responses (SCAS-P Mother χ2 = 4.66, p = 0.04). Again, there were no other significant differences between the groups achieving RC and CSC on SCAS or SCAS-P (Table 4).

TABLE 4.

Proportion of youths no longer meeting criteria for primary anxiety diagnosis and all anxiety diagnosis at post-treatment and follow-up

	ST group (n = 54)	PT group (n = 56)	Fischer’s
Post-treatment
Primary diagnosis	50.0% (27)	66.1% (37)	p = 0.122
All diagnosis	25.9% (14)	48.2% (27)	p = 0.019[*]
Follow up
Primary diagnosis	74.1% (40)	76.8% (43)	p = 0.826
All diagnosis	46.3% (25)	58.9% (33)	p = 0.252

Notes. ST, student therapist; PT, professional therapist

Significant at p < 0.05

Treatment satisfaction

According to answers on the ESQ, families in both the ST and PT group were highly satisfied with the treatment. For the statement ‘The treatment helped me/my child’ 43 (87.8%) youths, 44 (83.0%) mothers, and 40 (85.1%) fathers scored 2 (true) in the ST group, while 44 (83.0%) youths, 42 (77.8%) mothers, 41 (77.4%) in the PT group scored 2 (true). For the statement ‘I had confidence in my/our therapists’ 44 (89.8%) youths, 51 (96.2%) mothers, and 44 (93.6%) fathers scored 2 (true) in the ST group, while 46 (86.8%) youths, 53 (98.1%) mothers, and 48 (90.6%) fathers in the PT scored 2 (true).

Independent t-test revealed that there was no significant difference between the total satisfaction score in the ST and PT group, for youths (t(100) = —-0.92, p=0.362, 95% CI [−0.40, 1.09]) mothers (t(105) = −0.22, p = 0.834, 95% CI [−0.58, 0.73]) or fathers score (t(98) = −1.92, p = 0.058, 95% CI [−0.03, 1.78]).

Discussion

Anxiety disorders among youths are common, impairing, and with considerable personal and societal costs. There is need for effective treatments and enough therapists who can provide treatment for youths suffering from anxiety. The aim of this study was to evaluate the outcomes of a group-based CBT program (Cool Kids) treating youth anxiety disorders, conducted by psychology students in a university clinic, and compare these outcomes with those achieved by PT. By using multiple respondents, clinical assessment, standardized questionnaires, at three different time points, this study allows for a detailed study of the treatment effect of the ST treating youths with anxiety.

Findings from the current study indicate that psychology students with limited therapist training could successfully and effectively, provide group-based CBT to youths with anxiety disorders. The study showed that the treatment effects were statistically significant in clinician-, parent-, and self-reported measures of anxiety, after treatment and with an increased effect at three-month follow-up. Furthermore, the study showed that treatment outcomes in the ST were comparable to those achieved by psychologist in the PT group. Although the study did not find many significant differences between the two groups in regard to treatment outcome, there were some differences and tendencies. These findings could extend our current knowledge of student therapy, and provide possible guidance and recommendations for future treatment conducted by ST.

At baseline, there was a significant difference between the two groups in mean age, and in CSR for all diagnosis on the ADIS. The participants in the ST group sample was included based on availability with no age stratification, and the PT sample was derived from an RCT study with a higher mean age (24). There was also a difference between the two groups in mean CSR for all anxiety disorders, were the PT had a significantly higher mean compared to the ST group. These differences should be taken into consideration when interpreting the results, as there is a heterogeneity in the age of onset for different anxiety disorders, and that symptom severity have been related to more complex problems and predict less favorable treatment outcomes (42,43). The difference in age and CSR could therefore be disadvantageous for the PT group. No other significant pre-treatment differences in terms of gender, CSR for primary diagnosis, comorbidity, or type of primary diagnosis were found, indicating that the samples were relatively comparable. The main findings concurred with the initial hypothesis, showing that ST obtained changes in both clinician- and self-rated anxiety levels in youths, with large effect sizes. When compared to the PT, no significant differences between the groups were found regarding youths free of their primary anxiety diagnosis and reduction in anxiety levels on the clinical and self-reported outcome measures. There was a significant difference between the two groups relating to number of youths free of all anxiety diagnoses at post-treatment, in favor of the PT group. However, this difference was non-significant at the three-month follow-up assessment. These finding are in line with other studies investigating the effect of novice therapists conducting CBT treatment. In an RCT study, Masia Warner et al. showed that counsellors could treat social anxiety disorders among adolescents as effectively as specialized psychologists, using the Skills for Academic and Social Success program (SASS), specifically developed to treat social anxiety disorder in a school setting (44). Like their study, we found no significant differences in the remission of primary anxiety disorder among youth treated by student therapists or psychologists. We also showed that youths in the ST group showed significant improvements on other anxiety disorders as well. We were also pleased to see that the participants in the ST group were satisfied with the treatment they received. Both parents and youths stated that they were satisfied with the effect of treatment, and that they had confidence in the therapist. The satisfaction was also comparable to those achieved in the PT group.

Although few differences between the ST and PT group were found, there were some differences indicating a difference in treatment effect between the two groups. At post-treatment, there was a significant difference in number of youths free of all anxiety disorders, favoring the PT group. There was also a non-significant difference between the groups regarding number of youths free from their primary disorder, 10 more youths were free of their primary anxiety disorder in the PT group. There was thus a clear tendency for the PT to be more effective and achieving better remission rates faster than the ST. These differences were reduced at three-month follow-up. However, there was still a tendency for the PT condition to have higher remission rates. This difference could possibly be explained by a difference in treatment setting, where one professional therapist and three assisting graduate students treated the PT group, compared to the ST group where three student therapists conducted the treatment. Although the assisting graduate students in the PT group only had an assisting role, it is possible that the setting using a PT and three assistants allowed for more youth-therapist collaboration, thus making this setting more effective compared to the setting using three ST. Another explanation could be that the professional therapists were more effective in implementing the Cool Kids program than the ST due to prior clinical experience. This difference in setting and prior experience could give an advantage to the PT providing them with a greater capacity to focus on primary and other anxiety disorders, which in turn could lead to participants achieving greater reduction in all anxiety disorders at post-treatment, in the PT group.

Overall, our findings concur with previous studies finding of investigating the treatment effects of ST and other novice therapists treating anxiety disorders using CBT. The results from the present study are comparable to previous clinical effectiveness and efficacy trials of the Cool Kids program (13,24). The clinical rated pre- to post-treatment ES in the ST group (g = 1.36-1.82) were in range with the ES from a community sample (g = 1.06-1.44) from an effectiveness study of the Cool Kids (13). The ES were also larger than the ES from the waitlist group (g = 0.31-0.40) in a recent efficacy study (24). The remission rates for the primary anxiety disorder in the ST group at post-treatment are also comparable with the recovery rates reported in a recent meta-analysis of CBT treatments treating youths with anxiety (9).

In the current study graduate-level psychology students, received training in an EBT aimed at treating youth anxiety (20-h workshop and 20-h assisting treatment), and weekly group-based supervision (2-h). The study showed that following this education, training and supervision, the ST could effectively implement a manualized 10 session group-based CBT program, with statistically significant positive changes in clinical and self-reported symptoms of anxiety. The small differences in outcomes between the more experienced PT and the inexperienced ST may indicate that comprehensive previous clinical experience is not a necessity when conducting structured and manualized CBT, with sufficient training and supervision. As the PT showed a significantly greater effect compared to the ST in reducing anxiety symptoms across all anxiety diagnosis, novice therapists may benefit in the future from more training and supervision focusing on the treatment of comorbid anxiety disorders.

Limitations

The current study is not a RCT. Although the PT sample was derived from a RCT study (24), the ST sample was not randomly allocated and does not include a control group. Therefore, the possibility that the improvement in the ST group may be due to the effect of time alone cannot be ruled out. However, the pre- and post-treatment clinical rated ESs in the current study are within range of other intervention studies and larger than the ESs reported for other control groups (e.g. waitlist) in other efficacy studies of the Cool Kids program (33). The between groups effects were all very small and statistically insignificant. A post hoc power analysis found that the study was underpowered to detect significant effects between groups in the range of our findings. As there was a non-significant trend of larger ES and higher remission rates for the PT group for change in clinical rated measures, it is possible that larger samples could uncover significant differences between the two groups. The application of a last observation carried forward procedure could also affect the results of the analyses. However, the number of missing data was relatively small and the imputation method was not expected to have an impact on the results. No analysis on program adherence or therapist competencies were conducted in either of the treatment groups. Another possible limitation is the difference in clinical experience between the therapists in the PT group. However, there was no significant difference in treatment effect on the primary outcome between the two therapists in the PT group. The study lacks control of possible dependency in the data due to clustering of participants within therapy groups.

Clinical significance

As there is a need for more therapists providing effective anxiety treatments worldwide. There is also an implementation problem, getting EBT (e.g. Cool Kids), from the clinical researchers and out into community practices (10,12). There is no single solution to this problem. However, increased training and use of student therapist in clinical research and clinical therapy could contribute to a future solution. If clinical training using EBT was integrated in the education of clinical psychologists, this could possibly help bridge the gap between clinical research and clinical practice in the future.

TABLE 5.

Reliable and clinical significant changes at post-treatment and 3-month follow-up on SCAS, for intention to treat sample

SCAS Youth				SCAS-P Mother			SCAS-P Father
	ST (n = 54)	PT (n = 56)	Fisher’s	ST (n = 54)	PT (n = 56)	Fisher’s	ST (n = 51[a])	PT (n = 53[b])	Fisher’s
Pre-treatment
Over clinical cut-off	75.9% (41)	67.9% (38)	p = 0.40	90.7% (49)	85.7% (48)	p = 0.56	78.4% (40)	84.9% (45)	p = 0.45
Post-treatment
RC deterioration	0% (0)	0% (0)	-	0% (0)	0% (0)	-	0% (0)	1.9% (1)	p = 0.99
RC improvement	63.0% (34)	67.9% (38)	p = 0.69	81.5% (44)	69.6% (39)	p = 0.19	74.5% (38)	71.7% (38)	p = 0.83
RC but not CSC	18.5% (10)	19.6% (11)	p = 0.99	33.3% (18)	12.5% (7)	p = 0.01[*]	17.6% (9)	32.1% (17)	p = 0.11
CSC	44.4% (24)	48.2% (27)	p = 0.71	48.1% (26)	57.1% (32)	p = 0.45	56.9% (29)	41.5% (22)	p = 0.17
Three-month follow-up
RC deterioration	0% (0)	5.4% (3)	p = 0.24	5.6% (3)	10.7% (6)	p = 0.49	7.8% (4)	5.7% (3)	p = 0.71
RC improvement	63.0% (34)	50.0% (28)	p = 0.18	77.8% (42)	66.1% (37)	p = 0.21	66.7% (34)	56.6% (30)	p = 0.32
RC but not CSC	3.7% (2)	19.6% (11)	p = 0.02[*]	14.8% (8)	28.6% (16)	p = 0.11	21.6% (11)	15.1% (8)	p = 0.45
CSC	59.3% (32)	35.7% (20)	p = 0.02[*]	68.5% (37)	48.2% (27)	p = 0.04[*]	52.9% (27)	47.2% (25)	p = 0.70

Notes: ST, student therapist; PT, professional therapist; RC, reliable change; CSC, clinical significant change; SCAS, Spence Children’s Anxiety Scale; SCAS-P, Spence Children’s Anxiety Scale Parent version.

3 fathers did not participate;

3 fathers did not participate

Significant at p < 0.05