Case Management as a Significant Component of Usual Care Psychotherapy for Youth with Disruptive Behavior Problems.

Youth with disruptive behavior problems (DBPs) represent the majority of youth served in usual care (UC) psychotherapy, and are at high risk for maladaptive outcomes. Little is known about UC psychotherapeutic strategies utilized with this population. Researchers and clinicians suggest that case management (CM) is a major activity occurring in usual care. CM includes coordinating care with service providers and individuals, including schools, psychiatrists, and community-based services. This study assesses the prevalence and predictors of clinician use of CM in usual care. Results from this study suggest that CM is frequently used in UC psychotherapy with youth with DBPs. The extent of use of CM in UC may have implications for implementation of evidence-based practices in usual care psychotherapy.

Introduction

The majority of youths served in outpatient public mental health service systems are referred for disruptive behavior problems (DBPs; Garland et al. 2001; Weersing and Weisz 2002). These youths are at high risk for a variety of maladaptive adolescent and adult outcomes; DBPs are also challenging and costly for families and for society at large (Earls 1994). Improved knowledge about treatment for this patient population is therefore important. Research indicates that specific evidenced-based practices (EBPs) can have a significant impact on improving outcomes for youth with DBPs in randomized, controlled trials (Kazdin 2000; Waddell and Godderis 2005). EBPs such as Parent Management Training (PMT), anger management and social problem-solving interventions have been shown to be effective treatments for this population (Brestan and Eyberg 1998; Chambless et al. 1998; Hoagwood et al. 2001; Kazdin and Wassell 2000).

Despite the growing research base on efficacious interventions for youth with DBPs, little is known about how often or intensively psychotherapeutic strategies that are common in EBPs interventions for children with DBPs are being utilized by community-based clinicians (Garland et al. 2006a, b; Zima et al. 2005). There have been no large-scale studies identifying therapeutic strategies common in usual care (UC) psychotherapy practice for this population. A recent meta-analysis of 32 randomized trials that directly compared EBPs with UC found that usual care was not as effective as evidence-based treatments in treating youth with multiple diagnoses (Weisz et al. 2006). These researchers also found that descriptions of UC psychotherapy were so meager that they did not permit adequate characterization, concluding that an essential step in ultimately improving services is descriptive research about usual care. Our research group is conducting a large scale study characterizing UC psychotherapy for children with DBPs to meet the need for a greater understanding and improvement of usual care. An understanding of UC practice is a critical first step towards implementing evidence-based practices in community settings and identifying potential barriers to implementation and dissemination of evidence-based practices.

While one of the main goals of our research characterizing UC psychotherapy for children with DBPs is to examine the use of strategies common in EBPs for this population, we are also interested in examining other strategies that are highly relevant to clinicians’ work and may impact the implementation of EBPs. There are data to suggest that one of the major components of community-based treatment for youth is more general clinical case management (CM) (King 2006; McPherson et al. 2004; Ziguras and Stuart 2000). Research suggests that quality mental health care for children may include psychosocial interventions, familial involvement and appropriate linkage to other service sectors (Zima et al. 2005). Children receiving publicly-funded mental health services often require intensive services for extended periods of time because they have severe symptoms, low socioeconomic status, and are exposed to multiple psychosocial stressors (Zima et al. 2005). These stressors may include a history of physical or sexual abuse, living in out-of-home placements, parental mental illness, substance abuse, poverty, and/or a history of CPS involvement, which may all be partially addressed through case management. Research has shown that children with emotional and behavioral problems tend to miss school because of suspensions, fear of failure, or low self-esteem (Blanchard et al. 2006). Children with emotional, behavioral and academic problems are more likely to improve when supportive relationships among the family, school, and community are facilitated and integrated into treatment (Burns et al. 2000).

Coordination of this type of “extra-therapeutic” care may be particularly relevant for children with DBPs, as research suggests that these children have lower rates of participation in familial and social activities, which may be important as individual interventions in improving overall development and quality of life (Blanchard et al. 2006). It follows that treating clinicians will need to coordinate care with teachers and other health and social service professionals to varying degrees depending on the child’s functioning in different environments. Anecdotally, our clinician partners have identified CM as a critical and significant component of care, reporting that they provide a great deal of CM for most families. They also suggested that the intensity of CM is associated with child/family characteristics and therapist training and discipline. UC clinicians emphasize the importance of the child’s family, school, social and community context, suggesting that coordination of care across multiple contextual layers is critical. Research further supports the importance of integrating case management into treatment. Multisystemic Therapy (MST), which targets not only the child, but his/her environment, including family, peers, school and neighborhood, has a strong evidence base (Henggeler et al. 1998, 2002; Littell 2005). In MST, care is coordinated across multiple sectors, as case management is considered an essential aspect of treatment.

Case management is operationally defined in this study as coordinating care with psychiatrists, school personnel, and other service providers: facilitating respite care for caregivers; attending and reviewing individualized education programs (IEP), and making referrals to other home and community-based services for the family (Rapp 1998). This includes home- or community-based programs which provide parent training, social skills training, or other psychosocial interventions. For example, clinicians may involve Child Protective Services (CPS) when a child’s safety is at risk. They may also communicate with school personnel to coordinate services and consistently implement behavioral plans. Further, clinicians may refer children and families to community-based programs to foster social skills, encourage interaction with peers, and help give structure to their everyday lives (McGrew et al. 2003).

Given that CM may be particularly relevant in treating the complex needs of youth with DBPs served in UC outpatient psychotherapy services, and anecdotal reports from clinicians suggest that coordination of care is crucial, it is important to characterize how often and intensively CM is used in community-based psychotherapy and what predicts clinician use of this element of treatment. As such, our goal is to define and measure the prevalence of, and assess the predictors of, case management in UC as a first step towards examining the contribution of CM to treatment outcomes. Specifically, this study examines how often and intensively community-based clinicians at six outpatient clinics in a large, urban area use CM strategies to coordinate extra-therapeutic care for children with DBPs, and the youth and clinician factors that predict the use of this strategy. These potential factors include child symptom severity, clinician discipline, clinician theoretical orientation and demographic variables. We specifically hypothesize that youth with high symptom severity will be associated with more intensive coordination of care, given that their functioning is likely impaired across multiple domains. We further hypothesize that social workers will use CM more intensively than therapists trained in non-social work disciplines, such as psychology or marriage and family therapy, since social work training focuses on coordination of care across multiple service sectors and knowledge about various community-based resources.

Method

This study represents a preliminary sub-study of a larger project designed to characterize treatment processes and outcomes in UC psychotherapy for children with DBPs and their families, and to examine the extent to which UC psychotherapy is consistent and inconsistent with common elements of EBPs for this patient population (Practice and Research: Advancing Collaboration, PRAC; Garland et al. 2006a, b). To identify and operationalize common elements of evidence-based principles for youth with DBPs, the PRAC research team reviewed several comprehensive reviews of efficacious treatments for children with DBPs (Brestan and Eyberg 1998; Burns et al. 1999; Garland et al. 2008; Kazdin and Weisz 1998). Investigators consulted with a “Therapist Advisory Group” (TAG) to generate additional common elements of UC psychotherapy for children with DBPs. Case management was identified as an important strategy via this collaborative process, and was added to the observational coding system Therapy Process Observational Coding System—Strategies (TPOCS-S) used to characterize UC practice in this study. This investigation focuses specifically on this practitioner-identified element of treatment.

Participants

Family Participants

All new patients entering the six participating clinics who granted permission to be contacted by research staff through an initial screen at the clinic (according to HIPAA regulations) were recruited consecutively by the research staff. Participants were recruited into the study if they were beginning a new episode (defined as no treatment in the previous 3 months) of outpatient psychotherapy with a participating psychotherapist and if they met the following eligibility criteria as determined in the baseline telephone screening interview with research staff: (a) disruptive behavior problem was one of the presenting problems for treatment (including aggression, defiance, delinquency, oppositional behavior), (b) child age between 4–13 years, (c) primary language for child and caregiver was English or Spanish. Family participants were given financial incentives to participate in the study ($40 to the caregiver and $10 to the youth at the baseline interview), but were assured that their decision regarding participation would not impact treatment. There were no inclusionary criteria based on diagnosis. Not limiting participants based on diagnosis allowed us to include subjects representing a wide range of the patient population with a variety of symptom severity scores and comorbidities. Youth were excluded if they had documented mental retardation (IQ < 70), significant organic brain damage or major medical problems, as they may have required specific adaptations of psychotherapy and thus, their treatment would not be representative of UC practice with youths with DBPs.

The participants of this sub-study were 120 children ages 4–13 (M = 8.82, SD = 2.58) with DBPs receiving publicly funded outpatient mental health services in San Diego, California. Seventy-two percent of the sample was male (n = 86). Forty-seven percent were Caucasian (n = 57), 29% Latino (n = 35), 5% African–American (n = 6), and 18% Mixed/Other (n = 22). This was a partial sample of those for whom data were available at the time of these analyzes. See Table 1 for a summary of child characteristics.

Table 1

Characteristics of participating children (n = 120)

Child characteristics	n	M (SD) or %	Range
Child age at baseline	120	8.8 (2.5)	4–13
Child gender
Male	86	71.7%
Female	34	28.3%
Child race/ethnicity
Caucasian	57	47.5%
Latino/Hispanic	35	29.2%
Multiracial	22	15.8%
African American	5	5.0%
Native American	3	2.5%
Asian American/Pacific Islander	0	0%

Clinician Participants

The sample included 58 clinicians practicing in six community-based clinics in San Diego. Eighty-four percent (n = 49) of this sample was female, with a mean of 3 years of psychotherapy experience (SD = 1.94; range 0–15 years). Sixty percent of the clinician sample was Caucasian (n = 35), 7% Latino (n = 4), 3% African American (n = 2) and 16% Mixed/Other (n = 10). Multiple primary therapeutic disciplines were represented; 60% were Marriage and Family Therapists (MFT; n = 35), 26% Psychologists (n = 15), and 14% Social Workers (n = 8). Multiple theoretical orientations were also represented; when therapists were asked to identify their predominant theoretical orientation, 35% identified with Family Systems (n = 20), 26% Cognitive Behavioral (n = 15), 21% Eclectic (n = 12), 7% Psychodynamic (n = 4), 4% Behavioral (n = 2), 4% Humanistic (n = 2) and 5% Other (n = 3). Sixty percent of the clinicians were trainees (n = 35), while 40% were staff (n = 23). See Table 2 for a summary of clinician characteristics.

Table 2

Characteristics of participating therapists (n = 58)

Therapist characteristics	n	M (SD) or %
Gender
Female	49	84.5%
Male	9	15.5%
Years practiced		2.9 (1.9)
Race/ethnicity
Caucasian	35	60.3%
Asian American/Pacific Islander	7	12.1%
Multiracial	6	10.3%
Hispanic/Latino	4	6.9%
African American	2	3.4%
Other	3	5.2%
Filipino/a American	1	1.7%
Mental health discipline
Marriage, family counseling	35	60.3%
Psychology	15	25.9%
Social work	8	13.8%
Primary theoretical orientation
Family systems	20	34.5%
Cognitive behavioral	15	25.9%
Eclectic/integrated	12	20.7%
Psychodynamic/psychoanalytic	4	6.9%
Other	3	5.2%
Behavioral	2	3.4%
Humanistic/client centered	2	3.4%
Status	23	39.7%
Trainee	35	60.3%
Staff	23	39.7%

Measures

Information about child/family demographics and symptom severity were collected during the in-person baseline interview with youth and caregivers. The baseline assessment took approximately 60 min for the caregiver and 30 min for children over age 8. Data was not collected directly from children younger than age 9, but was instead collected from the caregiver. Clinicians completed a self-report questionnaire at baseline to assess age, gender, race/ethnicity, level of experience (staff vs. trainee), primary theoretical orientation and discipline.

The Eyberg Child Behavior Problems Checklist (ECBI; Eyberg and Ross 1978) was used to measure symptom severity at baseline in order to relate child clinical characteristics at baseline to clinician use of CM. The ECBI is a parent report measure consisting of 36-items designed to assess youth behavior problems (Eyberg and Ross 1978). This measure has two components: the Intensity score, which examines the frequency of behaviors on a 7-point Likert scale ranging from 1 (“never”) to 7 (“always”), and the Total Problem score, which indicates whether or not a behavior is a problem on a dichotomous “yes” or “no” scale. The psychometric characteristics of the EBCI are strong. Test-retest reliability coefficients for the Intensity and Problem scales were .86 and .88, respectively (Robinson et al. 1980). Internal consistency for the Intensity scale and the Problem scale is high (Eyberg and Robinson 1983; Robinson et al. 1980). Because the two ECBI scales were multicollinear (r = .738, p < .001), only the Parent Intensity scale was used. The Intensity scale was selected because it allowed us to test our a-priori hypothesis that greater symptom severity would be associated with more intensive use of CM.

Procedures

Treatment Process Measures

The TPOCS-S for child psychotherapy is a comprehensive coding system developed to characterize psychotherapy for youth (McLeod and Weisz 2005). The TPOCS-S was adapted specifically for the PRAC study, and is designed to assess a wide variety of intervention strategies. The TPOCS-S includes ratings of the occurrence (frequency) and intensity (thoroughness) of clinician use of 27 different types of intervention strategies. The therapy procedures checklist (TPC; Weersing et al. 2002) was a primary source for TPOCS-S content. The PRAC research team further collaborated with the Therapist Advisory Group (TAG), which included one clinician from each of the six clinics participating in this study (Garland et al. 2006a, b). The TAG reviewed the TPOCS-S and suggested additions of therapeutic strategies that were not previously included. Specifically, practitioners reported that they spend a lot of time in session with this patient population using CM and coordinating “extra-therapeutic” care. Thus, a “coordination of external care” code, reflecting CM, was subsequently added to the TPOCS-S. Importantly, the TAG reported that although a great deal of CM activities occur outside of therapy sessions (e.g., phone consultation, attending an IEP), observed, in-session CM is an adequate proxy for how much total CM is used with individual patients. This is because clinicians typically discuss these CM activities in-session with the child and/or caregiver (e.g., reviewing what occurred during the IEP).

While the TPOCS-S includes 27 individual psychotherapeutic strategies, this paper focuses exclusively on the code capturing CM (“Coordinating Child’s External Care”). This strategy is defined as in-session discussions relating to coordinating care with school staff, case managers, or referrals to or consultations with physicians/psychiatrists, social workers, clinicians, probation officers, etc. CM also includes discussions about community care and ancillary services such as Child Protective Services (CPS) and Individualized Education Program (IEP) meetings. Coordination of care also includes disposition planning, such as arranging referrals to other treatment settings and providers, and discussions about structured extra-curricular or community-based activities (e.g., after school programs). As with all TPOCS-S codes, both the occurrence and intensity of CM was rated. Occurrence indicates whether it was observed during a session, regardless of how intensively it was employed. Intensity reflects both the time spent on CM and the thoroughness with which it was pursued. Intensity was rated on a Likert scale of 0 (not used) to 6 (high intensity). Intensity scores of 1–2 = low, 3–4 = medium, and 5–6 = high intensity. A high intensity score was assigned if the coder observed the clinician spending considerable time reviewing care coordination, needs for other services, obstacles to receiving necessary services, or identifying community-based programs. For example, if a therapist simply mentioned to the caregiver that he/she spoke with the child’s teacher, this would be a low intensity example of therapist use of CM, and would be scored as a 1–2. If, however, the therapist went into great detail with the parent about this meeting, spending a significant amount of time providing psychoeducation as well as references to other community-based services such as after-school programs, this would be a more extensive example of therapist use of CM and would be scored as a 5–6. Thus, intensity scores depend both on the length of time the therapist spends on the topic of CM as well as the breadth of content. The presence (occurrence) and intensity of CM use were coded separately for youth and caregivers. That is, separate ratings were assigned to each potential “target” of CM.

In order to maximize the number of behavioral observations of CM, all sessions in which this strategy was observed were included in the analyzes, irrespective of whether the target was the caregiver or the child. In the case that CM was delivered to both the child and to the caregiver, the highest intensity rating was used for analysis.

Reliability

Forty-two percent (n = 153) of the total 364 sessions were randomly selected to be double-coded to test inter–rater reliability. Inter–rater reliability was assessed separately for child and for caregiver targets. The mean ICC for CM directed to caregivers = .80 and Kappa = .69. When CM was directed to children mean ICC = .74 and Kappa = .49. These reliability estimates reflect adequate to strong inter–rater reliability (Cicchetti 1994; Landis and Koch 1977).

Psychotherapy Process Data Collection

To gather observational data on clinician use of CM in treatment, all sessions were videotaped up to 16 months, or whenever the child stopped attending therapy. Consent for participation in the study included consent by the clinician, caregiver and child to videotape sessions. Videotaped sessions were randomly selected and assigned for coding according to the following schedule: 4 sessions for the first 4 months; 3 sessions for moths 5–8; 2 sessions for months 9–12, and 1 session for months 13–16 to gather the most data during the time frames in which most participants were still in treatment. The project coordinator set up video equipment prior to treatment and collected videotapes from clinicians. Coding was conducted by 10 trained coders who were all research assistants for this study (see PRAC Coder Training). None of the coders were involved in therapy with any of the PRAC participants at any time during the study. Coders entered data directly onto the computer while viewing sessions using a customized, Microsoft Access software program developed by one of the co-investigators.

PRAC Coder Training on TPOCS-S

Coder training was a multi-step process which involved four group sessions and six individual training sessions. Group training sessions included an overview of coding procedures and reviewing each individual code. Coders learned to specifically identify when clinicians were using CM. Coders learned how to use the computer coding program, coded sample sessions together as a group, and reviewed intensity ratings. Coders were considered “trained” when they reached 80% agreement (within one point of intensity ratings) with “gold standard” codes on at least three consecutive practice sessions.

Analyzes

Multilevel modeling was employed to examine the associations between child and therapists characteristics with average intensity of CM. Multilevel modeling was chosen over procedures that use ordinary least squares estimations because this technique provides better parameter estimates when data are nested (Roesch et al. 2009). MPlus software was used to account for the nested structure of the data, wherein sessions (n = 364) are nested within children (n = 120) are nested within clinicians (n = 58) are nested within clinics (n = 6). Primary analyzes were conducted on six-two-level models: sessions (level 1) nested within children (level 2), sessions (level 1) within clinician (level 2), sessions (level 1) within clinic (level 2), child (level 1) within clinician (level 2), child (level 1) within clinic (level 2), and clinician (level 1) within clinic (level 2). Because of the nested structure of the data, average cluster sizes were calculated to examine the data at each level.

Multilevel modeling was used to determine at which levels—session, child, clinician and/or clinic—a significant amount of variation was accounted for, indicating that variables at this level could account for differences in clinician use of CM. After identifying the levels at which differences in the use of CM were statistically significant, bivariate correlations, t-tests and ANOVAs were used to determine which level 2 predictors were significant at the child and clinician levels. Predictors that were significant at the bivariate level were then entered into MPlus to test multivariate models.

Results

Occurrence and Intensity of CM

Overall, CM was observed in 71.4% (n = 260) of all 364 therapy sessions directed to either caregivers or to children. In those 260 sessions in which CM was observed, the average intensity was 2.50 on a scale of 1–6 (SD = 1.43). In 28.5% of the sessions (n = 104), the intensity rating was moderate at 3 or higher. CM directed to the caregiver, specifically, was observed in 72.4% of the sessions in which a caregiver was present, with an average intensity of 2.83 (SD = 1.21). This was one of the most frequently coded items directed to the caregiver out of the 27 TPOCS-S codes. CM directed to the child was observed in 44.9% of sessions, with an average intensity rating of 2.02 (SD = 1.44).

Multilevel Modeling

Average cluster sizes are presented in Table 3. There were an average of 3.03 sessions coded per child (range 1–8), 6.27 sessions per clinician (range 1–26), and 60.77 sessions per clinic (range 14–128). There were an average of 2.06 children per clinician (range 1–7), and an average of 20 children per clinic (range 4–48). There were an average of 9.67 clinicians per clinic (range 2–22).

Table 3

Average cluster sizes

Level 2	Level 1
	Session (n = 364)	Child (n = 120)	Clinician (n = 58)
Child	3.03 (range 1–8)
Clinician	6.27 (range 1–26)	2.06 (range 1–7)
Clinic	60.77 (range 14–128)	20 (range 4–48)	9.67 (range 2–22)

The intraclass correlation coefficient (ICC) was generated using the null or the intercept-only model. The ICC provides an indication of variance for the outcome variable, intensity of use of CM, at both levels of the data structure. An ICC greater than .05 (or 5% of the variance) was used as a cutoff to warrant inclusion of the higher level (level 2) variable in each separate analysis. Design effects (DE) were also calculated, which take into account both the ICC and the cluster size. DEs indicate whether the variance for clustered or nested data is comparable to that of a simple random sample. Intraclass correlation coefficients are summarized in Table 4.

Table 4

Intraclass correlations for average therapist use of CM at session, child and clinician levels

Level 2	Level 1
	Session (n = 364)	Child (n = 120)	Clinician (n = 58)
Child	.162
Clinician	.103	.280
Clinic	.006	.011	.001

The ICC calculated for the nested data structure where sessions (level 1) are nested within children (level 2) was .162 (DE = 1.33), indicating that that 16.2% of the variance in intensity of therapist use of CM was due to differences among children. The ICC calculated for sessions (level 1) nested within clinicians (level 2) was .103 (DE = 1.54), indicating that 10.3% of the variance in intensity of use of CM across therapy sessions was accounted for by differences among clinicians. The ICC calculated for children (level 1) nested within clinicians (level 2) was .280 (DE = 1.30), indicating that 28% of the variance in intensity of CM across children was accounted for by differences among clinicians.

The ICC calculated for clinicians (level 1) nested within clinics (level 2) was not significant (ICC = .001, DE = 1.01), suggesting that when CM intensity scores were averaged across clinicians, clinics did not account for a significant amount of variance. The ICCs for sessions (level 1) nested within clinic (level 2) (ICC = .006, DE = 1.36) and children (level 1) nested within clinic (level 2) (ICC = .011, DE = 1.21) were also not significant, suggesting that there was no difference in the use of CM due to variation among clinics. As such, clinic predictors were not considered for further analyzes. The fact that all design effects were close to one further strengthens the reliability of this sample, as it suggests that the variance for the clustered or nested data is roughly the same as if the sample was drawn as a simple random sample.

Identifying Potential Predictors

We used bivariate correlations, t-tests and ANOVAs to identify potential predictors of clinician use of CM. Potential predictors categorized as “child characteristics” were child gender, age at baseline, symptom severity (ECBI intensity score) and child race/ethnicity. Predictors categorized as “clinician characteristics” were clinician gender, race/ethnicity, level of experience (staff vs. trainee), discipline (Social Worker, Marriage and Family Therapist (MFT), Psychologist), and self-reported theoretical orientation (Family Systems, Cognitive Behavioral, Eclectic, Psychodynamic, Behavioral Humanistic and Other). Based on our a-priori hypothesis that social workers would use CM more intensively than clinicians from any other discipline, we created the additional variable “social workers (SW) versus others.” These analyzes are summarized in Table 5.

Table 5

Associations between child- and clinician-level predictors and average therapist use of CM

Variable	Level 1 = SessionLevel 2 = Child		Level 1 = ChildLevel 2 = Clinician		Level 1 = SessionLevel 2 = Clinician
	Test statistic	p-value	Test statistic	p-value	Test statistic	p-value
Child
Gender (male = 0)	t = 2.56	p = .011	t = 3.45	p = .001	NA	NA
Sx severity	r = .163	p = .010	r = .201	p = .026	NA	NA
Race/ethnicity	–	–	–	–	NA	NA
Clinician
Gender (male = 0)	NA	NA	–	–	–	–
Race/ethnicity	NA	NA	–	–	–	–
Discipline (SW vs. Others)	NA	NA	–	–	t = 2.38	p = .018
Theoretical orientation	NA	NA	–	–	–	–
Level of training: staff versus trainee	NA	NA	–	–	–	–

Note: Therapist use of CM was aggregated at each level of the data structure. Associations that were not significant are indicated by a dash. Analyzes that were not conducted due to the nested structure of the data are indicated by the notation NA (not applicable)

Child Characteristics

Results indicated that when sessions (level 1) were nested within children (level 2), child gender was significantly associated with use of CM. Specifically, males were observed receiving more intensive CM (M = 1.96, SD = 1.67) than females (M = 1.45, SD = 1.56; t(362) = 2.56, p = .011). Greater child symptom severity, as measured by the baseline ECBI intensity score, was positively and significantly correlated with CM use (r = .163, p < .01). The more severe a child’s symptoms, the more intensively clinicians used CM. When children (level 1) were nested within clinicians (level 2), being male was associated with more intensive use of CM than being female (M = 2.18, SD = 1.46 vs. M = 1.22, SD = 1.15; t(118) = 3.45, p < .001), as was greater symptom severity (r = .20, p = .026).

Clinician Characteristics

When sessions (level 1) were nested within clinicians (level 2), there was a significant association between clinician discipline and how intensively clinicians used CM. Specifically, social workers used CM more intensively than psychologists and MFTs combined [t(362) = −2.38, p = .018].

Multivariate Analyzes

Variables that were significantly associated with therapist use of CM in the previous step were entered into two-level models to test their relationship to each other and to the dependent variable, intensity of use of CM. MPlus was used for the multilevel model testing to adjust for the influence of factors at both levels of the model. Predictors with z-scores >1.96 or <−1.96 were considered significant at p < .05 and were included in the model. When sessions were nested within children, child gender and symptom severity were significantly and uniquely associated with how intensively therapists used CM. More intensive CM was observed with males compared to females (z = 2.76, p < .05). Greater symptom severity also predicted more intensive use of CM (z = 3.42, p < .05). When sessions were nested within clinicians, clinician discipline (social work vs. others) was also significantly and uniquely associated with intensity of CM; being a social worker predicted more intensive use of CM than being a psychologist or MFT (z = 2.42, p < .05). Thus, both clinician and child characteristics predicted differences in how intensively clinicians used CM. Significant results of these multivariate analyzes are summarized in Table 6.

Table 6

Multilevel models significantly associated with average therapist use of CM

Variable	Level 1 = SessionLevel 2 = Child		Level 1 = ChildLevel 2 = Clinician		Level 1 = SessionLevel 2 = Clinician
	Test statistic	p-value	Test statistic	p-value	Test statistic	p-value
Child gender (males = 0)	z = 2.76	p < .05	z = 2.76	p < .05	NA	NA
Child sx severity	z = 3.42	p < .05	–	–	NA	NA
Clinician discipline (SW vs. Others)	NA	NA	–	–	z = 2.42	p < .05

MPlus could not estimate the standard errors of the children nested within clinician model because of low sample size at both levels, so multiple regression was used to analyze the model. The model summary suggested that the model was a good fit (R² = .137, F(2, 117) = 9.28, p < .001). The standardized regression coefficients revealed that gender and symptom severity both significantly and uniquely contributed to model fit (β = .309, p < .001 and β = .214, p < .01, respectively). According to Cohen (1988), the effect size was medium for gender and small to medium for symptom severity. Being male and having higher caregiver-reported symptom severity intensity scores independently predicted higher use of CM when both predictors were included in this model.

Discussion

Our findings suggest that case management strategies are not only used in usual care psychotherapy with children, but are highly prevalent, observed in 71% of sessions. In fact, CM was one of the most frequently observed of the 27 therapeutic strategies measured by the TPOCS-S system. CM includes coordinating care with psychiatrists and teachers, making referrals to community-based resources, including after-school programs and respite care. These CM strategies, on average, were observed at low to moderate intensity.

Research suggests that CM is particularly important for children with serious behavioral and emotional problems (Neill 2006). Specifically, our clinician partners indicate that a significant part of usual care is CM, which involves communicating and planning with people and agencies to attain positive outcomes for both child and family. Case management is an element of care that is not necessarily emphasized in many evidence-based practices. Although there are overlapping elements of psychotherapeutic interventions for children with DBPs, including youth skill-building and parenting skills, it appears that the more highly structured, empirically supported treatments may not explicitly emphasize the CM aspects of treatment. Concurrently, it is possible that the amount of time clinicians spend coordinating care for, and with, these families makes it challenging to implement structured EBP protocols. It should be noted, however, that this study is not designed to address the extent to which the emphasis on CM is essential to effective care; that is, we cannot determine if the high proportion of therapy time devoted to CM is absolutely necessary or if it is a “default” approach when therapists perceive a lack of alternative approaches.

We predicted a-priori that practitioners trained in the social work discipline would use CM more intensively than clinicians trained as either psychologists or MFTs. This hypothesis was supported by our results, which suggest that clinician discipline is indeed a predictor of how intensively clinicians use CM in outpatient psychotherapy with children with DBPs. This hypothesis is supported by existing literature, which suggests that CM has deep roots in social work, relying on interagency collaboration to marshal a range of resources within the community to benefit the child and the family, and is considered a major component of mainstream social work practice (Neill 2006). The role of social workers is often that of case manager, in that they integrate formal systems of care into the activities of families and other primary and community groups (Moore 1990). Accessing multiple services is emphasized in MSW training programs, such as utilizing interdisciplinary teams which include psychiatrists, nurses, social workers and counselors (Rapp 1998).

We also predicted a-priori that the severity of a child’s emotional and behavioral problems (as indicated by elevated ECBI scores) would lead to greater use of CM, due to the increased likelihood that the clinician will need to interact with other sectors of care. This was supported by our results, which suggest that children with greater symptom severity at baseline received more intensive coordination of care than those with lower baseline ECBI scores. Because severe symptoms may affect multiple aspects of a child’s world, including social and academic functioning, it is likely that treatment will not focus solely on the child, but also on the school and the larger community in order to maximize treatment effectiveness.

The final significant predictor of clinician use of CM was child gender. Males were observed receiving significantly more intensive CM than females. This may be because males in general tend to manifest their symptoms externally, while females show a tendency towards internalization (Fergusson et al. 1996; Hartung et al. 2002). This pattern is consistent with current developmental models of DBPs (Lahey et al. 2000). Males may be more aggressive in session and may engage in more aggressive acts out of session than females, which may compel clinicians to draw on other systems of care to supplement outpatient psychotherapy. An alternative or additional explanation is that we may see less intensive use of CM with females because they are more likely than males to openly discuss relationships and emotions, which are the focus of more “traditional” psychotherapeutic interventions. If females are indeed more engaged in “typical” psychotherapy than their male counterparts, clinicians may not spend as much time on CM. Further research is needed to determine whether these hypotheses hold merit. Notably, neither clinician orientation nor level of experience (staff vs. trainee) was significantly related to how intensively clinicians used CM. At the child level, demographic variables such as race/ethnicity and child’s age were also not significant.

Strengths and Limitations

There have been numerous calls to open the “black box” of UC services in order to better understand psychotherapeutic processes (Bickman 2000; Weisz et al. 2006). This is the first known study to rigorously examine psychotherapy treatment processes—specifically, the use of case management—with a broad, representative sample of youth receiving UC psychotherapy. The representativeness of this sample is supported by a recent national survey of 1,200 clinicians from 100 clinics across the United States (Glisson et al. 2008). In the national sample, 76% of clinicians were female and 71% were Caucasian, compared to 84% and 68%, respectively, in this study sample. Slightly more than half of the therapists in this current study were trainees, a finding supported by another sample of 42 therapists in four community-based clinics in Los Angeles, where 49% of therapists were trainees (Hawley and Weisz 2005). Our clinician sample is therefore relatively comparable to national and local samples on these basic demographic and experience characteristics.

Additionally, our methods were developed in collaboration with community-based clinician partners, which enabled us to examine elements of EBPs used in UC psychotherapy as well as other elements of care. The assessment of CM was specifically driven by clinician feedback and interest. In addition, our sample of clinicians and patients is generally representative of the publicly-funded, mental health sector in our county, as opposed to a highly selective convenience sample. Inter–rater reliability is adequate, lending support to the rigor of our methods. Furthermore, this study indicates that, with good partnership, both clinicians and families are willing to participate in research that is potentially intrusive in order to learn more about psychotherapy. Documentation of the prevalent use of CM in UC psychotherapy for children with DBPs has important implications, and informs our understanding of potential barriers to the implementation of EBPs in community settings.

One limitation of this study is the fact that there is no standard, operational definition of case management. This is not a well-researched topic. As researchers in different fields and disciplines define CM in various ways, it is a difficult construct to measure. However, because we defined CM based on input from clinicians who work in the field, we feel that this is an appropriately defined construct in the context of this study. The community-based clinicians who participated in this study reported that observed occurrences of CM in the therapy session are a satisfactory proxy for how much CM they use both in and out of session; we do recommend, however, that CM be clarified operationally and studied in greater depth.

We must consider also the possibility that other variables that predict therapist use of CM were not represented in this model. Findings at the child level might have been significant if other variables—such as caregivers’ physical and emotional health, socio-economic status, and history of mental illness—were included. Furthermore, we are unable to state how well our results generalize to other types of care, such as private insurance-funded care or care in other geographic locations. For example, there was a high representation of trainees and MFTs in our clinician sample, which may not be representative of privately-funded care. However, this may also be perceived as strength of the study, as it is representative of UC. In addition, we do not know how much the videotaping process affected practice itself, although we tried to minimize the impact of this by using small, unobtrusive cameras, and encouraged clinicians to record all sessions to establish a routine. Clinicians did report that both they themselves and their patients quickly acclimated to being videotaped, and did not feel that this was an obstacle to providing treatment as usual (TAG, personal communication). Additionally, although clinic differences did not account for a significant amount of variance, it is possible that the climate and culture of the clinics did impact clinicians’ use of CM, and that this was not captured due to the small sample size at the clinic level (n = 6).

Importantly, this study serves as a base from which other studies can be conducted regarding the role and importance of CM in the treatment of youths with DBPs. The next step in this process is to examine how treatment outcomes may be related to the use of CM, and whether or not more intensive use of this treatment strategy leads to clinical and functional improvement. Isolating elements of treatment that are commonly used in UC psychotherapy will help us understand what aspects of treatment are most effective, and can ultimately serve to facilitate more effective dissemination and implementation of EBPs by balancing the push for evidence-based practice with practice-based evidence. This study serves as an example of how researchers and practitioners can work together to facilitate increased knowledge about treatment as usual. It also suggests that, with good partnership, both youth patients and their caregivers are willing to be active participants in studies of psychotherapy to help characterize and ultimately improve usual care. As such, it is our contention that the gap between research and practice can be bridged via shared knowledge through communication and collaboration.