Development and evaluation of diversity-oriented competence training for the treatment of depressive disorders.

Studies in Europe indicate that some ethnic minorities have higher rates of mental disorders and less favorable treatment outcomes than their counterparts from majority groups. To date, efforts regarding training to reduce disparities have mainly focused on ethnocultural competences of therapists, with less attention paid to other aspects of diversity, such as sex/gender and socioeconomic status. In this study, we aim to determine the effectiveness of a population-specific, diversity-oriented competence training designed to increase therapists' competencies to integrate aspects of diversity features in clinical assessment, diagnosis, and treatment of depressive disorders in Turkish- and Moroccan-Dutch patients. A group of 40 therapists were location-based assigned to either training or a control condition (no training). Self-reported diversity competence, a knowledge test, and therapists' satisfaction with training were used to monitor the training and to measure competence levels at baseline, post-training, and three-month follow-up. Attitude-awareness and knowledge components of the self-reported diversity competence and test-measured knowledge increased in the training condition. Most gains remained stable at follow-up except test-measured knowledge after controlling for percentage of ethnic minority patients in caseload. There were no changes regarding therapists' self-reported skills. Therapists expressed medium-high satisfaction with the training, acknowledging the relevance of diversity competence for their daily practice. Future training must ensure better adjustment to therapists' pre-existing knowledge and be followed by long-term efforts to maintain competence levels and enhance competence transfer within teams.

Introduction

Recent cross-national research in Europe has drawn attention to the struggle of mental healthcare professionals (herein referred as therapists) to deliver high quality services to immigrant populations (Priebe et al., 2011). One-size-fits-all approaches, language barriers, socioeconomic constraints, ethnocultural differences, different illness beliefs and staff’s lack of awareness and experience have been commonly reported difficulties (Lindert, Schouler-Ocak, Heinz, & Priebe, 2008; Lindert et al., 2008; Priebe et al., 2011; Sandhu et al., 2013). At the same time, studies have documented that some groups of ethnic minorities are more at risk than their counterparts from majority groups for developing mental disorders such as depression (Lindert et al., 2008; Missinne & Bracke, 2012).

In the Netherlands, the case of Turkish-Dutch and Moroccan-Dutch minorities has drawn the attention of researchers and societal organizations in recent years, as society considers more carefully the implications of the mental health conditions and social problems experienced by these groups. Surveys have reported the one-monthly prevalence of depressive disorders as 14.9%–16.5% of Turkish-Dutch and 5.8%–6.6% of Moroccan-Dutch, compared to 1.1%–4.1% of native Dutch (de Wit et al., 2008; Schrier et al., 2010) with the highest prevalence among elderly minority group members (21.1%–73.5%; van der Wurff et al., 2004). At the same time, both levels of psychological distress and use of mental healthcare among these groups has been associated with their degree of acculturation, especially Dutch language proficiency, gender, and income level (Fassaert et al., 2011; Fassaert et al., 2009; Fassaert, Hesselink, & Verhoeff, 2009). Hence, the experience of depression among these two groups may be determined by other aspects of diversity than just their ethnicity and regional background.

To address this diversity, training programs need to address the multidimensionality and intersectionality of diversity without disregarding the importance that ethnic and regional background may have for many patients, therapists, and Dutch institutions, especially while considering that labor migration is (for many) a part of their history. In this paper, we present the design and evaluation of a diversity-oriented competence training program aimed at the treatment of depression in two specific ethnic minority groups, namely Turkish- and Moroccan-Dutch patients.

Culture, diversity, and intersectionality in mental healthcare

In recent decades, most efforts made to alleviate barriers and reduce inequity in mental healthcare have focused on the inclusion and strengthening of cultural competence. As a consequence, policies in North America and Europe currently require that therapists acquire cultural competence as part of their professional skills (APA, 2003; WHO, 2010). Initially, cultural competence in the USA was based on a narrow definition of culture that referred to ethnocultural aspects (language, geographic origin, ethnicity, and race) as rather fixed and homogenous characteristics (Kirmayer, 2012). Current approaches consider culture as a complex, dynamic concept constructed by ongoing processes within and between ethnocultural communities, in relation to broader political and social contexts (Gregg & Saha, 2006; Kirmayer, 2012; Marsella & Yamada, 2000; Yamada & Brekke, 2008). While this approach represents progress towards more equity in mental healthcare, voices have been raised in favor of a more diversity-oriented approach that does justice to the increasingly diverse demographics of populations worldwide, which has been referred to as “hyperdiversity” by some authors (Betancourt, 2006; Campinha-Bacote, 2003; Celik, Abma, Klinge, & Widdershoven, 2012; Dogra & Karim, 2005; Good & Hannah, 2015; Hannah, 2011; Hays, 2008; Sears, 2012; Van Mens-Verhulst & Radtke, 2006; Wear, 2003). In our current work, we depart from a definition of diversity that refers to “markers of difference that: a) are relevant to an individual’s societal position, and b) due to biological or social anchoring, are relatively immutable, such as age, gender/sex, ethnicity, sexual orientation, socioeconomic status, (dis-)ability and spirituality” (Van Mens-Verhulst, 2003, p. 26).

Among diversity-oriented frameworks of healthcare, the intersectionality model (Collins, 1990; Crenshaw, 1989; McCall, 2005; Van Mens-Verhulst & Radtke, 2011) that originally stemmed from gender studies is a recent development in the study of ethnic disparities in mental healthcare (Sears, 2012). Intersectionality acknowledges that aspects of diversity are not simply cumulative, but also determine, in dynamic, mutual interactions, the position that a person occupies in society, and as such also shapes their experiences of exclusion, inclusion, power or disadvantage (Knudsen, 2006; Wear, 2003). Intertwined diversity features also create different value systems and illness explanatory models (Sears, 2012; Van Mens-Verhulst & Radtke, 2011).

The intersectionality paradigm shows its relevance in mental healthcare in studies that examine, for instance, the threefold risk for illness among individuals of certain race, social class, and gender: thus, black women were found to express a stronger sense of self and lower rates of depressive symptoms than white women, but only when they held higher socioeconomic positions (Rosenfield, 2012). In such case, exclusive attention to any single aspect of diversity (e.g., race) would lead to inadequate appraisal of the individual and social context in which depression may arise, which might jeopardize the effectiveness of the treatment of depression.

Development and content of the diversity-oriented competence training

The diversity-oriented competence training we propose promotes the development of what we call “diversity competence” (Bekker & Frederiksen, 2005; Van Mens-Verhulst & Bekker, 2005). With diversity competence, we refer to a knowledge, attitude, and skill set that allows therapists to use the intersectionality paradigm to look at themselves, their patients and their interactions. In doing so, therapists are aware that patients' individual needs importantly stem from their various, interacting aspects of diversity (Bechtel & Ness, 2010). Similarly, therapists are also aware of their own aspects of diversity and how these characteristics may influence their therapeutic contacts (Celik et al., 2012; Kumas-Tan, Beagan, Loppie, MacLeod, & Frank, 2007; Wear, 2003). In addition, therapists are able to integrate population-specific insights in their interventions at the same time that they are conscious of the influence of salient diversity features (e.g., individual differences among members of the same population) and contextual factors (Van Mens-Verhulst, 2003).

In cooperation with Palet, a Dutch diversity knowledge platform, we developed a theoretical and practical training aimed at improving therapists’ diversity competence, with a particular focus on its applicability in clinical practice with Turkish- and Moroccan-Dutch patients with depression. The focus on this specific population was determined by the societal urgency to address the gap between high prevalence rates of depression found in these minority groups and the negative treatment experiences reported, among which, attrition and lower care uptake are salient problems (Blom et al., 2010; van de Beek, van der Krieke, Schoevers, & Veling, 2017. ).

Turkish- and Moroccan-Dutch groups in the Netherlands share a history of labor-migration, and later family reunification, which started in the late 1950s in search of better economic prospects. In our training, we take this particular group history seriously and carefully discuss the implications that patients’ diversity in terms of race, ethnicity, and regional backgrounds may hold for their illness models and symptomatic manifestations of depression, mental health status, and therapeutic transactions (Hwang, Myers, Abe-Kim, & Ting, 2008; Kirmayer, Groleau, Looper, & Dao, 2004; Rahiem & Hamid, 2012). Models of cultural competence and cultural frameworks such as D. W. Sue et al.’s (1982) tripartite model of competence, Arredondo et al.’s (1996) behavioral operationalization of cultural competence, and Berry’s (1992) model of acculturation were integrated in the training to explore the impact that migration and living in a (relatively) alien environment can have in the experience of depression (Bhugra & Ayonrinde, 2004). We integrated the population-specific component of the training with knowledge and good practices in the assessment, diagnosis, and treatment of depression of Turkish and Moroccan patients based on a review of published international and national studies and clinical guidelines (Sempértegui, Knipscheer, & Bekker, 2017). Given that studies addressing intersectionality or aspects of diversity related to depression among Turkish and Moroccan patients are still relatively scarce, the training primarily focused on ethnocultural background in relation to sex/gender, religion, and socioeconomic status. Whenever possible, we included information on other aspects of diversity such as age or sexual orientation in relation to the experience and treatment of depression.

We placed the intersectionality model as the basic underlying framework and principal analytical tool for therapists to reflect on their own diversity identity, the influence of their aspects of diversity on the interactions with their patients (e.g., I am a young, South-American-Dutch, non-practicing catholic, highly educated, female therapist treating a middle-aged, first-generation, Turkish-Dutch, practicing Muslim, unemployed, male patient), and the population- specific insights that were presented during the training. This model depicts individuals as complex entities with many identities and self-defining dimensions (Arrendondo et al., 1996), highlighting the sometimes predominant influence of factors such as poverty, insurance status or a bureaucratic health system determining mental health(care) (Yamada & Brekke, 2008).

The intersectionality perspective and population-specific insights were integrated in all the training modules. For instance, therapists were introduced to the diversity-sensitive interview (Bekker & Frederiksen, 2005) in order to learn how to pay attention to ethnocultural aspects such as migration history, generation, and biculturalism, while taking into account other aspects of diversity that might be more relevant for the health status of their Turkish- or Moroccan-Dutch patients with depression. Population-specific phenomena and determinants of depression were also presented and discussed in light of the intersectionality paradigm, paying attention to the influence of such phenomena on the diagnosis or treatment of depression. An example of such phenomena is the “temporary mentality”—that is, the idea among Turkish-Dutch that their stay in the Netherlands is only temporary and alienating. In the discussion, it was important to nuance the influence of ethnicity and make clear that findings concern especially first-generation, unemployed, females of Turkish origin with depressive symptoms (Can, 2010). Therapists could thus appreciate the added value of this nuance, which counterbalances the emphasis on ethnic and regional background and stresses other important aspects of diversity such as socioeconomic stressors particularly relevant for female group members. This nuance also points to other treatment strategies, such as case-management by social workers to alleviate economic stressors and efforts to expand the patient's social network of women in particular. In this sense, intersectional analyses help focus on what is most important at a given point in time, and on opportunities for improving the social situation of an individual, taking the separate aspects of diversity as well as the product of their interactions into account. In the training condition of our study, some therapists referred to the intersectionality framework as an “eye-opener”.

A recent review showed that there are few program evaluations that incorporate both diversity-oriented (patient-centered) and ethnocultural-related components to improve healthcare for ethnic minority patients (Renzaho, Romios, Crock, & Sønderlund, 2013). Most training programs described in the review showed positive outcomes regarding providers’ awareness and ethnocultural understanding. However, most studies were limited to medical staff and students in Canada or the USA. Only one considered the competencies of therapists and only two were performed in Europe (UK; Renzaho et al., 2013) In addition, none of the trainings placed the intersectional paradigm on the basis of their models or addressed a specific ethnic group or clinical disorder. The limitations outlined in this review highlight the potential benefit of a population-specific approach to diversity-oriented competence training. These benefits require further study, particularly in mental healthcare settings in Europe, and with a greater focus on the intersectional influences of different aspects of diversity.

Aim

With this study, we aimed to evaluate the effectiveness of a diversity-oriented competence training aimed at treatment of depression for Turkish- and Moroccan-Dutch patients. Our main research question was whether such training leads to increased diversity competence among therapists when compared to control therapists who did not receive the training. To evaluate the results, we used an objective measure of knowledge (knowledge test), in addition to measures of self-reported diversity competence. We assessed participant competence levels (including knowledge) at baseline, post-training, and three-month follow-up.

We expected that participation in the training would be associated with higher levels of self-reported attitudes, skills and knowledge, and better mastery of actual knowledge about the specific populations. We also expected that improvement in scores would be retained at three-month follow-up evaluation. Finally, as an integrative aspect of training evaluation, we also assessed the participants’ satisfaction with the training, which we expected to be positive.

Method

Participants

Participants were 40 therapists (diverse mental healthcare professionals, including psychologists, psychiatrists, psychotherapists, psychiatric social workers) of four Dutch outpatient psychiatric institutions located in the provinces of South Holland and North Brabant. The group that received the training consisted of two male and 18 female therapists, of which one was excluded from the main analyses due to lack of information on most variables. Their mean age was 43.11 (SD = 10.03); all reported being heterosexual, most were married or cohabitated (77.8%) and several had children (58.8%). Five of the professionals (27.8%) reported being non-native Dutch, four were first-generation migrants, and one second-generation. The control group consisted of one male and 19 female therapists. Their mean age was 35.75 (SD = 12.88); all reported being heterosexual, most cohabitated or were married (80.0%) and approximately one third had children (35.0%). In this group, eight professionals (35.0%) reported being non-native Dutch, three were first-generation and five second-generation migrants. There were no baseline differences in sociodemographic characteristics or acculturation between both groups (see Tables 1A & 1B).

Table 1A.

A comparison of sociodemographic characteristics and correlation with the measures at post-training.

	Study condition					Post-training (T2) measurec
	Training[a]	Control
Variable	M (SD)	M (SD)	t	df	P	Attitude- Awareness	Skills	Self- reported knowledge	Total diversity competence	Knowledge test
Age	43.11 (10.03)	35.75 (12.08)	1.94	36	.05	.30	.28	.23	.29	.03
Total education (years)	21.28 (3.78)	20.75 (3.72)	0.43	36	.66	.27	.09	−.01	.10	.19
Years at current function	5.72 (3.99)	3.99 (5.21)	1.10	36	.27	.39*	.25	.29	.34	−.14
Working hours per week	26.22 (9.35)	27.95 (6.94)	−0.65	36	.51	.16	.16	.03	.12	.11
% ethnic minority patients in case-load	48.83 (30.48)	32.71 (19.02)	1.86	33	.07*	.24	.56**	.52**	.51**	−.18
Previous training (hours)	9.00 (11.10)	5.58 (8.90)	1.05	36	.29	−.02	.05	.08	.05	.15
Acculturation-Maintenance	44.23 (6.48)	45.79 (4.46)	−.83	34	.40	−.01	.07	−.20	−.06	−.14
Acculturation-Adaptationb	35.00 (15.85)	36.04 (10.45)	−.14	11	.88	−.56	−.29	−.11	−.21	.08

Note: aOne female participant failed to fill-in the sociodemographic questionnaire. Tests and percentages shown excluded this case. bThe subscale Acculturation-Adaptation was only applicable to the non-native Dutch therapists (n = 13). *p < .05. **p < .01. ***p < .001.

Table 1B.

Comparison of sociodemographic characteristics and correlation with the measures at post-training (T2) (N = 40).

	Study condition
	Traininga (n = 20)		Control (n = 20)					Post-training (T2) measurec
Variables	N	%	n	%	χ2	df	p	Attitude- Awareness	Skills	Self- reported knowledge	Total diversity competence	Knowledge test
Gender
Female (0 = male, 1 = female)	18	94.7	19	95.2			.10b	−.12	−.17	−.23	−.19	−.06
Ethnicity
Non-native Dutch  (1; 0 = native Dutch)	5	27.8	8	40.0	.20	1	.65	.31	.32	.19	.37*	−.56**
First generation migrant	4	22.2	3	15.0
Second generation migrant	1	5.6	5	25.0
Turkish/Moroccan ethnicity  (0 = no; 1 = yes)	3	15.8	1	5.0			.34b	.25	.08	.18	.21	−.19
Language in daily life
Other than only Dutch	4	22.2	4	21.1			.10b
Education leveld
Middle-high vocational (1)	6	35.3	4	21.1	0.98	2	.61	.06	−.10	−.01	−.02	.05
University (2)	6	35.3	9	47.4
Psychotherapeutic specialization  with national registration (3)	5	29.4	6	31.6
Religious background
Catholic, Muslim	9	50.0	13	65.0	.36	1	.54
Not religious	9	50.0	7	35.0
Sexual orientation
Heterosexual	18	100	20	100
Marital status
Married or cohabitating	14	77.8	16	80.0			.10b
Single (incl. divorced)	4	22.2	4	20.0
Children (yes = 1)	10	58.8	7	35.0	1.25	1	.26	−.23	−.36*	−.40*	−.36*	.14
Experience with diversity competence (0 = no; 1 = yes)
Previous training in diversity or  cultural competence	9	50.0	7	35.0	.36	1	.54	−.003	−.12	−.10	−.08	−.22
Attendance to clinical  supervision	2	11.1	1	5.0			.59b	−.30	−.36*	−.30	−.32	.11
Previous training to work  with interpreters	8	44.4	3	15.0			.07b	.06	−.36*	−.39*	−.39*	−.24
Contact with ethnic minorities  outside work	13	72.2	16	80.0			.70b	.004	−.20	.15	−.05	−.01

Note: aOne female participant failed to fill-in the sociodemographic questionnaire. Tests and percentages shown excluded this case. bTwo-tailed Fisher's exact test (cells with less than expected count). cSpearman's rho. dNot robust. *p < .05. **p < .01. ***p < .001.

Research design and procedure

Our study was a quasi-experimental, multi-center study with a pretest–posttest design. Participants were recruited between November 2010 and February 2012. To this end, we approached several teams of Dutch mental healthcare institutions, and assigned the four teams that were willing to participate randomly (with the SPSS random number generator) to either the training or the control condition. Controls did not receive any training (care as usual). Randomization was done at team (rather than individual) level to avoid trained therapists transmitting competencies gained during the training to untrained control therapists within the same institution.

Participants in the training condition (n = 20) were trained in December 2010 and February 2011. Data collection among controls (n = 20) started in March 2011 and February 2012. From one team with 16 potential participants, the institution allowed only eight to participate in the study; these eight participants were selected randomly from the 16.

Participants of both the training and control condition were asked to fill out the pre-training survey one week before the training or after agreeing on participation (T1). After the training, or two weeks after the first measurement (control condition), participants filled out the post-training survey (T2), which they handed back two to four weeks after the training. Participants of both conditions received a three-month follow-up survey through email (T3), which they could fill out either electronically or with pen and paper. Additionally, trainees filled out evaluation questionnaires immediately after each of the four training modules.

The training comprised a set of evidence-based knowledge, guidelines, and skills to be taught in four modules of 4 hours over two days. The didactic methods included theoretical lectures, practical and imaginary exercises, role-plays, case evaluations, plenary discussions, and homework (see Appendix S1 in the Supplemental Material Online for an overview of the training). A reader with relevant articles and instruments was composed for the participants, which they were asked to read before the training. Instructors included the third author of this paper (certified psychologist, university professor in clinical psychology and supervisor of the current research) and three other experts in the field of diversity and mental healthcare (a cultural anthropologist-sociologist, a certified psychologist and a diversity advisor). Sections of the training were first field-tested and then refined with the comments of 19 therapists who voluntarily took this training as an elective course in their education program to become a registered psychologist in the Netherlands.

The research protocol was approved by the Dutch Medical Ethical Commission for Mental Healthcare institutions (METIGG) prior to the start of the study. At the time each team consented to participate in the study, they gave permission to the researchers to anonymously use all collected information for the study purposes.

Measures

Measurement at T1 included a sociodemographic questionnaire, a measure of the acculturation level, a self-report evaluation survey on attitudes, skills and knowledge, and a knowledge test about the specific target populations. Both T2 and T3 measurement included the self-report evaluation survey and, again, the knowledge test. All questionnaires were in Dutch.

Sociodemographic questionnaire

Demographics included participants’ gender, age, highest education degree, mental healthcare profession, current function and years in that function, percentage ethnic minority patients in their caseload, and experience and training working with interpreters. They were also asked whether they had attended education programs related to diversity or cultural competence (training, workshop, clinical supervision), and if positive, the content and duration of the program. Participants were asked about aspects of diversity, such as their ethnic and regional background, family composition (marital status, number of children), sexual orientation, religious background, parents’ ethnicity, and daily language used at home, at work, and with friends.

Diversity competence

No instrument that assesses population-specific diversity competence was available at the time of the study. The most well-known instruments assessing therapists’ competencies (i.e., Cross-Cultural Counseling Inventory-Revised [CCCI-R]: LaFromboise, Coleman, & Hernandez, 1991; Multicultural Counseling Inventory [MCI]: Sodowsky, Taffe, Gutkin, & Wise, 1994) do not distinguish between clinical groups, and focus mostly on the ethnicity-related social position of the patient (Kumas-Tan et al., 2007). To assess population-specific diversity competence, the Attitude-Awareness, Skills and Knowledge scale (AaSK; Stupar, Sempértegui & Bekker, 2010) was developed. This self-report instrument assessed specific competences related to the therapeutic contact with particularly Turkish and Moroccan ethnic patients. The AaSK contains 21 items divided into three subscales of seven items, each scored at a 5-point scale ranging from 1 (totally disagree) to 5 (totally agree). The attitude-awareness subscale assesses participants’ appraisal of their diversity-oriented therapeutic attitude (e.g., ‘I am aware of my own diversity identity and its influence on my professional practice’, ‘During therapy, I am curious about the life story of my patients of Turkish or Moroccan ethnic background’. The skills subscale assesses the self-reported ability of the professional to work, for instance, with traditional healers, interpreters and diversity-oriented intake interviews (e.g., ‘I am able to use a diagnostic questionnaire with the assistance of an interpreter’, ‘I can recognize when ethnic and cultural factors influence the therapeutic relationship’). The knowledge subscale assesses participants' evaluation of their knowledge about diversity in mental healthcare, especially about good practices in the therapeutic process with Turkish and Moroccan patients with depressive symptoms (e.g., ‘I possess knowledge about (ethnic) diversity in mental healthcare’, ‘I possess satisfying knowledge about the prevalence and manifestation of depressive symptoms among Turkish- and Moroccan-Dutch patients’).

Diversity and population-specific knowledge

We could not identify a measure of prior knowledge on diversity factors and ethnocultural characteristics relevant for the assessment and treatment of depressive disorders in Turkish-Dutch and Moroccan-Dutch people. Therefore, the Diversity Competence Knowledge Test (DCKT; Stupar, Sempértegui & Bekker, 2010) was developed. This 77-item multiple-choice test was designed as a measure of diversity competence (knowledge) level other than self-reported knowledge. The items reflect the main knowledge elements reviewed in the training, which include current models and theories on cultural and diversity competencies and evidence-based information on ethnocultural features interacting with other diversity factors, prevalence, etiology, manifestation, diagnosis and treatment of depressive disorders among ethnic minorities, in particular Turkish- and Moroccan-Dutch (see Appendix S1 in the Supplemental Material Online for content and methods of the training).

Participants are asked to choose one out of three options per item, even if they are not sure about the correct answer. Responses are dichotomously coded as right or wrong (1 – 0) and added for a total score. To illustrate, one question referring to the age-ethnicity interaction on the symptoms of depression: “To what extent is there increased suicidal ideation among Turkish- and Moroccan-Dutch adolescents?”: a) Turkish-Dutch adolescents report higher levels of suicidal ideation; whereas Moroccan-Dutch adolescents report lower levels of suicidal ideation than native Dutch peers. b) Both Turkish- and Moroccan-Dutch adolescents report higher levels of suicidal ideation than native Dutch peers. c) Both Turkish- and Moroccan-Dutch adolescents report lower levels of suicidal ideation than native Dutch peers. (The correct answer is c). We report the psychometric properties of the AaSK and the DCKT in the results section. Both instruments (in Dutch) are available upon request from the authors.

Satisfaction with training

After each module, participants were asked to evaluate five aspects of each element of the training, namely: (i) the quality of the content; (ii) the trainers’ expertise; (iii) the adjustment to prior knowledge; (iv) the quality of the experiential exercises; and (v) the applicability of the reviewed material to clinical practice. The ratings were made on a 5-point scale ranging from 1 (“very bad”) to 5 (“very good”). Participants were also asked to write down the elements they perceived were left out and/or needed improvement.

Data analysis

We performed statistical analyses using SPSS, version 21.0. We established significance at p-value <.05 and we calculated Cohen’s d effect size at each measurement moment. First, we examined the internal consistency of the used measures. Second, we used t-tests for independent samples or chi-square analyses to compare participants’ sociodemographic characteristics, acculturation level and baseline diversity competence between intervention groups. Third, we conducted linear mixed models to examine the effectiveness of the training to increase population-specific diversity competence and knowledge after the training and at three-month follow-up. Examination of the estimates of fixed effects allowed assessment of the significance of the changes between pre- and post-training and between post-training and follow-up, between conditions. Advantages of linear mixed models above repeated-measures ANOVA include robustness against violation of the assumptions of sphericity and independence of observations. Furthermore, cases with missing data are accommodated in the analyses (Blackwell, de Leon, & Miller, 2006). Analysis of the Bayesian Information Criterion indicated the use of compound symmetry covariance as the best fitting structure, which assumes equal covariance for all combinations of repeated measures as well as equal variances. The models included a fixed term for time, a fixed intercept and interactions between time and training condition to determine if the patterns of change over time differed between conditions. We included as confounders those variables that showed significant differences between the two groups at baseline, and that also correlated significantly with the outcome variables.

Before conducting the analyses, we examined the data for outliers. The variable “years of work at current workplace” contained a legitimate outlier, which was converted into a less extreme value (35 to 17) to conserve the case and preserve accuracy (Tabachnick & Fidell, 2007). Next, we visually inspected Q-Q plots showing that the unstandardized residuals of all outcome variables were normally distributed. The assumption of homogeneity of variances was violated by most outcome variables at pre-training (excepting skills), meaning that the conditions presented different variance distributions before the training. However, both groups had comparable sample sizes, which makes the F-test robust against violations of this assumption.

Results

Psychometric Properties of Measures

The attitude-awareness and the skills subscale of the AsSK showed medium to good reliability across the three measurements (Cronbach's α = .65 –.78; Cronbach's α = .69–.80, respectively). The knowledge subscale showed good reliability across the three measurements (Cronbach's α = .88–.92). The reliability for the total scale was very good across the three measurements (Cronbach's α ranged .90–.92). The subscales correlated positively and moderately with each other, showing simultaneously enough relatedness and differentiation properties (correlation coefficients ranged .53 – .78). As these results agree with what could be expected between various facets of (the same) construct of diversity competence, this can be seen as a first indication of good content validity.

The reliability of the total DCKT scale ranged Cronbach's α = .62 – .75 across the three measurements. A value above .60 is a fair reliability level for knowledge or achievement tests, as knowledge about multiple, not necessarily convergent subdomains was tested (Graham, 2006; Nunnally, 1978).

Descriptive analyses

At baseline, independent sample t-tests showed no significant differences between groups for the subscales and overall score of self-reported diversity competence or for overall score on diversity competence knowledge. Independent sample t-tests and correlation analyses with the outcome variables were used to determine possible confounders among the sociodemographic variables. Though there were no significant differences between the training and control conditions on sociodemographic characteristics or acculturation, the between-group differences on the variables age and percentage of ethnic minority patients in case-load were marginally non-significant. In addition, the chi-square test of the variable education level was not robust. Only the variable describing the percentage of ethnic minority patients in case-load showed significant correlations with the outcome variables, therefore it was analyzed as a confounder in a second linear mixed model. See Tables 1A & 1B for the descriptive analyses.

LMM analyses on effectiveness of the training

The linear mixed model revealed significant interactions between time and training condition on total knowledge test (F = 5.79, df = 55.93, p = .005), self-reported attitude-awareness (F = 3.89, df = 61.28, p = .02), self-reported knowledge (F = 9.28, df = 56.71, p < .001), and total self-reported diversity competence (F = 5.85, df = 58.47, p = .01). Regarding these aspects of diversity competence, the therapists of the training condition showed significant improvement after the training in comparison with the therapists of the control condition.

Examination of the estimates of fixed effects of the interaction between time and training condition shed light on the different patterns of change between conditions. Significant changes between pre- and post-training scores were found regarding total knowledge test (t = −3.40, df = 54.92, p = .001), self-reported attitude-awareness (t = −2.69, df = 59.54, p = .009), self-reported knowledge (t = −4.21, df = 56.21, p < .001), and total self-reported diversity competence (t = −3.25, df = 57.67, p = .002) of the participants of the training condition. The scores of the controls did not change between pre- and post-training measurement. The effect sizes of the differences on mean score at post-training between training and control condition were large (see Table 2).

Table 2.

Estimated Marginal Means and fixed effects of self-reported diversity competence and knowledge test.

	Pre-training (T1)			Post-training (T2)			Follow-up (T3)
	TG (n = 19)	CG (n = 20)		TG (n = 18)	CG (n = 16)		TG (n = 13)	CG (n = 8)		C	T	C*T	C*T T1 vs. T2	C*T T2 vs. T3
Variable (range)	M (SE)	M (SE)	d	M (SE)	M (SE)	d	M (SE)	M (SE)	d	F (df)	F (df)	F (df)	t (df)	t (df)
Attitude-Awareness (7–35)	25.07a (0.66)	25.29 (0.64)	−0.08	26.79 (0.67)	24.30 (0.70)	0.88	26.85 (0.76)	24.94 (0.92)	0.71	3.07 (42.41)	0.73 (61.28)	3.89 * (61.28)	−2.69** (59.50)	−0.47 (61.44)
Skills (7–35)	21.87 (0.78)	19.84 (0.75)	0.60	23.17 (0.78)	20.16 (0.81)	0.92	23.09 (0.87)	19.47 (1.04)	1.18	9.11** (42.94)	1.12 (59.97)	0.83 (59.97)	−0.90 (58.82)	0.46 (59.76)
Knowledge (7–35)	20.63 (0.95)	19.50 (0.92)	0.10	24.14 (0.96)	18.60 (0.97)	1.39	23.89 (1.03)	19.51 (1.16)	1.23	8.82** (40.85)	4.56 * (56.71)	9.28*** (56.71)	−4.21*** (56.21)	−0.90 (56.40)
Total diversity competence	67.19 (2.07)	64.61 (1.99)	0.29	74.10 (2.07)	63.12 (2.13)	1.27	73.87 (2.27)	64.01 (2.65)	1.24	8.95** (42.09)	2.86 * (58.47)	5.85** (58.47)	−3.25** (57.67	−0.35 (58.18)
Total knowledge test (0–77)	43.34 (1.55)	43.71 (1.47)	−0.05	49.94 (1.55)	44.74b (1.53)	0.80	46.20c (1.77)	44.05 (1.86)	0.39	1.35 (40.81)	10.93*** (55.93)	5.79** (55.93)	−3.40** (54.92)	−1.42 (56.35)

Note. TG = Training Group, CG = Control Group, C = Condition, T = Time. an = 19. bn = 17. cn = 10.

p < .05. **p < .01. ***p < .001.

The improvement seen at post-training for the training condition remained stable at follow-up for all variables (self-reported attitude-awareness, self-reported knowledge, and total self-reported diversity competence), meaning that the changes in mean scores seen at follow-up (a slight decrease in all cases) were not significant. However, these findings should be interpreted carefully as there was a high attrition rate at follow-up.

Analyses indicated a significant main effect for training condition regarding self-reported skills (F = 9.11, df = 42.94, p = .004). The skill scores of the training and control group differed across the measurement moments. On average, the participants of the training condition showed higher skill scores (see Table 2 for a complete summary of the results).

Controlling for the percentage of ethnic minority patients in the case-load of the therapists at baseline, there was a significant change in scores on total knowledge test between post-training and follow-up (t = −1.97, df = 50.15, p = .05). This finding showed that the scores of the participants in the training condition on total knowledge test dropped significantly at three-month follow-up.

Satisfaction with the training

Assessment of therapists’ satisfaction with the training showed positive outcomes. Participants (including the one excluded from the main analyses) gave the training an overall mean score of 4.12 (SD = .60), which was equivalent to a “good” evaluation. In detail, participants were satisfied with the quality of the content of the presentations and discussions (M = 4.03, SD = .19) and with the trainers’ expertise (M = 4.25, SD = .20). They especially appreciated the evidence-based focus of the presentations and the quality of the study material they were asked to read in preparation for each section of the training. Participants were moderately positive about the adjustment of the contents to their foreknowledge (M = 3.91, SD = .23), the quality of the experiential exercises (M = 3.91, SD = .22), and the applicability of the reviewed material to their clinical practice (M = 3.91, SD = .23). Accordingly, participants expressed they would have welcomed more concrete and more complex clinical cases as part of the training, with more extensive discussion and practice of specific diagnostic and therapeutic techniques. Some participants expressed that they had expected to receive more concrete guidelines to handle specific problematic situations during therapy with Turkish- and Moroccan-Dutch patients. Although a number of participants considered some exercises to be insufficiently attuned to their foreknowledge, some others stated that the training helped them realize that there is still a lot to learn about diversity competence for daily clinical practice.

Discussion

The current study presented and evaluated a diversity-oriented competence training program for therapists, which was particularly designed for treating Turkish- and Moroccan-Dutch patients with depression. The intersectionality paradigm was the main theoretical framework combined with current mental healthcare models of ethnocultural and diversity competence and up-to-date research findings and best practices in clinical work with these groups.

The findings partially supported the hypothesis that population-specific, diversity-oriented competence training increases diversity-related knowledge, awareness, skills, and attitudes. After the training, compared to controls, participants in the training condition had higher scores on self-reported attitude-awareness, knowledge, and total diversity competence; and they achieved higher total knowledge test scores. Significant improvement on self-reported skills was absent after the training. The absence of any effect on skills indicates that social desirability probably played a minor role in this study. On the other hand, skills are one – if not the most– important component of diversity competence, which in other studies have been positively influenced by training. The difference with those studies may lie in the different assessment of skills that we used with our scale. Whereas other instruments mainly assess skills related to acquiring knowledge and awareness (Kumas-Tan et al., 2007), our scale intended to assess self-reported skills related to actual practices in daily contact with patients, such as working with interpreters over the telephone or using existing diversity interviews during the assessment phase. For these skills, a two-day training program may not have been long enough for therapists to improve their skills or to integrate new skills and feel confident about mastering them. One of the limitations of the training might also have been that the strong theoretical focus on intersectionality and other competence models and knowledge occurred at the expense of opportunities to practice with new skills. The participants’ evaluations support this idea, as they were generally satisfied with the training and the quality of its content, but expressed the wish to include more discussion of clinical cases and more practical exercises in the training. Nevertheless, it may be unreasonable to expect that well-established clinical approaches change in a few days or weeks. In order to reinforce new skills, future training should integrate more skill-enhancement exercises, and long-term programs should be implemented that allow therapists to gain more insight into diversity competence and its applications in daily practice.

Concerning the effects at follow-up after three months, findings partially supported the second hypothesis. At follow-up, the effects achieved at post-training on self-reported attitude-awareness, self-reported knowledge, and self-reported total diversity competence remained stable, whereas there was a significant decrease in total knowledge test scores of the training group when we controlled for percentage of ethnic minority patients in caseload. This phenomenon could be caused by the high attrition rate at follow-up, but it most probably indicates difficulties in retaining the gained knowledge. Here again, long-term programs that help maintain competence improvements seem necessary.

Follow-up relapse may also be related to difficulty remaining attuned to the intersectional concept of diversity instead of the conventional focus on ethnic culture. The only other training program we could identify that included the intersectionality paradigm found that medical providers, particularly those with less academic education, had trouble switching their mind-set towards a more integrative way of looking at diversity (Celik et al., 2012). The authors of that study attributed these findings partly to an unsatisfactory adjustment of their training to fit the prior knowledge of participants. According to therapists’ feedback, different levels of prior knowledge also may have influenced the results of our study. Some participants noted that certain elements of the training were too basic, whereas others commented that most of the discussed material was new to them. In the future, better adjustment to the therapists’ prior level of knowledge would be desirable. However, given that multidisciplinary teams, with varied competence levels, are rather the rule than the exception in mental healthcare settings, a more realistic objective might be to help therapists to enhance diversity competence transfer within their teams.

We found that there was discrepancy between self-assessed competence and actual competence of the therapists in the training condition at follow-up. Self-assessed competence remained high, whereas scores on the knowledge test showed significant decrease. This discrepancy is consistent with authors’ findings indicating that self-reported competence is not a strong predictor of competent performance (Worthington, Mobley, Franks, & Tan, 2000). It also raises questions about the role of knowledge instruction. After training, retention of specific knowledge might decrease, but concepts or attitudes necessary to recognize aspects of diversity relevant to the therapeutic process might remain, which would explain the persistently high self-perception of competence. Further research is needed to clarify the mechanisms and consequences of this discrepancy.

Limitations

This study has several limitations. First, the sample size was not large enough to assess or control for some sociodemographic differences between groups and their possible role as predictors of change in competence. Together with the sample size, high attrition rate at follow-up might have also compromised the validity and generalizability of the findings. On the other hand, the use of mixed-model analyses allowed us to address our research questions. Second, this study relied on newly developed instruments to assess self-reported diversity competence and population-specific diversity knowledge. Though the reliability of these instruments and their subscales at the different measurement moments appeared to be satisfactory, other psychometric properties, such as construct validity, should still be thoroughly examined in the future. Further operationalization and standardization of measures of diversity competence and the application of the intersectionality paradigm are needed. In general, the field of diversity competence would benefit from common, reliable, and valid measures of diversity competence in the clinical practice. The use of video vignettes, video recordings, or evaluation reports by experienced supervisors might be better ways to evaluate real life competence and effectiveness of trainings. However, linking therapists’ diversity competence to clinical patient outcomes, such as symptom reduction, improvement of quality of life, satisfaction with treatment and adherence, is the ultimate evaluation method.

A third limitation involves the population-specific section of the training. There are concerns that population-specific information during training may reinforce stereotypes and that there is not yet evidence that gaining knowledge leads to improved healthcare outcomes (Dogra & Karim, 2005). Although population-specific insights might, at first sight, seem incongruent to the intersectionality paradigm, the reported training used both approaches complementarily. This method allowed us to pay careful attention to the intersection of aspects of diversity regarding the Moroccan and Turkish populations (i.e., influence of gender, age, education, etc.) and to highlight more precisely these aspects during the training. At the same time, we rendered enough importance to the ethnocultural-related factors which, given the antecedent of these groups in the Netherlands (labor migration), might be expected to still play an important role for some of the patients and therapists. In addition, conscientious efforts were made in the training condition to diminish the potential limitations of population-specific approaches by integrating scientific mindedness (forming and testing hypotheses), and dynamic sizing (knowing when to generalize and when to individualize; S. Sue, 1998) as analytical tools to consider within-group differences of both the therapists and the patients.

Conclusion

Our approach, like any other, has both benefits and limitations (Kirmayer, 2012), and it likely does not cover the full range of issues necessary to provide high quality care to diverse populations. Nevertheless, the current study has made some progress towards the evaluation and improvement of a diversity-oriented competence training that highlights the local history and contextual reality of the Turkish- and Moroccan-Dutch patients with depression, and at the same time that looks through the lens of intersectionality to expand the focus to as many salient aspects of diversity as possible. The intersectionality paradigm seems relevant to clinical practice, and provides a more realistic portrait of diversity, but this model has only rarely been integrated in mental healthcare settings in Europe. In this study, the availability of a reference group, who did not receive the training, and the use of diverse evaluation methods next to self-report instruments, allowed a careful evaluation of the effectiveness. Evaluation of effectiveness is a necessary first step towards linking therapists’ diversity competence with clinical patients’ outcomes, such as symptom reduction, improvement of quality of life, satisfaction with treatment, and adherence, which are the next necessary steps.