Adapting an internet-delivered intervention for depression for a Colombian college student population: An illustration of an integrative empirical approach.

BACKGROUND: Culturally adapted psychotherapy (CAP) studies are limited and until now there are few published examples that illustrate the process of cultural adaptation with internet-delivered treatments. AIM: This paper aims to illustrate an integrative approach to the cultural adaptation of an evidence-based internet-delivered cognitive-behavioural therapy intervention for depression (Space from Depression programme).
METHOD: Mixed method approach utilising quantitative and qualitative methods to assist in the cultural adaptation of the Space from Depression programme was used. The adaptation involved a framework for cultural sensitivity (CSF), alongside an ecological validity framework (EVF) and principles from cross-cultural assessment research. The method included the development of a theory-informed measure, the Cultural Relevance Questionnaire (CRQ), designed specifically for this research.
RESULTS: The adaptation included an establishment of CSF, which included the incorporation of Colombian cultural expressions. College students' (n = 5) and experts' (n = 7) evaluated the EVF based on cross-cultural assessment principles of a preliminary adapted version through the CRQ, showing reliability in the sample (Cronbach's Alpha 0.744). Qualitative analysis supported the culturally sensitive changes or incorporations made to the programme, such as: personal stories and textual translations from English and these were considered ecologically valid and representative.
CONCLUSIONS: The research provided support for the idea that CAP can be conducted systematically for internet-delivered interventions.

Introduction

High prevalence rates of depression have been reported across different cultural groups (Bromet et al., 2011; Ferrari et al., 2013; Kessler and Bromet, 2013; R. Kessler et al., 2009; Kirmayer et al., 2017) with an upward trend in prevalence in low-and middle-income countries (LMICs) (World Health Organisation, 2017), such as Colombia. This country has high prevalence rates reported, which are not far removed from prevalence rates found in high-income-countries (HICs). For instance, a study showed that 12-month prevalence rates of depression reported in Colombia were estimated to be between 6.6 and 10% (Gómez-Restrepo et al., 2011), while 12-month prevalence in the United States and Europe were estimated to be 6.6% (Ayuso-Mateos et al., 2001) and 8.5% (Kessler et al., 2010) respectively.

Recent studies on prevalence of major depression in Latin America place Colombia as having the second highest rates of major depression after Brazil (Bromet et al., 2011; Kessler et al., 2010; Kessler and Bromet, 2013). In line with the previous, a national survey from the Colombian Ministry of Health, (2015) estimated point prevalence of mild to moderate depressive symptoms at 15.6% and severe depressive symptoms at 4.2% of adults. Additionally, depressive disorders have been found to be more frequent among college students in comparison to the general population (Cuijpers et al., 2016). A recent systematic review shows a weighted mean prevalence of depression symptoms in college students of 30.6% (Ibrahim et al., 2013). These rates are comparable to reports conducted in other countries in Latin America (Cova Solar et al., 2007; Melo-Carrillo et al., 2012; Morales et al., 2013; Nogueira-Martins et al., 2004). Two studies carried out in Colombia have reported high prevalence of depression symptoms in college students, the first in 30% (Arrivillaga-Quintero et al., 2004)and the second in 36.2% (Richards and Salamanca-Sanabria, 2014) of the student population sample.

The consistent high prevalence of depression reported in the literature justifies the implementation of interventions for depression in Colombia. However, there are several barriers to access to mental health interventions in that country. In Colombia, the majority (85–95%) of individuals with mental health problems do not access or cannot access services they require or need (Ministry of Social Protection, 2012). Additionally, a National Survey from the Colombian Ministry of Health (2015) shows that about 50% of the population reports that personal stigma is one of the principal causes of not accessing a mental health service in Colombia, followed by geographical location and limited service availability (Ministry of Health, 2015). Furthermore, most of the treatments in LMICs are implemented without considering cultural context; very little research has investigated culturally adapted treatments (Vally and Maggott, 2015), while there is even less research in internet-delivered treatments (Arjadi et al., 2015; Martínez et al., 2018).

Consequently, it is relevant to establish effective treatments for depression and investigate the intervention effects across diverse populations (Kalibatseva and Leong, 2014; Vally and Maggott, 2015) such as LMICs. Given that most of the treatments for depression are developed in Western HICs (e.g., Europe, the US) attention has to be paid to the cultural sensitivity and ecological validity of treatments, so as to maximise their potential effectiveness and aid their effective dissemination in various countries (e.g., Latin America, Asia) (Bernal et al., 2009; Hwang, 2009).

Historically, several psychotherapeutic approaches for the treatment of depression have been developed over the last century, including cognitive-behavioural therapy (Beck and Alford, 2009; Cuijpers et al., 2013; Renner et al., 2013). Nevertheless, consideration of the need to culturally adapt these evidence-based interventions (EBIs) has received limited attention (Rathod, 2016). Over the past decade, studies have examined the potential of Culturally Adapted Psychotherapy (CAP) for depression in different cultural groups (Kalibatseva and Leong, 2014). However this field has experienced difficulties in achieving a convergence of methodologies to be used in the cultural adaptation of a treatment (Bernal and Domenech-Rodríguez, 2012), and empirical studies are limited (Bernal and Domenech-Rodríguez, 2012; Chowdhary et al., 2014; Chu and Leino, 2017; Helms, 2015).

There are several reasons for considering the cultural adaptation of treatments. First, evidence suggests that cultural influences affect the diagnostic and treatment processes (Canino and Alegría, 2008; Givens et al., 2007; Hall, 2001). Second, research supports the relevance of developing treatments based on diverse cultural groups' needs and contexts (Bernal, Jiménez-Chafey, & Domenech-Rodríguez, 2009; Chowdhary et al., 2014; Miranda et al., 2005). There exists cultural differences in the expression and understanding of depressive symptoms across cultures (Haroz et al., 2017; Karasz, 2005; Muñoz et al., 2005; Richards and Salamanca-Sanabria, 2014; Yusim et al., 2010). For example people from South America and Asia trend to express depression through somatic symptoms (e.g., pain, digestive problems), while Caucasian (Europeans) express depression using cognitive factors (e.g., beliefs, thoughts) (Parker et al., 2001; Richards and Salamanca-Sanabria, 2014; Yusim et al., 2010). Third, most outcome research of Evidence-Base Treatments are with Caucasian samples (Bernal and Scharró-del-Río, 2001; Chu and Leino, 2017; Griner and Smith, 2006; La Roche and Christopher, 2008), therefore the treatments are not necessarily generalisable to other ethnics groups (Chu and Leino, 2017; Lau, 2006). Fourth, studies have failed to analyse how cultural groups respond differently to interventions and there is evidence to support lower levels of engagement in treatments when they are delivered to minorities or diverse populations (Lau, 2006). Also, studies have failed to put efforts in the inclusion of a representative sample of minories (van Loon et al., 2013) and there are limited studies of CAP in the country of origin (Cuijpers et al., 2018; Vally and Maggott, 2015). And fifth, ethically, delivering relevant interventions has to consider optimising the potential benefits to the persons and communities; ethical research must be culturally sensitive to the manner in which clinical interventions are introduced and conducted with culturally diverse communities (Trimble et al., 2012).

Considering the previous, culturally adapting treatments implicates an understanding of both culture and psychotherapeutic interventions, which have a complex relationship and several factors need to be considered. Culture can be seen to involve several principal characteristics: It shapes behaviours (e.g., customs, habits); categorises perceptions (e.g., beliefs about mental illness); it involves aspects of experiences related to the context (e.g., family upbringing, socio-economic status); it is deeply shared by members of a particular society or social group; it provides an orientating guide to organise behaviour and it transfers values across generations (Rathod, 2016).

The American Psychological Association (2003) defines culture as a set of belief that influence values, customs and psychological process such as language and education. Castro et al. (2010) describes culture as the embodiment of a worldview through learned and transmitted beliefs, values, and practices, including religious and spiritual traditions. Some authors have considered culture as a structural framework that includes identity that consists of a unique combination of elements (e.g., gender, race, ethnicity), geographic (e.g., urban-rural, country), and associative (e.g., family, religion). It includes ethic group that consists of a collective of people who share common elements (e.g., beliefs, heritage, and status).

Culturally-adapted psychotherapy (CAP) is defined as a systematic change of an intervention protocols through which consideration of culture and context modifies the treatment in accordance with clients' values, contexts, and has cultural relevance to the target population (Bernal et al., 2009). For psychological interventions, adaptation involves the modification of key characteristics, elements, and methods of delivery, while maintaining the core theoretical components of the intervention (McKleroy et al., 2006; Nicolas et al., 2009). Until now there is no golden standard to adapt an intervention culturally. Therefore there are several methods available with a variety of possible adaptations. The following section outlines some of the main approaches.

Theoretical approaches to culturally-adapted psychotherapy

Over the last three decades, psychologists have considered research approaches for the cultural adaptation of EBTs for common mental health problems (Bernal and Domenech-Rodríguez, 2012; Chu and Leino, 2017). Some examples include the work of Griner and Smith (2006) and van Loon et al. (2013) who proposed that adaptations should consider (a) Assesment/discussions of cultural background (b) Ethnic/language matching between the client and the therapist; (c) Translations or adaptations of language; (d) Incorporating cultural values and beliefs in treatment; (e) Collaborating with the population group and adapting the treatment (e.g., focus groups).

The ecological validity framework (EVF) proposed by Bernal et al. (1995) involves eight dimensions that must be incorporated into treatment to increase both the ecological validity and the overall external validity of a treatment. These dimensions are: (a) language, which must be culturally appropriate and syntonic (oral and written); (b) persons refers to the client–therapist relationship; (c) metaphors, refers to the symbols and concepts; (d) content, refers to cultural knowledge about values, customs, and traditions (e) concepts, refer to the constructs of the theoretical model to be used in treatment; (f) goals, implies the establishment of an agreement between the therapist and client as to the goals of treatment; (g) methods, refers to the procedures to follow for the achievement of the treatment goals and (h) context indicates the consideration of the client's social, economic, and political context (Bernal and Sáez-Santiago, 2006).

Cultural Sensitivity Framework (CSF) proposed by Resnicow et al. (2000), is the most common method used in cultural adaptations (Hall et al., 2016). This framework involves either a top down approach (surface structure), where the research team includes cultural modifications to the programme based on observable social and behavioural characteristics (e.g. expressions, images or cultural metaphors about mental illness) (Resnicow et al., 2000; Wilson and Miller, 2003) or a bottom up approach (deep structure), where, for instance, through focus groups the research team analyses what part of the treatment is relevant based on deeper cultural, social, historical, environmental, and psychological factors.

Helms (2015) proposed principles from cross-cultural assessment in the cultural adaptation of an intervention. These are three types of cultural equivalence proposed by Lonner (1985), which have been conceptualised and extended by Helms (2015). Helms concluded that research in culturally-adapted psychotherapy did not include considerations for assessing ethnicity that is defined as traditions transmitted from generation to generation, which are internalised by the person as member of the ethnic cultural group. As a result of this the authors proposed that the process for culturally adapting an intervention should include functional equivalence, conceptual equivalence and linguistic equivalence to achieve an effective cultural adaptation Functional equivalence is defined as the extent to which the same ostensible behaviours (e.g., crying) are interpreted similarly in different cultural or racial groups, occur with equal frequency within these groups, and elicit similar reactions from other members of the groups. Conceptual equivalence, refers to the extent to which different concepts are analogous for the cultural group that is targeted for the treatment; and linguistic equivalence, indicates the language or dialect used during the process has been adjusted so that it has meaning to the person(s) being assessed. In addition, language must be considered in a process and outcome evaluations (Helms, 2015).

The current study attempted to integrate cultural sensitivity framework (CSF) (Resnicow et al., 2000; Wilson and Miller, 2003), cross-cultural principles (Helms, 2015) and ecological validity framework (EVF) (Bernal and Sáez-Santiago, 2006). This integration of theoretical approaches contributed to a systematic cultural adaptations of the internet-delivered intervention. The CSF facilitated modifications to the internet-delivered programme based on top down approach, while maintaining the core theoretical components of the intervention. EVF and cross-cultural assessment principles supported the programme's evaluation by members of the community and also experts.

Culturally Adapted Psychotherapy (CAP) for depression in Latin America

Previous reviews of CAP (Huey and Polo, 2008; Miranda et al., 2005) and meta-analyses (Benish et al., 2011; Griner and Smith, 2006; Smith and Trimble, 2016) have examined the effectiveness of cultural adaptations in numerous treatments targeting, such as family therapy, substance abuse, psychotic disorders, anxiety and depression. Culturally adapted treatments seem more effective than no treatment (d = 0.58), treatment as usual (d = 0.22) or non-adapted treatment (d = 0.32) (Benish et al., 2011; Huey and Polo, 2008) and show moderately strong benefit from pre- to post- intervention (d = 0.45) (Griner and Smith, 2006). A recent meta-analysis found the overall affect size was d = 0.67, which indicates that CAP has better outcomes than other conditions (non-adapted treatment or control group)(Smith and Trimble, 2016).

Recently, a systematic review of depression psychotherapies among Latinos in the U.S, found that studies incorporated between zero and six cultural adaptations, often a mix of deep and surface structure changes (e.g. material translated into Spanish) and nine types of deep structure changes (e.g. piloting of adapted therapy, inclusion of appropriate cultural metaphors and values) (Collado et al., 2016). However, more studies are needed on culturally adapted treatments for Latinos with depression to improve the intervention engagement and outcomes (Kanter et al., 2015).

On the other hand, studies of CAP for a majority population of a particular country, as opposed for a minority in a Western country, are limited. In Latin America Kalibatseva and Leong (2014) report a study in Puerto Rico, which documented a cultural adaptation of CBT with parents of Latino adolescents with depression. The intervention was refined and culturally adapted using the ecological validity framework (EVF), which outlines eight dimensions described by Bernal and Sáez-Santiago (2006). Qualitative analysis reported the intervention was successfully implemented (Sáez-Santiago et al., 2012). The adapted programme has demonstrated significant reductions in depression post treatment and also in comparison to waiting list group (Rosselló and Bernal, 1999, Rosselló and Bernal, 2005; Rosselló et al., 2012).

A recent meta-analysis found eleven studies in Low Middle Income Countries (LMICs) and only two in Latin America using CAP for depression treatment (Vally and Maggott, 2015). In Colombia, a CBT treatment was culturally adapted to target the mental health of dementia caregivers. The programme used was an adaptation of an intervention for Hispanic/Latino population in the U.S. (Gallagher-Thompson, 1992; Gallagher-Thompson et al., 2001). The results showed statistically significant post-treatment effects between the intervention group and control group on depression symptoms as measured by PHQ-9 and positive effects on a measure of satisfaction with life (Arango-Lasprilla et al., 2014). However, the adaptation process was not documented. Similarly, an intervention and material were culturally sensitised for women of limited means with depressive symptoms in Mexico, based on a CBT protocol developed in California (Munoz and Ying, 2002). The adaptation was made by researchers and hospital personnel. The outcomes showed a decrease in depressive symptoms for participants across the modules and treatment components as measured by the Centre for Epidemiological Studies Depression Scale (CES-D) (Lara et al., 2003).

These studies show that culturally-adapted treatments demonstrated efficacy but such treatments with Latin-Americans in their country of origin are limited. Vally and Maggott (2015) concluded in their meta-analysis that the evaluations and a systematic process of cultural adaptations have been absent or lacking in rigour. Therefore their quality remains undetermined.

Culturally adapted internet-delivered treatment for depression

Studies of culturally adapted low-intensity internet-delivered treatments have been reported. For instance, Kayrouz et al. (2015a) examined the efficacy and acceptability of a culturally modified therapist-guided CBT for Arab Australians with anxiety and depression. The cultural adaptation involved examples and educational stories relevant to the population based on feedback from members of the Arab community through focus groups and online survey combined with a literature review on acculturation conducted by the author (Kayrouz et al., 2015b). The results showed improvements significantly across all outcome measures.

Another study showed the efficacy of a culturally attuned internet CBT for Chinese Australians with weekly telephone support. The cultural adaptation included redrawing illustrations, incorporation of cultural expressions according to Chinese values, reframing skills and the revision of the programme by Chinese health professionals and consumers. The experimental group reported significantly reduced depressive symptoms compared to a control group (Choi et al., 2012). Furthermore, Ünlü Ince et al. (2013) examined an internet-based, culturally sensitivity self-help problem-solving intervention for Turkish migrants with depressive symptoms. The cultural adaptation involved migration stories and examples recognised by the target population, also Turkish people evaluated the culturally adapted intervention. The results showed no significant differences on depression between the experimental and control group. However, significant differences post-treatment were found. None of these studies alluded to the previous frameworks for culturally adapted psychotherapy.

Our literature search revealed that in Latin America there are no studies of culturally adapted internet-delivered treatment for depression conducted in the country of origin. However, there are some studies investigating internet interventions for depression developed in Latin America. For instance, naturalistic study monitoring an internet CBT treatment produced in Mexico demonstrated to be a valuable tool for Mexicans with depression in terms of accessibility and mental health support for women (Lara et al., 2014). Moreover, Tiburcio et al. (2016) in a usability study reported on the participants experience of an internet intervention for substance abuse and depressive symptoms in Mexico. They concluded that the intervention realised reductions in drug use and depressive symptoms among users and therefore could be used as an alternative to treatment-as-usual. Furthermore, Espinosa et al. (2016) evaluated the feasibility and acceptability of the internet-delivered CBT programme developed in Chile based on SUMMIT (Supportive Monitoring and Disease Management over the Internet) (German intervention). The programme provided support to clients after they finished their treatment for depression. Although the results show high rates of acceptance and satisfaction, the authors do not detail any cultural adaptions to the treatment, despite the programme being originally made in Germany.

A recent systematic review found only three studies on e-mental health interventions in Low and Middle-Income Countries (LMICs), which were in China and Romania (Arjadi et al., 2015). Likewise, Harper-Shehadeh et al. (2016) found in a recent review and meta-analysis on cultural adaptation of online interventions and bibliotherapy with minimally guided treatment that the details on the CAP methodologies were unreported. However, these studies were effective reducing depression and anxiety significantly. Despite the outcomes showing positive post-treatment results, more research is required to develop interventions that incorporate frameworks for cultural adaptation, especially as adaptation processes were not clearly or fully described in previous research (Harper-Shehadeh et al., 2016).

Space from depression programme

Space from Depression is the name of seven module internet-based cognitive-behavioural (iCBT) intervention for the treatment of depression (See Table 1) developed by SilverCloud Health. Space from Depression programme has shown to be efficacious in comparison to waiting list control group in a community sample in Ireland (Richards et al., 2015). The treatment was evaluated by users as acceptable and satisfactory (Richards et al., 2016).

Table 1

Space from Depression programme description intervention (Richards et al., 2015).

Module	Brief description
Welcome to SilverCloud	This module introduces to the platform. The module explains all of the functions, privacy, icons and buttons on the platform.
Getting started	Outlines the basic premise of CBT, provide information about depression, and introduce some of the key ideas of Space from Depression. Users are encouraged to begin to chart their own current difficulties with depression.
Understanding feelings	The focus in this module is on mood monitoring and emotional literacy. Users can explore different aspects of emotions, physical reactions, action and inaction, and how they are related.
Boosting behaviour	This module focuses on behavioural change to improve mood. Ideas about behavioural activation are included, and users can plan and record activities, and chart their relationship with their mood.
Spotting thoughts	This module focuses on noting and tracking thoughts. Users can explore the connection between their cognitions and their mood, and record them graphically.
Challenging your thoughts	This module supports users to challenge distorted or overly negative thinking patterns, with thought records, as well as helpful coping thoughts.
Bringing It all together	In this final module, users are encouraged to bring together all the skills and ideas they have gathered so far, note their personal warning signs, and plan for staying well.

The treatment includes self-monitoring, behavioural activation, cognitive restructuring, and challenging core beliefs (see Table 1). All modules have the same structure and format, which consist of quizzes, videos, educational content, activities with homework suggestions and a review (Salamanca-Sanabria et al., 2018). Also, users have a supporter, who gives feedback and monitor users' treatment each week (Richards et al., 2015). The supporters in the current study were Colombian postgraduate student in clinical psychology with supervised experience in CBT with adults (Salamanca-Sanabria et al., 2018) who also evaluated the preliminary cultural adaptation version of the Space from depression programme.

This study presents a description of the cultural adaptation of the Space from Depression programme and empirical assessment of its cultural relevance. The aim of this study was to investigate whether the initial top-down (researcher led) cultural adaptation was assessed as culturally relevant by the target population (college students) and by experts with experience in psychological intervention research and internet-delivered intervention research. To achieve this the paper illustrates an integrative approach for the tailoring of an evidence-based internet-delivered treatment for depression to the specific needs of Colombians. We offer an illustration of how we attempted to integrate cultural sensitivity framework (CSF), ecological validity framework (EVF) and cross-cultural assessment research in a systematic way. Furthermore, the idea was to add transparency to the process of culturally adapting an internet-delivered intervention.

Method

This adaptation consisted of a mixed method approach utilising quantitative and qualitative methods to assist in the cultural adaptation of the Space from Depression programme for Colombian population.

Procedure

The programme Space from Depression was culturally adapted for Colombian college student population. The method involved three distinct phases of adaptation, the first considering cultural sensitivity in the adaptation, the second one establishing ecological validity of the adaptation and the third one consisted of the cultural incorporations into the programme (See Table 2).

Table 2

Cultural adaptation methodology description.

Phases	Brief description
Phase a. Cultural sensitivity: Initial adaptation utilising principles of Cultural sensitivity (Research team, professional translators and video company in Colombia).	Programme translations and cultural incorporations: Translation from English to Spanish and incorporation of the cultural expressions in: general content, psychoeducation information, activities, tools, examples, personals stories, videos, quizzes and quotes.
Phase b. Ecological Validity:(Community and specialist evaluation).	Assessment of a culturally adapted treatment by users and experts: Involved the development of a Cultural Relevance Questionnaire (CRQ) measure to capture data on ecological validity of the adapted intervention and its use by users and experts.
Phase c. Cultural incorporations(Research team).	The integration of the feedback for further adjustments to already adapted intervention.

Phase a: cultural sensitivity

The process for the cultural sensitivity approach is described in the framework by Resnicow et al. (2000). This phase included the preliminary cultural adaptation of “Space from Depression” programme (see Table 2).

Phase b: ecological validity

The process used to establish ecological validity was developed using approaches proposed by Bernal and Sáez-Santiago (2006) and Helms (2015) that involved an evaluation of the adapted programme for cultural sensitivity (Phase 1) by Colombian college students and clinical experts using the Cultural Relevance Questionnaire (CRQ) that was developed specifically for this study, see Table 3 for description.

Table 3

Experts' evaluators' information.

Experts(N = 7)	Ph.D.	CAP experience	Internet-delivered treatment experience	Nationality
Expert 1	Clinical Psychology	Yes	No	Colombian
Expert 2	Psychology	Yes	No	Colombian
Expert 3	Public Health	No	No	Colombian
Expert 4	Clinical Psychology	Yes	No	Colombian
Expert 5	Clinical and Health Psychology	Yes	No	Colombian
Expert 6	Positive Psychology	Yes	Yes	Argentinian
Expert 7	Psychology	Yes	Yes	Spanish

Phase c: cultural incorporations

Once the college students and experts completed the CRQ (Phase c), the information was analysed and incorporated in the programme by the researcher. The analysis was carried out through a quantitative and a qualitative information, see Table 2 for description.

Sample of reviewers

Five psychology master students volunteers from a university in Colombia participated as volunteers in this research, they underwent the adapted Space from Depression programme from the position of a user (college student). Five experts in clinical psychology from Colombia evaluated the initial adaptation. In addition, two clinical psychologist experts from Argentina and Spain, who have experience with internet-delivered interventions and also with their translation and adaption for use in Latin America, evaluated the already adjusted programme (See phase 1) using the CRQ. These criteria aimed to guarantee a diversity of experience and skills to support a rigorous approach to assess and establish the ecological validity of the culturally adapted intervention. Experts were invited to participate in this research and they were selected on the basis of following criteria: a) native Spanish speaker from Colombia and/or other Spanish speaking countries; b) PhD in clinical psychology or in psychology; and c) experience in: CBT, depression, research, and low-intensity internet-delivered treatment (See Table 3).

Recruitment & procedure

A research group in a Colombian university sent an email to all of master in clinical psychology students (seventeen students) with a summary of the research study, its aims and a description of the task that was review the programme and completed the CRQ. Five participants signed the informed consent and after that they reviewed the programme and complete the CRQ after engaging with all of the treatment modules (See the appendix A). After 2 weeks students' volunteers completed the CRQ from the SilverCloud platform.

Nine experts were initially invited to participate in this research. The experts were researchers and clinical psychologists in Colombia, Argentina and Spain. Four Colombian experts were selected from two Colombian universities, where the Space from Depression programme were applied. Two Colombian Experts were selected based on recommendations from a research collaborator (LCC) in Colombia. Two Spanish and one Argentian investigators were recommended by the second author (DR), who were selected based on the background experience of internet-delivered interventions and cultural adaptations. The email contained information about the study including the aims of the research and informed consent, which was signed and returned by email. Seven experts reviewed the programme and completed the CRQ. The experts completed the questionnaire after 4 weeks, returning it by email. College students' volunteers and experts completed the first section of the CRQ. The second section of the CRQ was only completed by experts.

Qualitative information was analysed using a descriptive-qualitative analysis (Dey, 2003) to identify categories that could be later used to incorporate further changes (in addition to Phase 1 changes) to the intervention.

Assessments

The study consisted of three phases: (a) an initial cultural adaptation programme by the researcher and SilverCloud team (working with the digital content manager (RB) and the chief technical officer (JB) (b) assessed by experts and users using the Cultural Relevance Questionnaire (CRQ) and (c) the further feedback from the experts and users was used in the final version of the Space from Depression in Study 1 (See Table 4).

Table 4

Study 1 measures used.

Phase	Measure	Assessed variable	Evaluators team
a) Culturally sensitivity	None	Space from depression programme (English version)	Researcher and the SilverCloud team
b) Cultural Validity	Cultural Relevance Questionnaire (CRQ)	Culturally sensitivity programme	Completed by experts and users
c) Cultural modifications	None	Culturally adapted Space from depression programme	Researcher in discussion with the supervisors

Measure

Cultural Relevance Questionnaire (CRQ)

The Cultural Relevance Questionnaire was designed based on a literature review regarding culturally adapted psychotherapy processes and methods. The CRQ has two sections: (1) a general assessment of the programme and (2) an assessment of each of the modules. The first section, the general assessment of the programme in its entirety, it is composed of five items assessed on a 5 point Likert scale (1 represents that functional, conceptual or linguistic equivalence are not reflected within the programme and 5 means that all of these equivalences are reflected within the programme). Each question also includes an open request to elicit qualitative comments from the evaluators' detailing the reasons for their quantitative evaluation. This general assessment section is based on Helms (2015) and Bernal and Sáez-Santiago (2006). The first section of CRQ therefore examined:

Functional equivalence

The raters are asked to evaluated the extent to which the same apparent behaviours are interpreted similarly in different cultural or racial groups, occur with equal frequency within these groups, and elicit similar reactions from other members of the groups. Therefore, treatment components could be interpreted similarly by the target cultural group (e.g., personal stories). The category is composed of three items: (1) the programme involves familiar behavioural or emotional expressions for the cultural group being targeted, (2) the treatment reflects the people and cultural context (e.g., social, political, economic, ethnic, and historical), (3) the treatment goals are tailored to work with clients from this cultural context (e.g., examples, personal stories, tools). Each question also includes an open request to elicit qualitative comments from the evaluators' detailing the reasons for their quantitative evaluation.

Conceptual equivalence

This part of the questionnaire consisted of 1 item that evaluated the extent to which different concepts are the same or analogous between groups (i.e., adjectives used to describe depression). The conceptual equivalence item is: The treatment includes symbols and concepts shared by the cultural group; for instance cultural expressions of depression, ideas or analogies about mental illness are included in the programme. Again it is supplemented by an open-ended space allowing for comments in this area.

Linguistic equivalence

This part of the questionnaire consisted of 1 item that evaluated the level of oral and written language adjustments made for the programme (e.g. regionalism, slang). Also, it is supplemented by an open-ended space allowing for comments in this area.

The second section of the CRQ evaluates each module in the programme Space from Depression The programme is composed by seven modules, which correspond to seven items in the CRQ. Each module was divided into four components that were: Content, examples, personal stories, exercises. Each component collects quantitative judgment assessed on a 5 point Likert scale, based again on the functional equivalence, Conceptual equivalence and Linguistic equivalence principles. Likewise, it is supplemented by a space allowing for comments in each component on each component. Finally, there is an open-ended space allowing for comments for each of the module. The internal consistency (Cronbach's α) of the CRQ scale reached the value of α = 0.744. (See the CRQ into appendix A).

Data analysis

Descriptive statistics were reported based on results from the quantitative questions on the CRQ, which were analysed in SPSS. The CRQ data collected from the open questions were analysed qualitatively (Dey, 2003). Once all the questionnaires were collected, the information was analysed in Atlas Ti. This phase involved a descriptive-qualitative analysis, which consisted of categories generated by evaluators (users and experts) based on comments at the open-ended spaces in the CRQ (general and specific per module). The process of data analysis was carried out by one of the authors (AS) and it was reviewed by the authors (DR)(LT).

Results

The results are described in three distinct phases considering (a) an initial adaptation taking into account cultural sensitivity and (b) assessment of initial adaptation as to its ecological validity and gathering of further suggestions for changes and (c) the incorporations or modifications based on the feedback from the CRQ by the evaluators.

Phase a: initial adaptation of space from depression based on principles of cultural sensitivity

The cultural sensitivity approach to adaptation as described in the framework by Resnicow et al. (2000) was employed. This phase was carried out by the principal researcher (AS) in conjunction with collaborators from SilverCloud Health (developers), professional translators and video production teams from Colombia. The process resulted in an initial adapted version of the intervention. Firstly the programme was translated from English to Spanish by professional translators, and secondly the principal researcher reviewed the translated content in all sections. The revision include the incorporation of cultural expressions and examples in the programme. At the same time, the preliminary adaptation involved the production of new videos and the modification of personal stories to describe Latin American college student scenarios (e.g. economic problems, pregnancy, relationships) and stereotypes for Colombian college students, including the use of Colombian actors (e.g., overall age, clothes, hair style match and accent).

The outcomes in this phase involved major adjustments to several programme components including: (a) translations from English to Spanish; (b) rewriting of the personal stories; (c) remaking the audio recordings and videos; (d) revisions to the quizzes, quotes, and (e) development of the supporter training (manual). All sections were modified to incorporate culturally appropriate expressions and language in to make it more understandable for Colombian college students.

The principal researcher (AS), originally from Bogota, Colombia, made the adaptation based on observable characteristics, experiences, norms, values, behavioural patterns and beliefs of the target population (Resnicow et al., 2000). AS is a psychologist, she has worked as a therapist, researcher and lecturer with different populations in Colombia, including college students, clinical population from low, middle and high socioeconomic status people in different regions of Colombia. The adaptation process carried out by AS consisted on:

Language – Translations revisions: The translators' versions were reviewed several times in order to refine the grammar, making the content more understandable for the target population. For instance, informal language style aimed at college students was used (i.e. tú the informal you), known as “Tuteo” in Spanish. The Colombian iCBT programme (I can feel better programme/Yo puedo sentirme bien in Spanish) was a modified version of the clinically efficacious Space from depression iCBT Programme (Richards et al., 2015). The name I can feel better was selected as a culturally suitable descriptor for improving mood in order to avoid the stigma related to depression in the Colombian culture. Furthermore, the concept “Personal guide” (“Guía personal” in Spanish) was suggested and selected instead of “Supporter”, which does not have a direct meaningful translation in Spanish.

Personal Stories: College students' personal stories in Colombia were adapted based on AS' professional experience with potentially depressed student population. The topics in the stories included: (a) a pregnant student, (b) economic problems, (c) difficulties in romantic or peer relationships and (d) high self-criticism/perfectionism. Likewise, these personal stories were reviewed by a counselor (RG) in Colombia. Then, appropriate images were chosen to represent the Colombian characters portrayed in each personal story.

Audio recordings and videos: Four mindfulness exercises were re-recorded featuring a standard Colombian accent. Likewise, the videos in the programme were overlaid with a Colombian voiceover of the script. Additionally, eight introductory videos were made by Colombian actors.

Quizzes, quotes: Quizzes and quotes were revised the language and appropriate cultural topics and expressions. For instance a question was added to the introductory quiz “Do Latin Americans suffer from depression?” Moreover, quotes from Latin-American writers were incorporated (e.g. Gabriel Garcia Marquez, Isabel Allende among others) into the programme.

Supporter training: The training manual was translated into Spanish.

This phase consisted of the initial adaptation of the programme, based on cultural CSF (Top down adaptation). The end product of Phase 1 of the cultural adaptation of Space from Depression was an initial version of the programme adaptation, specifically tailored to a Colombian student population that was further adapted by utilising processes described below in Phase b.

Phase b: assessment of ecological validity of the initial adaptation (with suggestions for further adaptations)

This section is the assessment of initial adaptation as to its ecological validity [Agreement between users and experts respective perception of the intervention (Bernal et al., 2009)] and gathering of further suggestions for changes is presented here. Cross-cultural principles – Functional, Conceptual and Linguistic - assessment of the programme were evaluated by users and experts (See Table 5).

Table 5

General evaluation by users and experts.

Cross-cultural principles	Rangea	Users (n = 5)		Experts (n = 7)		Combined
		M	(SD)	M	(SD)	M	(SD)
Functional equivalence	(1–5)	4.53	(0.380)	4.38	(0.756)	4.44	(0.609)
Conceptual equivalence	(1–5)	4.60	(0.548)	4.29	(1.25)	4.42	(0.996)
Linguistic equivalence	(1–5)	4.20	(0.447)	3.86	(1.06)	4.00	(0.853)

1 = Functional, conceptual or linguistic equivalence are not reflected within the programme. 5 = All of the equivalences are reflected within the programme.

Evaluators provided positive feedback about the initial version of the culturally adapted programme. College students and experts assessment that the programme was mostly suitable in their evaluation of the intervention as a whole (Table 5). Experts evaluated its component modules as (Table 6).

Table 6

Functional, conceptual and linguistic equivalence per module.

Module	Rangea	Functional equivalence		Conceptual equivalence		Linguistic equivalence
		M	(SD)	M	(SD)	M	(SD)
Welcome to SilverCloud	(1–5)	4.64	(0.4532)	4.17	(1.1060)	4.17	(1.0677)
Getting started	(1–5)	4.78	(0.3037)	4.28	(1.0842)	4.17	(1.0649)
Understanding feelings	(1–5)	4.71	(0.3660)	4.28	(1.1273)	4.14	(1.0870)
Boosting behaviour	(1–5)	4.75	(0.4190)	4.22	(1.0746)	4.12	(1.0394)
Spotting thoughts	(1–5)	4.78	(0.3037)	4.35	(1.0880)	4.24	(1.0842)
Challenging your thoughts	(1–5)	4.82	(0.3134)	4.42	(1.1060)	4.29	(1.113)
Bringing It all together	(1–5)	4.74	(0.3819)	4.35	(1.0979)	4.24	(1.0897)

1 = Functional, conceptual or linguistic equivalence are not reflected within the programme. 5 = All of the equivalences are reflected within the programme.

Qualitative analysis

The results of the qualitative component to the CRQ supported five overarching categories: (a) “language's suggestions or changes” a category, which is understood as instances of language use that could be improved upon within the intervention; (b) “concept coherence” which is defined as the conceptual consistency and understanding of the cognitive behavioural therapy elements in the programme, (c) “quality of delivering” is defined as the manner in which the content is delivered; and (d) “positive impressions” relates to positive comments regarding the strengths of the intervention.

In qualitative comments, college students and experts highlighted some aspects of the intervention that required further attention. Their comments centred on the language used (clarification, colloquial language), relevance of examples and personal stories, conceptual clarity and the engagement promoting technical features of the programme. Specifically, college students suggested refining personal stories, which were evaluated as not so natural for them. Experts, for instance suggested renaming modules. For example, the title “Boosting behaviour” was not considered a suitable title for a module. “It has no meaning in the everyday context of the population” Language suggestions also extended to its more colloquial use (language's suggestions or changes category) including, for instance, Colombian expressions such as, “guayabo” (hung over), “ciclovía” (bike rides on Sundays). Furthermore, experts suggested changing some of the examples, such as: “Failing an exam” instead of “Losing a driver licence”, as the latter is not a common situation for students in Colombia.

The category conceptual coherence was only evaluated by experts, for instance, with a Mindfulness exercise “CBT module is teaching challenging thoughts the session teaches how to challenge and change thoughts but the Mindfulness training is more about the acceptance of thoughts”. The quality of delivery was typically commented by users and experts on in the context of videos: “Videos are not convincing due to low quality” “Consider animations instead of videos”, “The videos presented for each module are not very convincing”, “videos can be improved” In this category the college students also suggested including a quick introduction in the Mindfulness component before to start the exercise, in order to improve the instruction and experience for the user.

Encouragingly, the college students and experts also stated that “In most parts of the programme the language is suitable and understandable” For example, some quotes were: “The information is presented in a didactically and an understandable manner for the users”, “Elements of the programme are familiar to the targeted cultural group”, “Personal stories are close to our reality”, “The cultural context can be analysed through personal histories”, “The intervention presents clearly and simply the CBT basic aspects”, “The emotions' synonyms are well translated and help to understand the meaning of each feeling”, among others.

Phase c. Incorporation and modification of the programme based on the CRQ feedback

The end product of phase 2 was the incorporation and modification of the programme based on the feedback. Qualitative data collected using the CRQ showed that while in principle college students and experts were happy with the cultural adaptation, they also made several suggestions as to the language, examples used and personal stories used. They also commented on other features of the programme not necessarily related to cultural adaptation, such as the quality of the videos or conceptual match between different parts of the programme. The feedback provided was used in the final adaptation of the programme. Based on this type of feedback the research team made modifications to the titles used in the programme. Comments supported the revision of the language in some parts of the programme, such as the inclusion of modules' title. For instance, “How to start to motivate myself?” was used instead of “boosting behaviour” and “How to question my thoughts?” was used instead of “challenging thoughts”. Also, the term “Beating behavioural traps” was not used as a title in the programme Therefore, in the Spanish version, a more informal title was employed: “How to overcome the vicious cycles of depression?” Likewise, personal stories were adjusted and the details were extended to include colloquial jargon for college students to attune the personal stories making that more natural.

Furthermore, mindfulness activities were reviewed and included a quick introduction. The mindfulness activity in the “challenging thoughts” module was changed with more emphasis on acceptance of thoughts. Furthermore, the analysis contributed to increasing the quality of the presentation of the programme (e.g., videos replaced by animation). The animations included images of a typical college student in Colombia. For example, animations involved a male student with a traditional Colombian bag called a “mochila” that is originally made by Colombian indigenous groups. The image of an average Colombian girl with her hairstyle and clothes matching current styles was taken as the model to inspire the animations. Also, animations included the public bus transport in Bogotá called “Transmilenio”.

Adapting an intervention programme is essential to accommodate diversity of experiences and traditions (Domenech Rodríguez et al., 2011). Those examples illustrate features incorporated into the programme. Most of the modifications in this illustration might be obvious (e.g. renaming modules), however the modification implicates a flexibility approach. The culturally adapted Space from depression programme was tested in Colombia for college students (Salamanca-Sanabria et al., 2018).

Discussion

This study developed a robust and theoretically-informed methodology to adapt an internet-delivered intervention for depression in another culture. Limited studies about cultural adaptation in psychotherapy have been conducted, despite twenty years of efforts to conceptualise methods, frameworks and models in this field (Bernal and Adames, 2017; Chu and Leino, 2017; Helms, 2015), and even more limited research available in culturally adapted internet-delivered treatments (Arjadi et al., 2015; Harper-Shehadeh et al., 2016).

Evidence suggests that culturally adapted face-to-face treatments can lead to positive outcomes (Benish et al., 2011; Hall, 2001; Hall et al., 2016; Huey and Polo, 2008; Kalibatseva and Leong, 2014; Moodley et al., 2013). However, there is limited knowledge about any unique criteria that could contribute to an optimal adaptation process (Kalibatseva and Leong, 2014). Likewise, culturally adapted treatments have been critiqued for their lack of theoretical grounding, due to the absence of a systematic adaptation (Chu and Leino, 2017; Helms, 2015). This study is a contribution to expand this field and attempt a systematic modification of an internet-delivered intervention using an integrative organising theoretical framework.

The method was composed of three phases. The first phase facilitated the tailoring of the internet-delivered programme (e.g., language), while maintaining the original treatment components (fidelity). The second phase involved cross-cultural principles assessment research proposed by Lonner (1985) and extend by Helms (2015), and the elements of the ecological validity framework proposed by Bernal and Sáez-Santiago (2006), which were evaluated by users and experts through the Cultural Relevance Questionnaire (CRQ). Finally, the third phase consisted of the incorporations into the programme Space from depression based on the feedback in the CRQ.

In general, users and experts were positive about the culturally adapted programme, while they suggested further changes. The quantitative analysis of the CRQ showed that the initial version of the programme was assessed positively by users and experts regarding the cultural validity of the treatment [Functional equivalence, Conceptual equivalence, Linguistic equivalence]. Functional equivalence scores suggest that the programme included components (e.g. examples, personal stories), that are relevant for the cultural group (Regnault and Herdman, 2015). Furthermore, conceptual equivalence scores suggest that the online intervention used constructs that should be understood by the target population (e.g. adjectives that describe depression) (Helms, 2015). Finally, linguistic equivalence scores suggests that the language used in the programme could be improved (e.g. inclusion of cultural expressions) within the population. Expert's ratings were slightly lower than ratings of users in the general section in the CRQ, which might support a more rigorous analysis of items from the experts, who also provided several comments to the programme components compared to users.

The methodology documented in this paper is a first experience that illustrates an attempt to systematically modify an evidence-based internet-delivered intervention for depression, which is replicable in future studies. This experience supports the inclusion of community members in the process of adapting or developing a treatment as an essential component to increase its validity (Bernal and Domenech-Rodríguez, 2012). Furthermore, studies have demonstrated participation of the community in cultural adaptations is of utmost important to attract and retain participants (Kalibatseva and Leong, 2014).

Studies have argued the importance of developing measures and refining existing measures of cultural validity as the next step in this field (Castro et al., 2010; Helms, 2015). The integrative approach adopted for the current study with the development of the associated CRQ measure is a first contribution to integrate a paradigm for developing measures to incorporate ethnic and racial experiences, previously discussed by Helms (2015). The CRQ can be further refined and elaborated upon in future studies. The authors recommends in the culturally adapted assessment process based on the CRQ to include the original version of the programme and the adapted version to verify the cultural equivalences' content in the programmes for future studies as is suggested by (Helms, 2015). In this study, not all of our evaluators could check the English version, due to limited knowledge of the language. Therefore, the assessment was based on the observations from the culturally adapted Space from depression programme and thus can be rather seen as assessing cultural relevance than equivalence.

The inclusion of diverse populations are generally underrepresented in scientific and clinical analyses and are known to be a hard-to-reach population for research purposes (Arjadi et al., 2015; Cuijpers et al., 2018). The current study supports the inclusion of diverse population in research and the relevant necessity to use culturally adapted internet-delivered interventions for depression.

Limitations

Our approach was characterised mostly as top-down (the initial adaptation was prepared by an expert and the researcher) and therefore likely misses some of the richness that could be gained from a more “balanced” approach that would incorporate more fully the views, interests, opinions of users and clinicians on the ground, such as opinions from a focus groups. It is relevant that Colombia is a diverse cultural country, where socioeconomic status also present cultural differences (Culture Ministry of Colombia, 2013). There are challenges and risks to adapting an intervention, such as presuming global understanding from knowledge of any one culture or subculture (Rathod and Kingdon, 2014).

Another limitation is that the college students' sample who evaluated the preliminary cultural adaptation of the programme did not include college students with depressive symptomatology. The Cultural Relevance Questionnaire (CRQ) showing reliability in the sample (Cronbach's Alpha 0.744). However the study not included a representative sample that allows identify factors loadings analysis and therefore a validation of the CRQ. Also, the second section of the CRQ was completed by only the experts, leaving out comments and suggestions for college students.

Future directions

Internet-delivered interventions for depression have shown significant effects post treatment (Richards and Richardson, 2012), specifically in Western high income countries. Low and middle income countries around the world have scarce access to mental health services (World Health Organization, 2008). A strategy to utilise the internet to provide more widely-available and low cost mental health care has potential. Additionally, internet-delivered treatments could contribute to the globalization of mental health services and psychological interventions, for which cultural adaptation is key (Arjadi et al., 2015). Therefore, establishing a standardised procedure for cultural adaptation of treatments is relevant.

The authors recommend the empirical examination of the process for cultural adaptation and to incorporate multiple methods of assessment, including measures such as the CRQ, and indeed improving upon this initial CRQ, which can assess the ecological validity of a culturally adapted treatment. Furthermore, community evaluations are a relevant part of the culturally adapted programme; therefore we suggest to include their evaluations before, during and at the end of the intervention, which is recommended by others authors (Castro et al., 2010).

Cultural adaptations methodologies appear to be critical for engaging the users and delivering services effectively (Domenech Rodríguez et al., 2011). Research in the area needs more work to empirically document the impact of culturally adapting existing evidence based internet-delivered interventions. The information presented in this paper may be of interest to the community of researchers, who are integrating knowledge and procedures with the realities of working in ethnically, culturally, socio-economically, and otherwise diverse communities.

Implications

The culturally adapted programme (Space from Depression) described in this paper is currently being researched in a randomised control trial (RCT) to examine the efficacy and feasibility of internet-delivered treatment for Colombians with depression (Salamanca-Sanabria et al., 2018). The authors recommend conducting RCTs of culturally adapted interventions, and to continue to assess the adaptations, in order to achieve efficacy and engagement with the treatment.

Conflicts of interest

Alicia Salamanca is the recipient of an Irish Research Council (IRC) scholarship award [EPSPG/2014/98] under the Enterprise/Academia collaboration scheme. The scholarship is funded by the IRC and SilverCloud Health. Since March 2018 Alicia Salamanca is a part time employee in SilverCloud Health. Dr. Derek Richards is Director of Clinical Research at SilverCloud Health. Dr. Ladislav Timulak provides a research consultancy for SilverCloud Health.