Comparison of Patient Versus Trained Observer Assessments of Healthcare Providers' Use of Motivational Interviewing Techniques for Patients Experiencing Depression and Anxiety in the Dominican Republic.

In settings with limited mental health system capacity, integrated care and the improvement of patient-provider communication surrounding common mental disorders is critical to advancing treatment outcomes. We trained primary care providers in the Dominican Republic in motivational interviewing (MI) to improve communication with patients experiencing depression and anxiety. Providers were randomized to an intervention group, which received MI training, or a control group. To evaluate the training's effectiveness, patients assessed their clinical encounters using the Motivational Interviewing Measure of Staff Interaction (MIMSI). Trained research assistants (RAs) rated a sub-set of those interactions using an adapted MIMSI instrument. Overall, patients (n = 36) perceived their interactions with providers (n = 10) very positively; however, the RAs' ratings strongly indicated that providers' application of MI behaviors was insufficient. Patients generally could not distinguish between intervention and control providers. Findings underscore the need to carefully consider optimal training delivery and cultural influences surrounding the implementation of MI mental health interventions in settings where directive communication is highly valued.

Introduction

The global prevalence of common mental disorders such as depression and anxiety has rapidly increased in recent decades, particularly in low- and middle-income countries (LMICs), where these disorders diminish human capital and potential for sustainable development (Friedrich, 2017; Lund et al., 2018). In the Dominican Republic, depression is the seventh leading cause of nonfatal health loss, and anxiety disorders are ranked sixteenth (IHME, 2020). Despite this disease burden, insufficient resources have been dedicated to mental health in the Dominican Republic, and there is limited health system capacity to provide acute care and long-term follow-up for persons with chronic mental illness (Caplan et al., 2018). Stigma, lack of mental health literacy, and insufficient provider training often impede treatment-seeking in the specialty mental health care sector.

Integrated care—the provision of mental health care within the primary care setting—is one strategy to enhance access to mental health services and increase the quality of care, yet it remains underutilized in many low-resource settings. For example, observations of a sample of 19 primary care clinics throughout the Dominican Republic indicated that only 5 (26%) offered mental health services for adults, and less than half offered antidepressant medication (Luciano et al., 2019). Moreover, Dominican primary care providers often lack adequate preparation to provide mental health care and harbor negative attitudes towards persons with behavioral health disorders; thus, they often feel uncomfortable discussing such issues with their patients (Caplan, 2016). The resultant miscommunication may generate distrust among patients of their diagnosis, the scope of their problem, and treatment options, leading to non-adherence to psychotherapy or medications (Carpenter-Song et al., 2010). Evidenced-based methods to enhance patient-provider communication surrounding common mental disorders in the primary care setting is critical to supporting patient behavior change and improving treatment outcomes.

Motivational interviewing (MI) and collaborative communication strategies are brief interventions that can foster lifestyle changes that may have an effect on chronic diseases such as depression (Soderlund et al., 2011). Over the past two decades, MI has evolved from the principles developed by Miller and Rollnick (2013) and has grown from use primarily in substance abuse and mental health settings to implementation in primary care for a wide variety of illnesses (Barnes & Ivezaj, 2015; Bóveda et al., 2013; Morton et al., 2015; Pace et al., 2017). Its suitability in the primary care setting is due to the fact that it can be delivered as a brief intervention and is helpful for creating collaborative treatment decisions (Miller et al., 2008). MI principles can be categorized as relational or technical (Miller & Rollnick, 2013). The relational aspects of MI techniques, referred to as “MI spirit,” consists of empathy and the core principles of collaboration, affirmation, and autonomy (Miller & Rose, 2009). These are evidenced in MI-consistent behaviors and basic skills such as using open-ended questions, affirming the client’s response, reflections, and summaries. Technical components consist of the evocation or elicitation of clients’ intentions to change, including recognizing and reinforcing change talk, and “rolling with resistance” to avoid confrontation (Miller & Moyers, 2006).

Efforts to equip primary care providers with MI skills have drawn on a variety of training strategies. In their systematic review, Soderlund et al. (2011) found that professional MI trainings in primary care settings in the United States (U.S.), Canada, and Europe have ranged from a 20-min video to intensive workshops lasting more than two days (Soderlund et al., 2011). Common training elements include lectures, role-plays with patient actors or colleagues, audio or video recordings and subsequent analysis of counseling sessions. Several studies have also suggested the importance of follow-up sessions and post-training supervision for building providers’ skills in MI (Madson et al., 2016). For example, in their assessment of a training of US primary care providers to integrate MI into caring for substance abuse patients, Dunn et al. (2015) describe how patient visits were recorded, and providers received regular feedback from the trainer about their use of MI during the visits. Additionally, these recordings were reviewed during monthly group meetings, generating discussion about their strengths and weaknesses (Dunn et al., 2015). Among several studies, observations and self-report questionnaires have largely demonstrated favorable training outcomes in terms of improved MI competence and clinical use of MI among trained primary care providers, although inconsistent results and poor methodology in such studies have been noted (Cheng et al., 2015; Lin et al., 2016; Soderlund et al., 2011).

To maximize the impact of MI for minority or immigrant groups in the U.S., processes for the “cultural adaptation” of MI have been developed. Cultural adaptation consists of identifying ways in which an intervention may conflict with socio-cultural norms, and subsequently modifying the intervention’s content, concepts, tools, and methods, with a focus on culturally congruent communication. A series of studies of culturally targeted MI interventions among Hispanics in the US diagnosed with depression or alcohol abuse demonstrated that addressing stigma and incorporating patients’ social contexts, such as the immigration experience and family dynamics, resulted in greater satisfaction with care (Lee et al., 2011) and higher levels of retention and medication adherence as compared to usual care (Interian et al., 2010, 2013; Lewis-Fernandez et al., 2013).

Despite growing evidence of the effectiveness of MI in English-speaking and/or high-income countries (HICs), including among Hispanic populations, evidence of its application in LMICs is limited. In LMICs, cultural adaptation of MI strategies is particularly important to ensure that they align with local needs and preferences, which may vary considerably from those of the settings where MI strategies were originally implemented and evaluated (Castro et al., 2010; Escoffery et al., 2018). For example, in many less developed LMICs, a number of studies have demonstrated that doctors are consistently viewed as unquestionable authorities and highly credible, especially when compared to other information sources such as family members, friends, and educational materials (Hwang et al., 2016; Rothstein et al., 2020; Souza et al., 2020).

Moreover, while a few studies have employed qualitative and quantitative methods to explore patient experiences with MI, they have been restricted to HICs to date (Angus & Kagan, 2009; Jones et al., 2016; Madson et al., 2015). In cross-cultural contexts, an understanding of patient perceptions of MI strategies is critical to ensuring their relevance, particularly in light of inconsistent evidence surrounding the effectiveness of adaptations for racial and ethnic minorities in the United States (Lundahl et al., 2010). To our knowledge, only one previous study used multiple perspectives, including clinician self-report and objective observers’ ratings, to assess the effectiveness of MI training (Wain et al., 2015). In the present pilot study, we aimed to address this methodological research gap. We explicitly incorporated patient perspectives into the evaluation and sought to compare their observations with those of trained research assistants. Specifically, our objectives were to (1) assess the effectiveness of a culturally adapted training in MI relational techniques for primary care providers in the Dominican Republic to enhance communication for patients experiencing common mental health disorders; and (2) identify challenges related to the cultural adaptation of MI in LMIC primary care settings. The outcomes reported here are the perceptions of providers’ communication and behaviors from the perspective of trained observers and patients.

Methods

This project was based upon an ongoing collaborative effort between Northern Illinois University (NIU; DeKalb, IL), Universidad Autónoma de Santo Domingo (UASD), the largest public institution of higher education in the Dominican Republic, and the Dominican Ministry of Public Health. Ethical approval for the study was obtained from NIU’s Institutional Review Board and Etikos, a national bioethical review panel in the Dominican Republic. We used a quasi-experimental design to assess outcomes of the Motivational Interviewing Training for Patients with Depression and Anxiety [Entrevista Motivacional para Pacientes con Depresión y Ansiedad (EMPDA); hereafter referred to as the “MI Training”] from the perspectives of both patients and trained observers.

Study Setting

All study activities took place at a government-sponsored primary care clinic [Unidad de Atención Primaria (UNAP)] located in the urban municipality of Santo Domingo Este, a provincial capital with a population of approximately 890,000 (Oficina Nacional de Estadística, 2012). The study clinic, which was selected by the Office of the Ministry of Public Health as the study site, is located in Isabelita, a working-class neighborhood with a poverty rate of 26% (MEPyD, 2014).

Participants and Sample Selection

A convenience sample of 24 primary care providers, including nurses, physicians and community health workers (CHWs), were recruited from the UNAP. To recruit providers, the principal investigator (PI) or a co-investigator (Co-I) explained that the study purpose was to extend training in screening and diagnosis of depression and anxiety, and to provide training in MI as a communication strategy to help patients overcome ambivalence to help-seeking. Participation was voluntary and written consent was obtained from all provider participants. The inclusion criteria for providers were age 18 and over, experience of at least six months at the clinic, and plans to be present at the clinic for a minimum of 15 hours per week for the subsequent four months.

Thirty-six patients at the primary care clinic were recruited at the time of a non-urgent primary care visit after all clinical providers had received the MI Training, described below. Patients were eligible for study participation if they were 18 years of age or older, Spanish-speaking, and were determined to have symptoms of depression or anxiety during the clinic visit. Prior to initiating the visit, the provider asked the patient if he/she approved of having the visit observed by a research assistant (RA) to assess the provider’s communication. This procedure was consistent with the normal activities of a government-run health care facility that serves as an academic teaching institution, as visits are frequently observed by medical providers in training and/or outside providers. Specific to this study, RAs were baccalaureate-prepared licensed psychologists. After the patient provided verbal consent to have the visit observed, an RA entered the room. For all consenting patients, during the initial history, the provider administered the two-item Patient Health Questionnaire (PHQ-2) and the two-item Generalized Anxiety Disorder Questionnaire (GAD-2) (Kroenke et al., 2003; Spitzer et al., 2006). This screening procedure is consistent with the World Health Organization’s recommendations (WHO, 2016) and was put into effect after providers received the initial primary care training (prior to the training, there was no formal screening for depression or anxiety conducted by primary care clinic staff). Positive screens were followed by the longer nine-item Patient Health Questionnaire (PHQ-9) and the seven-item Generalized Anxiety Disorder Questionnaire (GAD-7), respectively (Kroenke et al., 2001; Spitzer et al., 2006).

At the conclusion of the usual visit and after addressing the patient's chief complaint, the results of the depression and anxiety screenings were reviewed with the patient. If the patient met criteria for depression or anxiety based on the scoring algorithms (i.e., a score of 5 or greater on the PHQ-9 or GAD-7), the patient received treatment as usual, which consisted of watchful waiting or—if the provider determined that there was a need for more intensive assistance based on the PHQ-9 or GAD-7 scores or clinical judgment—referral to the in-house mental health provider, who would evaluate the patient and determine the most appropriate treatment options. Any patient who screened positive for depression or anxiety and met the additional inclusion criteria were recruited for the study, irrespective of symptom severity or referral status. Immediately following the clinical encounter, an RA asked them if they would be willing to participate in a study to assess their interactions with their provider.

Intervention

The day-long provider training took place at the primary care clinic and was provided by the PI, an Advanced Practice Nurse Practitioner, and the Co-Is, Dominican Clinical Psychologists, who were trained in the relational aspects of MI. The first half of the training was the same for all providers and consisted of the epidemiology, symptoms, screening and diagnostic tools (the PHQ-2, PHQ-9, GAD-2, and GAD-7), and referral criteria of anxiety and depression, including suicide risk. For the second half of the training, providers were randomly assigned to participate in the MI Training (intervention group), or to receive a 90-minute lecture in nutrition and behavioral health (control group). The MI Training was based on an intervention that was developed in Spanish and effectively implemented for antidepressant adherence among Caribbean Hispanics in the United States (Interian et al., 2013). Training strategies included basic definitions of the MI spirit, video analysis of use of MI strategies, role plays, and modeling of communication strategies for patients ambivalent about seeking treatment for depression or anxiety. The focus of the training was supportive, non-directive communication and collaborating with the patient, rather than fostering changes in behavior, as would more commonly be seen in addiction treatment or other brief MI interventions used in primary care.

For several weeks following the initial training, feedback was provided to intervention group providers in two ways. First, the PI and Co-I provided brief, individualized feedback to each provider immediately after observing their interactions with the patient, focusing on the areas of MI communication where the provider was not reaching the objectives. Providers were given examples of alternative behaviors aligned with MI principles that they could have practiced during the specific clinical encounter. In addition, more in-depth feedback sessions were delivered to the providers individually or in small groups three weeks following the training. These sessions reviewed the purpose of the study, key MI principles and steps, and examples of the behaviors that were being observed per the MIMSI. Feedback time was approximately 15 minutes with each provider during the second session. During the feedback, written materials from the initial workshop were used. All clinical providers in the intervention group participated in at least two feedback sessions.

Instrument

The effectiveness of the training was evaluated through the Motivational Interviewing Measure of Staff Interaction (MIMSI), a ten-item scale that measures the extent to which a patient perceives that a provider is communicating in a manner that is consistent with the principles of MI (Hohman & Matulich, 2010). We selected the MIMSI because, unlike more widely used scales for evaluating MI such as the Motivational Interviewing Treatment Integrity code (MITI) and the Behavior Change Counselling Index (BECCI), the MIMSI rates provider behavior from the perspective of the patient and is one of the few validated measures for this purpose (Lane et al., 2005; Moyers et al., 2016).

The MIMSI Likert scale items were designed to reflect the frequency of occurrence of three elements of MI spirit: (1) collaboration, or working in partnerships with the patient and honoring their experiences; (2) autonomy/support, defined as encouraging a patient’s decision-making, self-direction, and informed choice; and (3) evocation, or the elicitation of a patient’s own ideas about change (Hohman & Matulich, 2010). Response options ranged from 1-Never to 5-Always or almost always. In Hohman and Matulich’s original study, the three factors demonstrated alpha coefficients of internal consistency ranging from 0.73 to 0.91, while the overall scale had an alpha of 0.90. In the present study, the alpha coefficient was 0.74. For the present study, we employed both the original and an adapted version of the MIMSI in order to evaluate the providers’ behaviors from both the patients’ and trained observers’ perspectives.

Data Collection

Data collection was carried out at the clinic between October 2019 and March 2020. Staff participants self-administered a brief socio-demographic questionnaire at the beginning of the training session, which included questions related to their mental health training and perceived knowledge and self-efficacy related to treatment of patients experiencing depression and anxiety.

Data were collected from patient participants directly following the clinical encounters of interest. An RA interviewed each patient about their socio-demographic characteristics, perceived health status, and mental health history and treatment-seeking. The RA then administered the MIMSI, which provided an opportunity for the patient to assess the provider’s communication style and behaviors. During these procedures, the RAs encouraged the patients to provide their honest opinions, stating that all responses would be anonymous, data would be kept confidential, and ensuring that study participation would not affect their healthcare.

A sub-set of clinical encounters (n = 21) were observed by trained RA observers, who were assigned to specific providers based on their schedule and availability and blinded to study arm designation. These observations were performed by the three trained RAs and one of the two Co-Is (two at a time). Following the observed clinical encounters, the trained RAs completed an adapted MIMSI to record their perspectives on the extent to which the provider exhibited MI-consistent behaviors. For the initial observations, the PI reviewed the scoring and rationale for scoring with both observers together until a consensus could be reached on scoring and fidelity to interview procedures was obtained. During daily debriefing sessions, RAs also shared observations about non-verbal cues and other behavioral indicators that contributed to their particular ratings.

The MIMSI and the socio-demographic questionnaires were translated and back-translated according to the method put forth by Brislin (1970). The translations were then reassessed with the Dominican research team and pre-tested with Dominican college students to ensure the appropriateness and clarity of all questions. For the other clinical measures (PHQ-9 and GAD-7), previously validated Spanish translations were employed.

Data Analysis

Questionnaire data were double entered by trained personnel and compared for consistency. Data analyses were performed using Stata Statistical Software version 13 (StataCorp LP, College Station, Texas, USA). All MIMSI variables were assessed for normality using the Shapiro–Wilk normality test, and descriptive statistics for nonparametric data, such as medians and interquartile ranges were used to summarize all variables, including both individual items and total scores for the MIMSI.

Interrater reliability between the trained observers’ ratings of matched pairs of participants was calculated through percent agreement as well as a weighted Cohen’s kappa (McHugh, 2012). Using the average of the two trained observers’ ratings for each interaction, differences in the patients’ and trained observers’ ratings were assessed using the Wilcoxon signed-rank test for nonparametric data, which tests the null hypothesis that matched pairs of observations come from the same distributions. The Mann–Whitney U test was used to assess whether the patients recorded significant differences when interacting with providers in the intervention group versus those in the control group; similar tests were applied to the trained observers’ data using the average ratings.

Results

Participant Socio-Demographic Characteristics

A total of 110 patients were screened for study eligibility and 36 ultimately participated in the study. As displayed in Table 1, the majority of patient participants were female, between 35 and 64 years of age, and native Dominicans. More than half of the sample was not employed at the time of their clinic visit, with only six (16.7%) participants working full-time. In the socio-demographic questionnaire, a majority of patient participants rated their mental health as “good”, while approximately one-third of the patients perceived their mental health status as “regular” and three (8.3%) perceived it as “poor” (Table 1). Of the 110 observed patient-provider interactions, 30 patients met the criteria for depression (27.3%), whereas 6 (5.5%) met the criteria for anxiety. The prevalence of depression is similar to other studies of depression in primary care (Ferenchick et al., 2019; Urtasun et al., 2019) and is higher than that observed in the general population in Santo Domingo (18%) (Luciano t al., 2019). Less than one-third of patient participants reported having read about mental health issues in the past. All patient participants completed the MIMSI following their clinic visits.

Table 1

Socio-demographic characteristics and perceived health status of participating patients (N = 36)

Variable	N (%)
Socio-demographic characteristics
Sex
Male	7 (19.4)
Female	29 (80.6)
Age (years)
< 25	3 (8.3)
25–34	5 (13.9)
35–49	11 (30.6)
50–65	11 (30.6)
66 and older	6 (16.7)
Country of birth
Dominican Republic	28 (77.8)
Other	8 (22.2)
Marital status
Married	3 (8.3)
Co-habitation	18 (50.0)
Separated	2 (5.6)
Never married	6 (16.7)
Widowed	6 (16.7)
Employment status
Employed full time	6 (16.7)
Employed part time	4 (11.1)
Not currently working	13 (36.1)
Student	3 (8.3)
Disabled	7 (19.4)
Retired	3 (8.3)
Perceived health status
Perceived physical health status
Excellent	1 (2.8)
Good	15 (41.7)
Regular	17 (47.2)
Poor	1 (2.8)
Very poor	2 (5.6)
Currently suffering from chronic disease
Yes	21(58.3)
No	15 (41.7)
Perceived mental health status
Excellent	1 (2.8)
Good	20 (55.6)
Regular	12 (33.3)
Poor	3 (8.3)
Very poor	0
Patient health questionnaire (PHQ-9) score
Minimal (0–4)	6 (16.7)
Mild (5–9)	8 (22.2)
Moderate (10–14)	11 (30.6)
Moderately severe (15–19)	6 (16.7)
Severe (20–27)	5 (13.9)
Has read about mental health issues in the past
Yes	11 (30.6)
No	25 (69.4)

Twenty-four providers employed by the primary care clinic were recruited, consented to, and participated in the training. Ten of these providers participated in data collection events for purposes of the study (Fig. 1). Of the 24 trained providers, the majority were female, had completed a university degree or higher, were part-time employees, and had more than five years of work experience at the UNAP (Table 2). Findings from the survey administered to providers prior to the training indicated that most providers had intermediate to high levels of perceived knowledge and self-efficacy related to mental health and service provision; overall, they expressed slightly lower self-efficacy in their ability to identify patient goals and discuss depression and anxiety with patients (Table 2). The socio-demographic profile of the ten providers who participated in the study was similar to that of the larger group. These ten participants included five doctors, three CHWs, and two nurses; six providers were assigned to the intervention group while the remaining four were assigned to the control group.

Fig.1

Distribution of providers participating in study

Table 2

Socio-demographic characteristics and mental health training of participating providers (N = 24; N = 10)

Variable	Providers trained in MI (N = 24)N (%)	Provider study participants (N = 10)N (%)
Socio-demographic characteristics
Sex
Male	4 (16.7)	2 (20.0)
Female	20 (83.3)	8 (80.0)
Age (years)
< 35	8 (33.3)	4 (40.0)
35–49	10 (41.7)	5 (50.0)
50 and older	6 (25.0)	1 (10.0)
Educational achievement
Did not complete bachelor’s degree	2 (8.3)	0
Completed bachelor’s degree	3 (12.5)	2 (20.0)
Technical school	1 (4.2)	0
Completed university degree	8 (33.3)	3 (30.0)
Masters, specialty, or doctorate degree	10 (41.7)	5 (50.0)
Type of health provider
Physician	7 (29.2)	5 (50.0)
Nurse	6 (25.0)	2 (20.0)
Community health worker	11 (45.8)	3 (30.0)
Length of time working at UNAP
< 1 year	1 (4.2)	1 (10.0)
More than 1 year, less than 5 years	6 (25.0)	3 (30.0)
More than 5 years, less than 10 years	15 (62.5)	5 (50.0)
More than 10 years	2 (8.3)	1 (10.0)
Type of employment
Full time	5 87.5)	2 (20.0)
Part time	19 (79.2)	8 (80.0)
Mental health training
Has received specialized training in mental health
Yes	21 (87.5)	8 (80.0)
No	3 (12.5)	2 (20.0)
Perceived knowledge of depressive symptoms
Low	1 (4.2)	0
Intermediate	12 (50.0)	6 (60.0)
High	11 (45.8)	4 (40.0)
Perceived knowledge of symptoms of anxiety
Low	1 (4.2)	0
Intermediate	13 (54.2)	7 (70.0)
High	10 (41.7)	3 (30.0)
Perceived knowledge of factors contributing to depression and anxiety
Low	0	0
Intermediate	13 (54.2)	6 (60.0)
High	11 (45.8)	4 (40.0)
Self-efficacy for identifying the patient’s goals
Low	4 (16.7)	3 (30.0)
Intermediate	8 (33.3)	3 (30.0)
High	12 (50.0)	4 (40.0)
Self-efficacy for using empathic statement that demonstrate active listening
Low	2 (8.3)	1 (10.0)
Intermediate	11 (45.8)	6 (60.0)
High	11 (45.8)	3 (30.0)
Self-efficacy for speaking with patients about behavior change
Low	1 (4.2)	1 (10.0)
Intermediate	14 (58.3)	7 (70.0)
High	9 (37.5)	2 (20.0)
Self-efficacy for speaking with patients about depression and anxiety
Low	3 (12.5)	2 (20.0)
Intermediate	10 (41.7)	4 (40.0)
High	11 (45.8)	4 (40.0)
Self-efficacy for communicating with clinical colleagues about depression and anxiety
Low	0	0
Intermediate	10 (41.7)	5 (50.0)
High	14 (58.3)	5 (50.0)

Perceptions of Clinical Encounters from Patients and Trained Observers

The median scores from the patients’ MIMSI ratings indicated that the majority of patients perceived their interactions with providers very positively. Nearly all items pertaining to use of MI strategy, including collaboration, discussing the patient’s feelings, and supporting autonomy, were highly rated; for more than half of the MIMSI items, the median ratings assigned by patients were the most positive possible scores (“5-Always” for the MI-consistent behaviors captured in items 1, 2, 3 and 5, and “1-Never” for the MI-inconsistent behaviors captured in items 8 and 10; Table 3).

Table 3

Comparison of matched patient and observer ratings of provider interactions using adapted MIMSI scaled (N = 21)

		Patient ratings(N = 21)Median (IQR)	Average observers’ ratings (N = 21)Median (IQR)	p value
1	The provider appears to believe that the patient knows what is best for him/herself	5 (3, 5)	1 (1, 1)	< .001
2	The provider is interested in helping the patient resolve their problems in their own way	4 (2, 5)	1.5 (1, 2)	< .001
3	The provider helps the patient recognize his/her own strengths	5 (4, 5)	1 (1, 1.5)	< .001
4	The provider asks the patient about his/her goals	1 (1, 4)	1 (1, 1)	.048
5	The provider is curious about the patient’s thoughts and feelings	5 (3, 5)	2 (1, 3)	< .001
6	The provider helps guide the patient to make good decisions for him/herself	4 (3, 5)	1 (1, 1)	< .001
7	The provider helps the patient to look at both sides of a problem	5 (3, 5)	1 (1, 1)	< .001
8	The provider seems hurried when talking with the patient	1 (1, 1)	1 (1, 1)	.843
9	The provider finds it easier to tell the patient what to do instead of asking the patient for their input	4 (1, 5)	1 (1, 1.5)	< .001
10	The provider gets angry with the patient when the patient does not agree with what the provider is telling him/her	1 (1, 1)	1 (1, 2)	.870
Total score		32 (30, 38)	13.75 (11.5, 15.5)	< .001

In contrast, MIMSI ratings of these encounters from the RAs’ perspectives were significantly less positive than those of the patients, revealing that their perceptions of providers’ use of MI strategies was extremely deficient. The kappa statistic for the two RAs’ ratings was 0.61 and percent agreement was calculated as 74%, indicating that interrater reliability was substantial. In many cases, RAs reported that the providers never or very rarely displayed MI-consistent behaviors, with the exception of question #9, which may not have been understood well. As displayed in Table 3, the Wilcoxon signed-rank tests for the majority of scale items resulted in statistically significant p values (< .05), indicating that the distributions are significantly different. However, the p values for two of the scale items—“The provider seems hurried when talking with the patient” (item 8) and “The provider gets angry with the patient when the patient does not agree with what the provider is telling him/her” (item 10)—suggest that there was overall agreement between the patients’ and RAs’ observations of these behaviors.

These findings are supported by anecdotal data that was recorded from the RAs’ debriefing sessions. In some cases, the RAs reported that staff had been directive, cajoling or coercive during the clinical encounter, yet the patient did not react negatively to this approach. In such instances, the provider would speak to the patient aggressively, at times even berating the patient in a joking manner for not adhering to healthy behaviors as advised. In response, the patients would at times lower his/her head in a gesture of acceptance of blame, and later express gratitude for having been reproached. In these instances, patient participants’ responses to the MIMSI, as well as their informal conversations with the RA, generally revealed satisfaction with the care and communication in the interaction.

MIMSI Ratings Stratified by Intervention Arm

Analyses of patient and trained observer ratings stratified by providers’ study arm demonstrates that overall, neither group was able to distinguish between intervention group providers as compared to the control group (i.e., those who received the MI Training versus those who did not) (Tables 4 and 5). Of the 36 clinical encounters rated by patients, 28 took place with providers in the intervention arm, while the remaining eight were with control providers. The only MIMSI item for which patients’ ratings of the intervention providers differed significantly from those of the control group providers was the second item, “The provider is interested in helping the patient resolve their problems in their own way” (p = .040). Here, intervention providers received higher median ratings than those in the control arm. Of the 21 clinical encounters evaluated by two RAs (16 intervention and 5 control providers), the only significant difference between the intervention and control group providers was for the tenth item: providers in the intervention group were rated as less likely to get angry with patients when the patients did not agree with them.

Table 4

Patient ratings of providers in intervention group versus control group

		Intervention group providers (N = 28)Median (IQR)	Control group providers(N = 8)Median (IQR)	p value
1	The provider appears to believe that the patient knows what is best for him/herself	5 (4, 5)	5 (4, 5)	.657
2	The provider is interested in helping the patient resolve their problems in their own way	5 (4, 5)	1.5 (1, 5)	.040
3	The provider helps the patient recognize his/her own strengths	5 (4, 5)	5 (4, 5)	.527
4	The provider asks the patient about his/her goals	3.5 (1, 5)	1 (1, 3)	.182
5	The provider is curious about the patient’s thoughts and feelings	5 (4, 5)	2.5 (1.5, 4.5)	.052
6	The provider helps guide the patient to make good decisions for him/herself	4.5 (3, 5)	5 (3.5, 5)	.633
7	The provider helps the patient to look at both sides of a problem	4 (3, 5)	5 (1.5, 5)	.870
8	The provider seems hurried when talking with the patient	1 (1, 1)	1 (1.5, 1)	.342
9	The provider finds it easier to tell the patient what to do instead of asking the patient for their input	4 (2, 5)	5 (1, 5)	.606
10	The provider gets angry with the patient when the patient does not agree with what the provider is telling him/her	1 (1, 1)	1 (1, 1)	.950
Total score		34.5 (30, 39.5)	31 (26.5, 35.5)	.153

Table 5

Trained observer ratings of providers in intervention group versus control group

		Intervention group providers (N = 16)Median (IQR)	Control group providers(N = 5)Median (IQR)	p value
1	The provider appears to believe that the patient knows what is best for him/herself	1 (1, 1)	1 (1, 1)	0.740
2	The provider is interested in helping the patient resolve their problems in their own way	1.5 (1, 2)	1 (1, 1.5)	0.567
3	The provider helps the patient recognize his/her own strengths	1 (1, 1)	1.5 (1, 2.5)	0.139
4	The provider asks the patient about his/her goals	1 (1, 1.25)	1 (1, 1)	1.000
5	The provider is curious about the patient’s thoughts and feelings	2.25 (1, 3)	1 (1, 2)	0.365
6	The provider helps guide the patient to make good decisions for him/herself	1 (1, 1)	1 (1, 1)	0.809
7	The provider helps the patient to look at both sides of a problem	1 (1, 1)	1 (1, 1)	0.587
8	The provider seems hurried when talking with the patient	1 (1, 1.25)	1 (1, 1)	1.000
9	The provider finds it easier to tell the patient what to do instead of asking the patient for their input	1 (1, 1.5)	1 (1, 1.5)	0.849
10	The provider gets angry with the patient when the patient does not agree with what the provider is telling him/her	1 (1, 1.25)	2.75 (1.75, 3.25)	0.017
Total score		13.75 (11.5, 14.75)	16.25 (11.25, 20)	0.70

Discussion

In this study, we aimed to assess the effectiveness of a culturally adapted MI training program to improve communication between Dominican primary care providers and patients experiencing depression or anxiety. Through the use and adaptation of an instrument designed to assess MI-consistent behaviors from the patient perspective, we were able to compare the observations of trained RAs with those of patient participants for the same provider-patient interactions; to our knowledge, this is the first study to employ such an approach. Interestingly, we found significant discrepancies between the RAs’ and the patients’ ratings for 80% of the MIMSI items, with the RAs’ ratings overwhelmingly lower and less positive. These findings raise a number of important considerations for the cultural adaptation of MI techniques, and the requirements for healthcare provider trainings, in the Dominican Republic and similar LMIC settings.

Data collected from the RAs indicate that providers were for the most part unable to convey the spirit of MI during their clinical encounters, communicating in a non-collaborative, overly directive, and at times coercive manner. For all MIMSI items related to the MI principles of autonomy/support and collaboration, the median RA ratings were between 1 and 2, indicating that the providers consistently failed to display MI-consistent behaviors during the 21 observed interactions (Table 3). The only MIMSI item that was perceived in a positive light by the RAs was the eighth item (“The provider seems hurried when talking with the patient”); this was also assigned a low score (indicating never or rarely occurs) by the patients. These findings align with those of a small number of studies in the field that have documented the limitations of culturally adapted MI, and highlight the need for further research on intervention content, behavior change among providers, and patient outcomes (Grodensky et al., 2017; Lee et al., 2019; Ornelas et al., 2019).

At the same time, the lack of measurable change in provider behaviors may have also resulted from the limited length and dosage of the intervention. Our initial MI Training period of three hours was slightly low when compared to results of a systematic review of MI trainings in primary care or non-mental health settings, which shows an average of one day of training. Miller and Mount (2001) question the likelihood of sustainability with brief trainings, unless followed by multiple feedback sessions (Miller, & Mount, 2001; Soderlund et al., 2011). Although our study incorporated multiple feedback sessions with providers to foster experiential learning, including both one-on-one and group discussions, our findings suggest that these sessions were not as effective as intended. It is possible that an initial training of greater duration and intensity and/or more extensive feedback sessions are necessary for changing deeply ingrained provider behaviors.

In contrast to the RAs, our patient participants assigned overwhelmingly positive ratings to their providers, and for several MIMSI items, patients gave equally high ratings to providers in the intervention group and those in the control group. These findings may be due, in part, to cultural norms in which doctors and other health providers are viewed as indisputable authorities. High levels of trust and positive attitudes towards health providers have been observed in several studies conducted in Latin American countries and among Latinos living in the United States (Abraido-Lanza et al., 2011; Cabassa et al., 2007; DeVoe et al., 2009; Souza etal., 2020). Abraido-Lanza et al. (2011), for example, found very low levels of medical mistrust among a group of Spanish-speaking women in New York who were predominantly of Dominican descent, and particularly among older participants (Abraido-Lanza et al., 2011). In an analysis of a representative sample of > 16,000 patients in the US, DeVoe et al. (2009) demonstrated that Hispanic patients were significantly more likely to report positive perceptions of interactions with their providers as compared to non-Hispanic whites (DeVoe et al., 2009). As a result of the authority and expertise generally ascribed to physicians, Hispanic populations such as those in our study may be hesitant to challenge or question them.

Cultural norms related to preferences for communication styles and medical decision-making may have also played a role in our findings. MI’s focus on empathy and collaborative communication reflects a more general Western perspective that emphasizes truth-telling and mutual decision-making in patient-provider communication. In contrast, in many non-Western contexts, a directive communication style is not only prominent but also highly valued (Hurley et al., 2017; Matusitz & Spear, 2015). In the US, a number of studies have found that Hispanic individuals are significantly more likely to prefer that doctors make decisions rather than engaging in shared decision-making; Patel and Bakken (2010) demonstrated that this was particularly true in relation to mental health issues (Levinson et al., 2005; Patel & Bakken, 2010). In our study context, such preferences and normative expectations may have influenced patients’ positive ratings of provider communication even though evidence provided by trained observers did not generally support the use of MI-consistent behaviors.

Another important consideration is the fact that some patients in our sample had longstanding relationships with their providers. Regardless of cultural preferences for communication style, it is possible that these patients were more inclined to rate providers in a positive light given their close relationships. Furthermore, these relationships were often characterized by warmth and providers’ frequent affectionate use of touch and verbal expressions of endearment; this may have also contributed to patients’ perceptions that their providers displayed a high degree of empathetic communication.

Limitations

This study faced several limitations. First, the sample size of providers was smaller than anticipated for several reasons. Of the 24 providers who received training, data collection was limited to only ten providers. The other 14 providers were employed part-time and therefore were not always present or not seeing patients during the times that the RAs were performing observations. Moreover, several of the CHW providers who received training had no clinical interaction with patients and were assigned to record-keeping or other clerical tasks during the study period and, therefore, could not be observed using the intervention strategies. Lastly, the sudden closure of the clinic due to the Covid-19 pandemic in mid-March 2020 resulted in our inability to ascertain providers’ feedback in a final interview after all of the observations and feedback sessions were completed.

Our sample of patient participants was also lower than projected for a number of reasons. After initiating the study in October 2019, the population seeking clinic services slowed dramatically during the long holiday period between the middle of November and Día de los Reyes (Three Kings Day) in early January. Furthermore, recruitment rates at the clinic were reduced given that many patients were accompanied by their children and it was not appropriate to administer the PHQ-2 orally when children were present, and because the sub-population of Creole-speaking Haitian patients did not meet the eligibility requirement for Spanish fluency. The clinic sites’ closure due to Covid-19 prematurely ended recruitment and, thus, the optimal sample size was not achieved. Yet despite the relatively small sample sizes, the consistency of the distribution of responses suggests that the results are reliable. Similarly, the moderately strong inter-rater reliability between RAs lends credibility to the observational data.

Second, it is possible that the method we employed for administering the MIMSI for patients, in which data collectors interviewed them face-to-face, may have contributed to a social desirability bias or a reluctance to say anything negative about the providers. These behaviors would be consistent with cultural norms related to hierarchical relationships and appropriate display of deference and respect, as described above. Nevertheless, data collectors aimed to reduce any bias by emphasizing that all responses would be confidential and would not affect participants’ healthcare in any way. In future studies, the integration of qualitative data collection alongside the use of the MIMSI may provide further context and depth of understanding of patients’ experiences. For example, in the “think-aloud” method employed by Pollak et al. (2015), patients listened to audio-recorded conversations with their providers and identified specific instances in which the provider acted in a helpful or unhelpful way (Pollak et al., 2015). Finally, the study was conducted in only one primary care clinic in an urban sector of the Dominican Republic and therefore may not be generalizable outside of this area.

Despite the limitations of this research, strengths of the study design include the use of randomization, a validated measure, and behavioral observations as opposed to self-report or analysis of audio-recordings, the use of, and the blinding of data collectors to interventional status of participating providers.

Conclusions

This study provides valuable insights into multiple challenges inherent to cross-cultural adaptations of psychological interventions, even when the intervention is widely accepted, evidenced-based and shown to be effective in ethnic minorities who speak the same language as the country for which the adaptation is intended. Although studies indicate that simply changing the language of a psychological intervention increases its effectiveness (Griner & Smith, 2006) and integrating relevant cultural and social elements further increases efficacy (Manuel et al., 2015), such cultural adaptations have been tested among targeted populations that may have acculturated to some aspect of the host culture. Our findings also indicate that the competencies needed for effective delivery of complex MI interventions may not be attainable through brief training sessions (Forsberg et al., 2008); it is possible that more intensive initial trainings followed by frequent refresher sessions are necessary to facilitate the transfer of MI skills into actual clinical practice.

Ultimately, this study suggests that cultural adaptations may not be sufficient to overcome deeply rooted cultural preferences in communication styles, particularly in non-Western and LMIC settings. Although training often fosters the development of MI skills, it may be difficult for providers to overcome habituated communication strategies, which may be inconsistent with MI principles. Therefore, a fundamental challenge of culturally adapted MI is whether the adaptation can stay faithful to the essence of the intervention and meet the rigorous criteria of this counseling method, while also remaining culturally relevant. Prior to cultural adaptation of MI, there is a need for more substantive formative research into the fit between particular elements of MI and the target population. The use of qualitative methods such as focus group discussions with medical providers could provide insight into their current understandings of appropriate communication styles and how to negotiate acceptable ways to integrate MI principles into their practice. Identifying the best strategies for demonstrating the value of MI to providers—such as drawing on the evidence base, leveraging patient anecdotes, or getting stakeholder buy-in—may also play an important role in shifting cultural norms around patient-provider communication. In addition, evaluating and accounting for the sources of risk that influence mental health problems among the Dominican population (such as poverty, unemployment, family-related stress) could also increase the local relevance of an MI Training program. Other demographic factors may influence the acceptability of collaborative versus directive communication styles in LMICs such as level of education, age, and gender.

Future implementation research should aim to determine the optimal duration of training programs as well as the most effective means of delivering training to overstretched primary care providers, perhaps through taking advantage of advances in virtual training. Alternative approaches to providing feedback and refresher sessions, such as through peer coaches or by using audio recordings to isolate and examine provider utterances, should also be explored. Finally, it is also critical for future studies to obtain providers’ perspectives on the trainings, which was precluded by external circumstances in our study. Understanding their levels of satisfaction, areas of confusion, and the perceived strengths and weaknesses of a given program will allow for the iterative development and optimization of MI trainings for primary care providers in the future.