Challenges and opportunities in the science of research to practice: lessons learned from a randomized controlled trial of a sexual risk-reduction intervention for psychiatric patients in a public mental health system.

OBJECTIVE: Human immunodeficiency virus (HIV) prevention efficacy trials with psychiatric patients have been conducted in research settings in high-resourced countries, establishing short-term efficacy for reducing sexual risk behavior. None has been implemented within systems of care. In the last decade, overcoming this research-to-practice gap has become a focus of implementation science. This paper describes the first and only HIV Prevention intervention trial for psychiatric patients conducted in real-world outpatient psychiatric settings facilitated by trained clinic-based providers.
METHODS: The HIV Prevention intervention, which uses the Information-Motivation-Behavioral Skills model to achieve sexual risk-reduction, was rigorously adapted to the local context and clinic services' needs. Participants from eight clinics were randomized to HIV Prevention or Health Promotion conditions.
RESULTS: HIV Prevention participants showed significant improvement in Information-Motivation-Behavioral domains; in this group, behavioral intentions were associated with significantly fewer unprotected sex occasions, but reduction of unprotected sex occasions was similar in both conditions.
CONCLUSION: Our trial was conducted before implementation studies became widely funded. Transporting an intervention to a new culture or into real-world practice settings may require adaptations. Our results demonstrate that clear guidelines are needed regarding whether to conduct efficacy, effectiveness, and/or implementation research as the most appropriate next step. CLINICAL TRIAL REGISTRATION: NCT00881699.

RCT Entities:   Population Interventions Outcomes

Introduction

Worldwide, human immunodeficiency virus (HIV) infection rates among people with severe mental illness (SMI) are much higher than the general population.1 Global initiatives that strive to end the HIV epidemic must include this vulnerable population.2 More than 30 risk behavior studies, conducted primarily in the United States, have shown that, compared to the general population, adults with psychiatric illness have higher rates of sexual abstinence, but when sexually active, they have higher rates of many risk activities, including having multiple sex partners, having sex partners with known HIV risk behaviors, exchanging sex for money, shelter or goods, using condoms infrequently, having sex partners of unknown or positive HIV status, and sex between men.

HIV Prevention interventions have focused on reducing these prevalent sexual risk behaviors in this population. Efficacious interventions all incorporate aspects or principles of behavior change derived from testable models of risk-reduction – predominantly the Information-Motivation-Behavioral Skills (IMB) model – and use multiple-session, small group delivery formats. The IMB model posits that three fundamental components are required to reduce HIV-related sexual risk behaviors: 1) information regarding the routes of HIV infection/transmission and methods of risk-reduction/prevention; 2) motivation to reduce HIV risk behaviors (e.g., condom use attitudes, intentions) by acting on existing knowledge; and 3) behavioral skills (e.g., technical skills in condom use and sexual assertiveness/negotiation).3 Studies have shown that the IMB model is appropriate to the needs of adults with SMI who have misinformation about how HIV is transmitted and reduced self-efficacy4; components of the model have been shown to predict sexual risk behavior changes over time.5

When the present study began, six intervention trials provided evidence of the feasibility and efficacy of HIV-sexual-risk-reduction interventions tailored to people with SMI; these studies were conducted in the United States by researchers outside clinical settings. The results showed increases in HIV-related knowledge, positive attitudes toward condom use, increased intentions to reduce risk behaviors, increased condom use self-efficacy, and reductions in risk behaviors from 3 to 6 months post-intervention that diminished over time.4,6-11 Limitations included relatively small samples, single-gender samples, lack of attention-control conditions, variability in measures of risk behavior, look-back periods, follow-up intervals and definitions of efficacy, small effect sizes, and testing of the interventions in research sites rather than clinical settings. This paper describes the first and only sexual risk-reduction intervention trial for people with mental illness to be conducted in outpatient psychiatric programs facilitated by trained clinic-based providers.

Brazil’s robust response to the HIV epidemic has served as a model for HIV prevention and treatment policies around the world, making Brazil an ideal site to adapt and test efficacious U.S. HIV Prevention interventions for psychiatric patients.12 We obtained U.S. National Institute of Mental Health (NIMH) funding to conduct two consecutive studies in Rio de Janeiro, RJ, Brazil (R01MH065163; principal investigator [PI] MLW, both studies). These studies (PRojeto Interdisciplinar em Sexualidade, Saúde Mental e AIDS, or Interdisciplinary Project on Sexuality, Mental Health and AIDS – PRISSMA) were conducted before implementation science was a NIMH funding priority. PRISSMA 1 involved adapting the six efficacious U.S. interventions into one intervention that was culturally tailored and piloted within the public mental health system in Rio de Janeiro.13-15 This paper reports on PRISSMA 2, a randomized controlled trial (RCT) conducted in real-world clinics and facilitated by clinic providers to reduce sexual risk.16 Both PRISSMA studies were consistent with current calls to adapt interventions in a manner that enhances the likelihood of successful implementation in new settings.14

Methods

For context, we briefly describe PRISSMA 1, a multicomponent, 4-year study that used: community-based participatory research14; simultaneous training of Brazilian and U.S. investigators in research methodology; ethnographic observations, focus group discussions, and key informant interviews with psychiatric patients and mental health care providers; translation of the extant published U.S. efficacious risk-reduction interventions4,7,8 into Portuguese and adaption of the interventions to local culture and clinic services’ needs; translation of relevant Brazilian materials into English so that the U.S. research team could learn local theoretical approaches and message delivery strategies; translation and adaption of assessment instruments2 plus training of local research staff to administer them; and community participation in the adaptation and implementation of the intervention through recruitment of a diverse and representative community advisory board (Figure 1).

Figure 1

Overview of PRISSMA studies. PRISSMA = PRojeto Interdisciplinar em Sexualidade, Saúde Mental e AIDS (Interdisciplinary Project on Sexuality, Mental Health and AIDS); RCT = randomized clinical trial.

All stakeholders focused on balancing fidelity to the efficacious interventions and fit to the new setting (both a new culture, Brazil, and a new setting, public outpatient psychiatric clinics). Culturally-based modifications to delivery strategies included considering the impact of cultural beliefs and mental illness stigma on sexual risk behaviors, incorporating interactive and expressive activities, considering low literacy rates, and determining that mixed-gender groups would fit best into clinic activities and reinforce safer peer norms in a predominantly heterosexual patient population. An attention-control intervention manual used in a previous U.S. RCT addressing common comorbid medical conditions was translated and adapted to specific local needs. Details of these approaches can be found elsewhere.8

The resulting interventions (HIV Prevention and Health Promotion) were manualized, pre-piloted with adult psychiatric patients in one of the psychiatric clinics, and finalized after incorporating all feedback. Assessments were pre-piloted and refined for use in the pilot phase. We conducted the feasibility pilot at a different psychiatric clinic to prevent contamination from the pre-pilot. Randomization to intervention conditions was carried out successfully. Baseline data demonstrated high rates of sexual risk behavior among sexually active participants. At 3 months post-intervention, there was a reduction in the mean number of unprotected acts among sexually active participants in the HIV Prevention group and a corresponding increase in the Health Promotion group. Thus, the results were in the desired direction, although the pilot study was not designed to test efficacy.15 There were no adverse events. Quality control demonstrated good to very good facilitator adherence and competence. Two-thirds of the participants attended five or more of the eight sessions, with 94% reporting being very satisfied with the intervention. The feasibility of conducting a two-arm trial was established.

The current study (PRojeto Interdisciplinar em Sexualidade, Saúde Mental e AIDS, or Interdisciplinary Project on Sexuality, Mental Health and AIDS – PRISSMA 2)

When applying for funding for this RCT, we followed the NIMH and the National Advisory Mental Health Council standards which, at that time, called for efficacy testing when adapting evidence-based interventions for new settings rather than using either effectiveness or nascent implementation designs. The resulting large, multisite RCT was the first such trial to evaluate the efficacy of U.S. HIV risk-reduction interventions adapted for adult psychiatric patients in a lower-resourced country and embodied five innovations, being: 1) adapted from existing efficacious interventions using a community-based participatory research approach; 2) clinician-delivered; 3) conducted within public mental health settings; 4) adapted to and conducted in a lower-resourced country; and 5) designed to establish long-term (12-month post-intervention) outcomes. We tested the hypothesis that the HIV Prevention condition would reduce sexual risk significantly more from baseline to 12 months post-intervention than an attention-matched Health Promotion control condition.

Participants

Participants were recruited from four community mental health clinics and four hospital-based outpatient mental health clinics; in Brazil, these are the most common types of clinics in which public mental health care is delivered. Patients whose primary psychiatric diagnosis is an alcohol or other drug use disorder are treated in designated specialized settings. Flyers and posters were distributed in the clinics. Clinic providers were informed about the study and asked to refer all patients to the study recruiters, who screened all provider- and self-referred patients for eligibility. Our community advisory board included one clinician and a relative of one patient from each of the eight RCT clinic sites and advised the research team every 2-3 months. Given the high percentage of participants with sexual risk behaviors in our sexually active pilot sample (78%), the community advisory board urged us to include all sexually active patients having vaginal and/or anal intercourse, whether protected or not, in the 3 months prior to enrollment. There were two ethical reasons for this: 1) concern that patients might engage in risk behavior for the purpose of becoming eligible to participate in the study (even in the absence of reimbursement for participation, which was not permitted in Brazil); and 2) concern that sexually active low-risk patients might not remain so. The consensus reached was that no sexually active patient should be denied the potential benefit of participation. Eligible participants were at least 18 years old, sexually active (vaginal and/or anal sex) in the prior 3 months regardless of condom use, receiving psychiatric treatment in the study sites, and capable of giving written informed consent. Patients with primary alcohol or other drug use disorder and/or intellectual disabilities were not eligible to participate, and those with acute psychosis or active suicidality at the time of the screening interview also were excluded unless eligible upon reassessment once they were stable. Although the existing U.S.-based efficacious interventions were developed for psychiatric patients with diagnoses consistent with SMI, this trial was conducted in clinical rather than research settings and was not restricted to those with SMI; instead, patients meeting study eligibility criteria also included those with common mental disorders (e.g., depression, anxiety, post-traumatic stress disorder, etc.).

Procedures

After eligibility screening, evaluation of capacity to consent, obtaining written informed consent, and baseline assessments (see Measures), qualified participants were randomized in each cohort by gender in a 1:1 ratio within each clinic, blind to condition, using procedures to ensure unbiased randomization into the HIV Prevention intervention or the attention-matched Health Promotion control intervention, which focused on chronic medical conditions commonly seen in psychiatric populations and included the provision of HIV information as one of the topics. Both intervention conditions comprised eight 2-hour group sessions that took place weekly in each clinic and three booster sessions delivered 6 months after the completion of the initial eight sessions. Participants in both study arms received the same assessments on the same schedule: immediately post-intervention; 3- and 6-months post-intervention; immediately post-booster (7-months post-intervention); and 12-months post-intervention (about 15 months post-baseline assessment). Assessment at all timepoints included the primary outcome: sexual risk behaviors during the past 3 months. Psychiatric diagnosis was assessed at baseline. Participants were provided with transportation and food vouchers valued at $10 per visit. The U.S.-Brazil research team conducted training of recruiters and interviewers, weekly supervision, review of recorded sessions, and inter-rater reliability evaluation of the diagnostic and risk behavior assessments. Trained mental health professionals, blind to the participants’ intervention condition assignment, carried out all assessments through face-to-face interviews. The interviews were audiotaped, and every seventh tape was reviewed for quality assurance. Each group session in both intervention conditions was co-led by two facilitators: usually of different genders, from a range of disciplines (e.g., psychologists; social workers; occupational, music, and art therapists; nurses), and from within the system of care. Experienced supervisors trained recruited staff in group facilitation using the manualized interventions. Two different teams of facilitators conducted the experimental and control interventions to minimize bias and/or contamination between intervention conditions. We video-recorded the intervention and booster sessions and tracked attendance. Supervision included corrective feedback and observation of sessions.

Measures

To test the hypothesis that the HIV Prevention intervention would reduce sexual risk significantly more from baseline to 12 months post-intervention than the attention-matched control Health Promotion intervention, we measured sexual risk behavior and participant characteristics.

Demographic and clinical characteristics

Participants were asked their age, gender, sexual orientation, race/ethnicity, marital status, and education level. The Mini-International Neuropsychiatric Interview-PLUS, a structured psychiatric interview validated for the DSM-IV and ICD-10 with both U.S. English and Brazilian Portuguese-speaking patients, was used to assess both severe and common mental disorders.17

Sexual risk behavior

The primary outcome measure, self-reported frequency of unprotected vaginal and/or anal sex occasions during the last 3 months, was assessed using the Sexual Risk Behavior Assessment Schedule (SERBAS), which has undergone extensive psychometric testing with a variety of populations, including people in psychiatric care in Brazil.18 Consistent with previous SMI reliability studies in the United States, The Brazilian version of SERBAS showed excellent test-retest reliability for being sexually active in the past 3 months (kappa = 1.0), and fair to good agreement on the frequency of anal and vaginal sex, including unprotected occasions (intraclass correlation = 0.5-1.0).15,18

The SERBAS employs gender-specific interviews to elicit detailed information regarding sexual practices and related behaviors in the past 3 months, including the number, gender, and type of sex partner (steady, casual, or exchange); the types of sexual acts performed during each sex occasion; whether sexual acts were protected by condoms (male or female); whether sex was preceded by substance use (e.g., alcohol/drugs); whether sex was bought or sold (e.g., exchange sex); and the participant’s self-reported history of HIV testing and status and knowledge of his/her partner(s)’ HIV testing history and status.19

Intervention model measures

The measures also included rigorously-translated and back-translated instruments examining the IMB model intervention components.3 HIV knowledge about transmission and prevention was assessed using the Brief HIV Knowledge Questionnaire, an 18-item true/false scale that has been used with SMI populations in the US.20 The translation and back-translation resulted in the elimination of one item due to confusing double-negative phrasing in Portuguese. Thus, scores range from 0 to 17, with higher scores indicating greater knowledge. The questionnaire has acceptable internal consistency (= 0.75-0.89), and test-retest reliability (= 0.76-0.94).15,20

Motivation for behavior change was assessed by three measures. Behavioral intentions regarding HIV were measured with four items that assess strength of intentions to reduce HIV risk behavior (e.g., “I will use a condom the next time I have sex”). Items are scored on a four-point Likert scale (1 = “definitely will not do” to 4 = “definitely will do”); higher scores indicate greater intentions to act to reduce risk.21 Two items from the access to condoms measure were used to assess whether participants had condoms either with them or in their homes. Both measures have been used in prior research (average alpha = 0.8221), including with SMI populations. The Condom Attitude Scale consists of 10 items that assess participants’ attitudes toward condom use on a four-point Likert scale (1 = “strongly disagree” to 4 = “strongly agree”; range 10 to 40), with higher scores representing more positive attitudes toward condom use.8

Behavioral skills were assessed through the Condom Self-Efficacy Scale, which consists of eight items that measure perceived self-efficacy or confidence in using a condom in risk-related situations in five domains (condom confidence, condom discussion, partner resistance, regular/new partner, and impulse control). The items are scored on a four-point Likert scale (1 = “very unsure” to 4 = “very sure”; range 8-32), with higher scores indicating greater self-efficacy.19

Intervention

The goals of the experimental intervention were to: 1) increase information about HIV risk behaviors and safer sex options; 2) increase identification of personal risk for HIV and enhance motivation to engage in safer sex behaviors; and 3) enhance skills to achieve safer sex, particularly negotiating safer sex, using condoms, and good communication in relationships. Each session had “homework” assignments to be reviewed in the following sessions that aimed at increasing knowledge and motivation and practicing skills. Interactive exercises in the sessions reinforced the prevention principles that mediate the intervention.

The attention-control intervention focused on common chronic medical conditions that are comorbid with psychiatric illness (e.g., diabetes, hypertension), or conditions common to the local context (e.g., malnutrition, tuberculosis). We chose the control intervention during the ethnographic phase of our first study by asking clinic patients about their unmet service needs; their providers and our Brazilian partners ratified this choice. The control intervention also had interactive exercises and “homework” assignments.

Data analysis

We aimed to achieve 90% statistical power to detect an effect size corresponding to a difference of 6.9 unprotected sex occasions (with standard deviation [SD] of 17 sex occasions) between the two groups.7 This effect size derived from Study 1. A total of 255 participants would be required to achieve 90% power for testing the primary hypothesis with a type I error rate a = 0.05, two-tailed. There were three other considerations in the sample size calculation: participant attrition, cross-condition contamination, and variance inflation due to clustering (as the intervention sessions were provided in groups). We anticipated no more than 20% attrition and assumed that approximately 5% of the control participants would be exposed to the active intervention through peer interaction and 5% in the active intervention would fail to receive it. Further, an intraclass correlation coefficient no greater than 0.02 for an average cluster size of 10 meant recruiting a total of 448 participants (224 per group) to adjust for the above three factors. Because we planned to impute for the missing data in our primary analysis, the actual study power might be greater than 90%. Descriptive statistics were generated for the total sample and according to intervention group for demographic variables at baseline and 12 months. The a priori primary analysis, following intent-to-treat principles, compared the intervention participants to the control participants in terms of change from baseline to 12-months post-intervention in total number of vaginal and anal intercourse occasions not protected by condoms. We used generalized linear models with log link function.22 The model included the group indicator for intervention participants (vs. control), the indicator for time (12-month follow-up vs. baseline), and the interaction of the two indicators. The regression coefficient corresponding to the group-by-time interaction term estimated the logarithm of the ratio of the two study group population rate ratios (i.e., the ratio of the mean number of vaginal and/or anal intercourse occasions unprotected by condoms at 12 months vs. baseline), and thus represents the effect of the intervention on change in unprotected sex occasions. We used generalized estimating equations to account for the effect of intracluster correlations introduced by multiple assessments on the same participant.23 An over-dispersion parameter was used to account for the between-subject heterogeneity. Rubin’s24,25 multiple imputation method with 11 repeated imputations was used to impute the missing endpoint for the intention-to-treat analysis.

In secondary analyses, we evaluated intervention effects on major secondary outcomes (i.e., those measuring IMB), which included HIV knowledge, behavioral intentions, condom attitudes, and self-efficacy, by comparing the intervention group to the control group regarding the change from baseline to the 12-month follow-up. We used generalized linear models with identity function for the above analyses.22 Similar to the primary analysis, we used generalized estimating equations to account for the correlation due to repeated measurement, and the model also included the group indicator, the time indicator, and their interaction. The regression coefficient corresponding to the group-by-time interaction term estimates the difference in mean change over time between the two study group populations and, therefore, represents the intervention effect on these secondary outcomes. We also evaluated how the change in secondary outcomes impacted the change in primary outcome via linear regression analysis. We reported each of the regression coefficients (beta) of interest (i.e., the beta for the interaction terms for assessment of the intervention effect on the primary and secondary outcomes and the beta for the change in secondary outcomes while evaluating their impact on change of primary outcome), as well as its corresponding p-value. Findings with a p-value ≤ 0.05 were considered statistically significant. The data were cleaned and processed in SPSS version 20 and the regression models were run in SAS version 9.3.

Ethics statement

The institutional review boards of all involved institutions and the Brazilian Ministry of Health approved all study procedures and consent forms. A U.S. Performance and Safety Monitoring Board monitored the progress and safety of the study, and no adverse events were reported.

Results

Participants and disposition

A total of 3,811 individuals were screened for inclusion, of whom 464 were eligible for the study, agreed to participate, and underwent 1:1 randomized treatment assignment (HIV Prevention vs. Health Promotion). Figure 2 describes participant recruitment. Twelve-month rates of follow-up interviews were comparable in each study arm.

Figure 2

Flow of study participants.

A total of 464 participants were randomized. The demographic and clinical characteristics and HIV-risk behaviors of the two intervention groups at baseline did not differ statistically, indicating successful randomization (Table 1). Table 1 also includes 12-month post-intervention follow-up and HIV testing information for both groups.

Table 1

Sample demographics, baseline and 12-month follow-up characteristics and intervention adherence

	Total sample (n=464)	HIV (n=233)	Health (n=231)	Sig.*
Age, mean (SD)	43.3 (10.1)	43.5 (10.5)	43.2 (9.7)	0.746
Gender
Male	44.0	44.2	43.7	0.991
Female	56.0	55.8	56.3
Orientation
Heterosexual	92.8	93.5	92.1	0.699
Homosexual or bisexual	7.2	6.5	7.9
Race/ethnicity†
Black (negro)	20.3	17.6	22.9	0.516
White (branco)	32.3	34.3	30.3
Brown (pardo)	36.6	37.3	35.9
Other	10.8	10.7	10.8
Marital status
Single	34.7	32.6	36.8	0.765
Married/long-term relationship	46.1	48.1	44.2
Separated/divorced/widowed	18.5	18.9	18.2
Education completed
< Grade school	29.7	31.3	28.1	0.303
Grade school	12.1	12.4	11.7
High school incomplete	15.7	13.7	17.7
High school complete	24.4	27.0	21.6
Beyond high school	18.1	15.5	20.8
Diagnosis/symptoms
Schizophrenia	35.3	35.6	35.1	0.676
Bipolar disorder	21.1	20.2	22.1
Major depression with psychosis	9.3	9.9	8.7
Schizoaffective disorder/psychosis not otherwise specified	6.7	6.4	6.9
Common mental disorders	27.6	27.9	27.3
Comorbid substance use disorder	1.9	0.9	3.0
HIV-relevant history
HIV test (lifetime)	62.5	59.2	66.1	0.153
HIV+	1.3	0.9	1.7	0.405
HIV-risk behaviors
Consistent condom use (every time)
Baseline	26.7	28.0	25.3	0.514
12-month follow-up	35.3	38.0	32.5	0.303
Unprotected vaginal/anal sex occasions, mean (SD)
Baseline	12.5 (24.6)	11.4 (26.2)	13.5 (23.0)	0.367
12-month follow-up	10.5 (23.6)	9.1 (19.9)	12.0 (26.9)	0.212
Number of sexual partners, mean (SD)
Baseline	1.4 (1.3)	1.3 (0.8)	1.5 (1.6)	0.063
12-month follow-up	1.0 (0.9)	1.0 (0.7)	1.0 (1.1)	0.463
HIV+ sexual partner
Baseline	1.9	2.1	1.7	0.746
12-month follow-up	0.5	0.5	0.5	0.981
Used alcohol/drugs before sex
Baseline	21.0	19.4	22.7	0.383
12-month follow-up	17.8	15.7	20.0	0.306
Intervention attendance (dose adherence)
Eight main intervention sessions, mean (SD)	5.5 (2.4)	5.5 (2.3)	5.5 (2.5)	1.00
Three booster sessions, mean (SD)	1.5 (1.3)	1.5 (1.3)	1.5 (1.2)	1.00

Data presented as %, unless otherwise specified.

HIV = human immunodeficiency virus; SD = standard deviation.

t-statistic for continuous variables, chi-square statistic for categorical and ordinal variables.

Race/ethnicity categories typically used in Brazilian studies.

Intervention adherence

Attendance at the main and booster intervention sessions did not differ significantly between the groups (Table 1). Only two participants attended zero sessions, for an attrition rate of less than 1%. This was accomplished by randomizing only participants who came to a brief orientation session prior to the first session of each intervention. Of eight intervention sessions, the mean (SD) number attended was 5.5(±2.4) or 68.8%, which compares favorably with prior studies.4 Twenty-six percent of the participants attended all eight sessions, whereas 62.1% of the participants attended six sessions or more; 10% attended only one session. Of the three booster sessions, the mean (SD) number attended was 1.5(±1.3) or 50%. About one-third (34.7%) of the participants did not come to any booster sessions; 30.8% attended all three booster sessions.

Main outcomes

Although the magnitude of unprotected sex occasion reduction was greater in the intervention group, the intent-to-treat analysis showed no statistically significant differences between intervention groups from baseline to the 12-month follow-up assessment (beta = -0.12, p = 0.65) (Figure 3).

Figure 3

Comparison of mean number of unprotected sex occasions for both intervention conditions over all timepoints. HIV = human immunodeficiency virus.

Thus, our primary null hypothesis was retained: sexual risk behaviors in the intervention group were not significantly reduced compared to the control group. Both groups reported reductions in the number of vaginal and anal intercourse occasions unprotected by condoms from baseline to the 12-month follow-up assessment.

Secondary outcomes: Information-Motivation-Behavioral Skills

Major secondary outcomes related to the IMB model demonstrated that, compared to the control group, participants in the HIV Prevention group had significantly greater improvement in all IMB model domains from baseline to the 12-month follow-up (Table 2), including HIV knowledge (beta = 0.80, p < 0.001), motivation for safer sex (beta = 0.25, p < 0.01), HIV behavioral intentions (beta = 0.50, p < 0.001), and HIV prevention behavioral skills (beta = 0.30, p < 0.001). Further, regardless of the intervention arm, change in HIV behavioral intentions from baseline to the 12-month follow-up was associated with significantly fewer unprotected sex occasions (beta = -7.45, p < 0.001) in all participants (Figure 4).

Table 2

Comparisons of intervention to control participants in improvement from baseline to the 12-month follow-up assessment for major secondary outcomes

	Control		Intervention
Major secondary outcome	Baseline*	12-month follow-up*	Baseline*	12-month follow-up*	p-value†
Information
HIV knowledge	12.10	13.17	12.14	13.91	< 0.001
Motivation
HIV Behavioral Intentions	1.98	2.10	1.98	2.39	< 0.01
Personal Access to Condoms	0.79	0.97	0.76	1.43	< 0.001
Condom Attitude	1.83	1.87	1.74	2.11	< 0.001
Behavioral skills
Self-efficacy	1.60	1.70	1.59	1.96	< 0.001

HIV = human immunodeficiency virus.

The numerical value showed the model predicted mean for each group at each of the timepoints. The statistical model included the group indicator (intervention vs. control), the time indicator (12-month follow-up vs. baseline), and their interactions.

The p-values were used to test whether the group-by-time interaction equals zero. Any p-value < 0.05 indicated that there existed a significant difference (between the intervention and control) in mean change over time (i.e., from baseline to the 12-month follow-up) of the corresponding major secondary outcome.

Figure 4

Information-Motivation-Behavioral model mediation of unprotected sex. HIV = human immunodeficiency virus; IMB = Information-Motivation-Behavioral Skills. * p < 0.001, † p < 0.01.

Discussion

This paper reports on the outcome of a decade of NIMH-funded research to adapt efficacious U.S. interventions to reduce HIV risk behaviors among psychiatric patients attending public mental health settings in Brazil, a country where such interventions had not yet been tested. To our knowledge, this is the only randomized, clinic-provider-facilitated, HIV prevention efficacy trial to have been conducted in a public system of outpatient care for adults with mental illness. Consistent with NIMH-supported studies, we identified the most appropriate and effective evidence-based interventions for the target population; maintained fidelity to their core elements; conducted systematic assessments of the target population’s current risk factors, behavioral determinants, and risk behaviors; and focused on agency capacity, the potential for collaborations with other partners, the need for cultural proficiency, and evaluation of outcomes with rigorous efficacy testing.

This HIV trial suggests that, even in lower-resourced public systems of care, clinicians can implement interventions designed to reduce the likelihood of high-risk sexual behavior among their adult patients with mental illness. However, we did not demonstrate intervention efficacy. Despite the methodical rigor of 4 years of formative preparatory work, successful randomization, a collaborative community partnership, and evidence that the intervention model had the intended effect on the intervention group, unprotected sex occasions were reduced in both intervention conditions by similar degrees. It is of note that asking patients detailed questions about their sex lives did not result in increased risk behavior, as has often been feared, and it might be speculated that repeatedly asking detailed questions about sex and risk might have contributed to risk-reduction in both intervention groups, either through real reductions or the Hawthorne effect. The psychometric properties of the SERBAS argue for the former, although we cannot be certain that self-reported behavior reflects actual behavior.

Secondary outcomes demonstrated that the carefully adapted HIV Prevention intervention resulted in significantly greater improvements in all IMB model domains, indicating that people with mental illness acquired the skills and self-efficacy taught in this model. Moreover, one of the model’s components, behavioral intentions, was associated with fewer unprotected sex occasions across intervention conditions, demonstrating that focusing on behavioral intentions may be a productive pathway toward reduced sexual risk behavior, which suggests promise for applying the IMB model in future interventions.

The gap between successful application of a model and the ability to demonstrate real-world efficacy or “effectiveness” is an important dimension of this study. Several decision points likely contributed to this gap and highlight the most important limitations of our study. First, the patients in our intervention had unexpectedly low rates of HIV risk behavior compared to sexually active patients in our local formative and pilot work, where 78% reported unprotected vaginal or anal intercourse in the past 3 months,15 and to a national multicenter sample of adults with mental illness in Brazil (n=2,475), which similarly found a prevalence of lifetime unprotected sex of 80.3%.26 Based on this finding, our community partners urged us not to exclude individuals who were sexually active in the prior 3 months, but had no risk behavior at baseline, believing that sexual risk activity would fluctuate due to psychiatric symptoms and unstable living circumstances, and that the intervention would help people maintain safer behaviors and/or help them develop new or improved skills (e.g., condom negotiation). While this approach seemed compelling at the time, a much larger proportion (56.4%) of the RCT participants were not risky at baseline (i.e., 100% condom use) compared to 22.2% in the pilot, which compromised our ability to reach the most at-risk patients and demonstrate intervention efficacy or “effectiveness”. Future studies should involve careful ongoing monitoring practices to ensure that risk levels are commensurate with the need for an intervention and to adjust sampling appropriately.

Contamination may have played a role in between-group risk-reduction similarities, given that we conducted 4 years of formative work at a clinic in the same city as our RCT sites and also organized two well-attended city-wide dissemination conferences. Unmeasured potential contamination effects could have included changes in staff practices, such as an increased focus on the risk of HIV infection among patients both before and during the RCT. Evidence suggestive of this influence includes the observation that patients in the pilot clinic already had a high rate (54.1%) of lifetime HIV testing relative to the rate of lifetime HIV testing among psychiatric patients in Brazil nationwide, which hovers around 27%, and in Rio de Janeiro, which is 19%.27,28 Participants in the RCT had an even higher rate (62%) of lifetime HIV testing. Based on our experience, we would urge future studies to employ rapid ethnographic formative work to inform intervention development in order to reduce contamination. In addition, we also recommend using a cluster RCT design to randomly select clinics so that all patients in one site would get either the active or the control intervention.

Though our statistical tests forced us to conclude that our intervention did not significantly reduce sexual risk behavior, in spite of significant changes in the IMB model domains, the HIV intervention could have been tested to ascertain its benefits for reducing sexual risk in real-world care settings relative to standard care or brief individual-level educational interventions, which are the types of HIV prevention services most commonly found in psychiatric care settings.29 While we can accelerate research by implementing successful interventions without repeating an efficacy component, there’s also a risk that an intervention that proved to be effective in one setting will not work as well in another setting. We offered an intensive Health Promotion intervention that included a brief HIV information component. The state-of-the-science required establishment of efficacy against an attention-control (same dose) intervention instead of effectiveness against treatment as usual before taking even a rigorously adapted intervention into the public psychiatric system in Brazil. More and better guidance is needed to inform the adaptation and testing of evidence-based practices to new settings.

Our results also highlight the tension between an intervention’s effectiveness and its implementation. Accommodations to community preferences and priorities were made to increase the likelihood that this intervention would be successfully implemented at the study sites. In fact, although the study was not designed to evaluate the implementation of the intervention, we did collect information on implementation processes and outcomes that corresponded with the widely used Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework.30 Reach was assessed by the percent and representativeness of study participants. The number of participants enrolled in the clinical trial exceeded expectations, representing 65% of the eligible patients. “Effectiveness” (or efficacy) was assessed by the primary and secondary outcomes described above. Although the intervention was not effective in its primary outcome, it did achieve secondary outcomes. Further, the adapted intervention was used with fidelity by clinic providers; for both intervention conditions, adherence to the manual and competence of delivery were rated as “well done” or “very well done” more than 90% of the time. Adoption was assessed by the percentage and representativeness of participating settings and staff: 100% of providers recruited to participate in the study agreed to do so and underwent intervention training. Implementation was assessed in terms of the consistency or fidelity of delivering the intervention and adaptations made to it, and maintenance was assessed by the long-term effects on individual and setting levels. Maintenance could not be assessed even though both clinics and clinic providers wanted to continue the intervention upon conclusion of the study; however, a priori consensus prevented the clinics from doing so. While the trial did achieve the RE-AIM outcomes of reach, adoption, implementation, and potential maintenance, it failed to achieve the outcome of efficacy or “effectiveness.” Just as the interventions themselves are subject to compromise between adaptation and fidelity, research designs to evaluate effectiveness and implementation outcomes may require similar compromises, perhaps by adopting rigorous alternatives to the traditional RCT.31

Implementation science has been the focus of research only in the last decade, after the scientific community became aware of the important research-to-practice gap, where few health innovations, especially mental health and behavioral interventions, made it through the efficacy, effectiveness, and implementation pathway.32 We hope that lessons learned from our 10 intensive years of rigorous research will help future investigators conduct implementation studies in real-world settings, especially when those studies use community-based participatory research. Considerable attention will be needed during implementation trials to determine how best to balance scientific rigor with community input to make sustainable practice changes.

Disclosure

The authors report no conflicts of interest.