A protocol to develop and study the effectiveness and implementation of social skills training that improves supported housing retention for persons with serious mental illness.

INTRODUCTION: Supported housing-which offers permanent, community-based housing and supportive services-effectively addresses homelessness. Yet, many persons with serious mental illness (SMI) struggle to retain housing in these programs. Social skills-which facilitate social interactions and instrumental tasks-predict premature exits from supported housing. Though social skills training effectively improves social skills and functioning for persons with SMI, this intervention is essentially absent from supported housing initiatives.
METHODS: This study will use literature review, key informant interviews and an expert panel to adapt social skills training for supported housing, aiming to improve housing retention among SMI persons in these programs. In the Department of Housing and Urban Development-VA Supported Housing program (HUD-VASH) at VA Greater Los Angeles, we will conduct an effectiveness-implementation hybrid type I trial of the adapted intervention. Baseline and 6-month assessments (measuring social skills, mental health status, healthcare utilization, social networks, money management, and housing outcomes) will be conducted with intervention participants (n = 30) and a usual care control group (n = 20). We will use the generalized linear mixed model (GLMM) to compare change over 6 months between groups, capturing the intervention's effectiveness on factors strongly associated with housing retention. Qualitative data and surveys with staff, leadership, and participants will gather data on factors relevant to the intervention's future implementation in routine care. DISCUSSION: Few effective psychosocial interventions for persons with SMI have been adapted or studied in supported housing initiatives. To our knowledge, this is the first study to adapt and study the effectiveness and implementation of social skills training in supported housing programs.

Introduction

Supported housing−which offers permanent, community-based housing and supportive services−is an evidence-based practice to address homelessness among adults with serious mental illness (SMI) [1,2]. The Department of Housing and Urban Development (HUD)-VA Supported Housing (HUD-VASH) program is the nation's largest supported housing initiative [3]. Yet, 25% of HUD-VASH participants exit the program yearly [4], often returning to homelessness [5]. Among persons with SMI, social skills are an important determinant of premature exits from supported housing [6]. Social skills encompass expressive, receptive, conversational, and assertiveness communication, as well as instrumental activities of living (e.g., money management) and illness self-management (e.g., medication adherence). [7] We conceptualize SMI broadly [8], encompassing mood, psychotic, and anxiety disorders resulting in significant functional impairment [9]. Though social skills training effectively improves social skills and functioning for persons with SMI [10], this intervention is essentially absent within supported housing.

Among persons with SMI, there are strong relationships between social skills and functional outcomes, like vocational activities. Unfortunately, many persons with SMI have deficits across a breadth of social domains. For this vulnerable group, social skill deficits persist throughout illness [10]; these deficits stem, in part, from cognitive dysfunction among persons with SMI [11]. Social skills training addresses social skill deficits to improve community functioning [12]. For persons with schizophrenia, these interventions have a moderate mean effect size on community functioning; a large weighted mean effect size on social skills; and a small to moderate mean effect size on symptoms [12]. Of note, these effects add upon improvements in positive and negative symptoms−and resultant benefits in social skills−produced by traditional pharmacotherapy, as nearly all patients in trials of social skills training also receive medications [7].

Within its psychosocial rehabilitation programs for persons with SMI, the VA has nearly completed a national rollout of social skills training. This rollout employs a paradigm developed by Bellack and colleagues [13], training groups of persons with SMI in social skills. At each session, group leaders provide rationale for a given skill, solicit relevant experiences from participants, model the skill, and engage participants in role-plays with feedback.

The VA social skills training rollout provides instruction and support to clinicians who work with persons with SMI. However, only a paucity of trained providers work in VA's homeless program; even fewer work in HUD-VASH. The intervention does not include training in instrumental skills that are critical for housing retention, e.g., money management. Moreover, social skills training's benefits derive from behavioral instruction that simulates lived experiences [12]; there is a disconnect between the lived experiences of many persons with SMI who engage in VA's psychosocial rehabilitation programs (who generally live in institutions or with family) and the needs of their homeless counterparts (who often have fewer supports and greater living skill impairments).

Current social skills training for persons with SMI aims to improve social functioning, not housing retention. Homeless persons have distinct needs, e.g., for shelter [14], which compete with their ability to access traditional rehabilitation services. As compared to traditional social skills training, persons with SMI in supported housing may benefit more from an intervention that addresses applications of social skills that are specifically relevant to independent living. Moreover, role-plays of scenarios relevant to supported housing will be more acceptable to and useful for this population than traditional social skills training. To our knowledge, this protocol is the first effort to adapt, implement, and study the effectiveness and implementation of social skills training in supported housing settings.

Materials and methods

Study aims

The first aim of this study is to develop an adapted social skills training intervention that improves retention in independent housing among persons with SMI in HUD-VASH, by combining and tailoring effective social skills training interventions. The second and third aims comprise an effectiveness-implementation hybrid type I trial of the adapted intervention. Specifically, the second aim studies the adapted intervention's effectiveness on factors strongly associated with housing retention among persons with SMI in HUD-VASH. The third aim examines barriers to and facilitators of future implementation of the adapted intervention in routine HUD-VASH care. This protocol is approved by VA Greater Los Angeles' Institutional Review Board and the trial is registered at clinicaltrials.gov (NCT03646169).

Setting

On a single night in January 2017, Los Angeles had more homeless Veterans (4,476) than any U.S. city [15]. Over three-quarters (76%) of these Veterans were unsheltered, living on the streets, abandoned buildings, or other places not intended for human habitation [15]. VA Greater Los Angeles’ HUD-VASH program is the largest in the nation, with resources to house and provide supportive services to 6376 Veterans throughout metropolitan Los Angeles. Among Veterans in HUD-VASH at this site, most (91%) are male and over half (56%) self-identify as ethnic minorities [16].

HUD-VASH

HUD-VASH is the VA's “Housing First” program, offering independent housing with community-based supportive services, including non-mandated referrals to medical and mental health care [1,2]. HUD-VASH serves VA healthcare eligible Veterans who meet HUD-specific income requirements for a Housing Choice voucher (a national-level financial subsidy for housing), who are homeless or on the verge of homelessness, and who have an identified need and willingness for case management [17]. Participating Veterans pay 30–40% of their monthly income (often from disability benefits or unemployment) towards the rent of apartments in the community and the Housing Choice voucher pays the remainder of rent due. Though Veterans receive case management and referrals to clinical and social services (within and outside VA), neither treatment nor sobriety is mandated [4,17].

A team of social workers, nurses, and consumer providers (known as “peer support specialists”) delivers usual HUD-VASH care. Each team works with Veterans seeking housing in a specific geographic region of Los Angeles (“Service Planning Area”). As part of usual care, teams perform diverse tasks, including: outreaching to homeless Veterans; screening Veterans on HUD-VASH eligibility criteria; assisting with application materials; guiding apartment searches; securing temporary housing before apartment move-in; and performing case management using Housing First principles, i.e., flexible supports and linking Veterans to care without treatment mandates.

With a national FY18 budget of >$408 million, HUD-VASH is the linchpin of the VA's strategic plan to end Veteran homelessness. These substantial resources, superimposed upon the high rates of Veteran homelessness [15], provoke concern about the 25% of participants who exit the program each year [4].

Conceptual frameworks

This study is guided by a theoretical framework and implementation science framework: the Behavioral Model for Vulnerable Populations [18] and the Consolidated Framework for Implementation Research (CFIR), respectively [19]. The Behavioral Model for Vulnerable Populations (Fig. 1) models health service utilization for vulnerable populations; it reflects the interplay of contextual and individual characteristics for homeless persons [18]. At the contextual level, it reflects community factors, policies, and the organization of services; at the individual level, it identifies factors that predispose individuals to access services (e.g., demographics), which interact with enabling factors (e.g., social skills and cognition) and needs to influence behaviors and ultimately outcomes. Of note, this framework traditionally models access to health services among vulnerable groups. [18] We adapted the framework to model the relationships between diverse factors associated with housing retention and mental health outcomes for homeless Veterans.

Fig 1

Behavioral model for vulnerable populations18, adapted for this study protocol.

To complement this framework, we use the CFIR to characterize factors influencing the intervention's core implementation outcomes, e.g., adoption, feasibility, and sustainability. The CFIR has five domains [19]: 1) intervention characteristics (intervention source, robustness of support from research evidence and clinical experience, relative advantages vs. alternatives, adaptability to local context, complexity, quality of design/packaging, and cost); 2) the outer setting (socioeconomic and political context of the organization implementing the intervention); 3) the inner setting (organizational level structural, political, and cultural contexts influencing implementation); 4) the individuals involved (patient and provider knowledge and beliefs about the intervention, and other personal attributes); and 5) the process of implementation (planning, engaging, executing, and reflecting and evaluating).

This study's aims correspond to constructs of the CFIR process domain [19]. The construct of planning is aligned with developing the adapted intervention. Studying the intervention's effectiveness corresponds to the constructs of engaging and executing. In turn, studying barriers to and facilitators of implementation of the intervention in routine care is aligned with the constructs of reflecting and evaluating.

Participants, measures, procedures, and analyses by aim and CFIR construct

Aim 1: planning

To develop an intervention that tailors and combines effective social skills training paradigms, with the goal of improving housing and mental health outcomes for homeless persons with SMI, we will use: literature review, key informant interviews with national leaders in the delivery of these interventions; and an expert panel, following the RAND/UCLA appropriateness method [20], that convenes local and national homeless program stakeholders. The literature review will begin with meta-analyses and systematic reviews on social skills training for persons with serious mental illness. We will use keywords in PsycINFO and MEDLINE to identify relevant literature, including: social skills training, skills training, serious mental illness, severe mental illness, and schizophrenia. We will ask authors of these articles and other clinical leaders in the area for additional salient literature on this subject (including journal articles, clinical manuals, and unpublished documents).

To complement this literature review, we will conduct semi-structured interviews (∼30 min, interview guide available as Appendix A) with clinicians, administrators, and researchers (n = 15–20) who are experts in psychosocial rehabilitation for persons with SMI; initially, we will conduct interviews with authors of the meta-analyses and systematic reviews identified in our literature review. Informants will be asked to discuss factors associated with the effectiveness of social skills interventions in settings for persons with SMI (who may or may not have experienced homeless). We will ask informants to identify skills they perceive as highly relevant (or of limited relevance) to housing retention and mental health for homeless adults with SMI. Barriers to and facilitators of implementing social skills training in HUD-VASH will be explored, as well as additional effective practices that address factors related to social skills for this population. We will ask interviewees for the names of other knowledgeable persons in these areas, subsequently contacting named individuals for additional semi-structured interviews. All interviews will be recorded and professionally transcribed; we will systematically create summaries of the interviews that correspond to domains in our interview guide [21]. Matrix analyses methods will be used to examine these data [22] and synthesize key themes by domain; each interviewee will comprise a row of the matrix and domains will comprise the columns. The matrix will provide a streamlined framework to systematically identify similarities, differences, and trends among the interviewees. [22] Themes identified will be used to synthesize an executive summary of interview findings.

To identify key design elements of the adapted intervention that are grounded in strong evidence and that can be adapted for delivery by HUD-VASH team members, we will use the RAND/UCLA appropriateness method [20] to convene a panel of experts (n = 10–15) in homelessness and/or psychosocial rehabilitation. Reputation-based snowball sampling [23]−beginning with a list of experts identified by the authors−will be used to recruit a panel with demonstrated experience in the rehabilitation of persons who have a history of homelessness and/or serious mental illness. Prior to the panel, participants will receive the executive summary derived from our key informant interviews, as well as relevant literature on social skills training and the interplay between social skills and housing attainment/retention. The key informant interviews will allow us to identify relevant content (i.e., social skills) to include in the intervention. We will include this content in a pre-panel survey distributed to panelists; using a Likert scale (from 1 (highest support) - 9 (lowest support)), panelists will rank each content item (social skill) in two domains aligned with the CFIR intervention characteristics constructs [19]: 1) evidence strength and quality, or likelihood to influence housing retention for Veterans engaged in HUD-VASH; and 2) adaptability for delivery by HUD-VASH team members.

The panel itself will aim to build consensus around items with rating dispersion in the pre-panel survey. We will not discuss items with pre-panel survey consensus (positive or negative). For each discussed item, each panelist will receive a document that lists each of his/her initial rankings, as well as mean rankings. The evidence surrounding each content area with rating dispersion will be discussed; mean/standard deviation of rankings will be presented to the group. Panelists will discuss these modules and subsequently re-rate the items discussed on the same scales and items used in the pre-panel survey. Content items scored by 75% of panelists as 1–3 in all ratings will be included in the final version of the intervention [20].

Aim 2: engaging and executing

Intervention

Though the intervention's specific content and format will be developed in this study's planning phase, we expect that it will resemble other effective social skills interventions for persons with SMI [24,25], but with adaptations relevant to the setting and context of supported housing. Fig. 2 depicts the projected structure of the intervention. Specifically, we anticipate a 3-month intervention, combining a single, individual goal-setting session (to orient participants to the intervention and identify his/her personal housing goals) with longitudinal classroom-based sessions−delivered by dyads of HUD-VASH team members−with weekly, 1-h sessions to groups of 10–15 Veterans who are recently housed via HUD-VASH. The group sessions will use behavioral instruction, e.g., interactive role modeling, to teach social skills relevant to independent housing. Interventionists will audio-record their individual goal-setting sessions and groups to assure fidelity to the social skills paradigm (using fidelity checklists that are well-established in effective social skills training interventions [24]). After the group-based intervention, we anticipate booster sessions (for 8 weeks) to bridge skills learned to real-world apartment living. The intervention will complement usual HUD-VASH care.

Fig. 2

Anticipated structure of the intervention.

Study design and participants

We will use a hybrid type I effectiveness-implementation design [26], testing the effectiveness of the intervention on factors strongly associated with housing retention among HUD-VASH participants with SMI, while gathering qualitative data relevant to the intervention's future implementation in routine care. Specifically, in this study's engaging and executing phase, we will examine the effects of the intervention among 30 participants with SMI who are recently (within 3 months) housed on one of two HUD-VASH teams at the VA Greater Los Angeles. We will compare the intervention group with a control group of 20 persons with SMI receiving usual care on any of VA Greater Los Angeles' other six HUD-VASH teams.

Measures

In-person assessments and medical record review at baseline and 6-months (Table 1) will be used to study the intervention's effectiveness on factors strongly associated with supported housing retention (e.g., social skills, service use, social interaction, and money management). Demographic data will be collected at baseline, including age, gender, race, ethnicity, and marital status. Diagnostic information (SMI diagnoses, substance use disorders (SUDs), and general medical diagnoses common among homeless persons [27]) will be abstracted from the medical record at baseline. Mental health (at baseline and 6 months) will be captured with the 15-item Illness Management and Recovery Scale-Client Rating [28], a self-reported measure of recovery-oriented mental health status. Additional survey data collected at baseline and 6-months will include the Social Skills Performance Assessment (SSPA) [29], a measure of social skills among persons with SMI; the Service Use and Resources Form (SURF) [30], which collects information on inpatient and outpatient service utilization for psychiatric and medical issues (inside and outside VA); the Social Capital Resources Generator [31], which assesses social network resources among persons with SMI; the money management domain of the self-reported Independent Living Skills Survey (ILSS) [32], a validated community functioning scale for persons with schizophrenia. At both data collection points, information from the SURF will be complemented with review of the VA medical record, capturing service use over the past 6 months, including the number of: Emergency Department visits, inpatient bed days, primary care visits, SUD treatment visits, and ambulatory mental health visits. Housing (at baseline and 6 months) will be captured with the Residential Time-Line Follow Back (TLFB) inventory [33], a retrospective event history of consumers' residences over six months, generating percent days in stable housing.

Table 1

Measures relevant to studying the intervention's effectiveness.

Domaina	Variable	Medical Record Review	In-Person Assessment
Predisposing	Demographics¶		Age, gender, race/ethnicity, marital status
Enabling	Social skillsa		SSPA
Need	SMI diagnoses¶	Chart diagnoses
	Mental health status		Illness Management and Recovery Scale-Client Rating
	SUD diagnoses¶	Chart diagnoses
	General medical diagnoses¶	Chart diagnoses
Behaviors	VA and non-VA service use	VA service use over the past 6 months, # of: Emergency Department visits, inpatient bed days, primary care visits, SUD visits, ambulatory mental health visits	SURF, capturing VA and non-VA service use over the past 6 months
	Social interactions		Social Capital Resource Generator
	Money management		ILSS (money management domain)
Outcomes	Housing		Residential TLFB Inventory

Note: SMI = serious mental illness; SSPA = Social Skills Performance Assessment; SUD = substance use disorder; VA = Veterans Administration; SURF = Service Use and Resources Form; ILSS = Independent Living Skills Survey; TLFB = Time-Line Follow Back.

Measures performed at baseline only. All other measures performed at baseline and 6 months.

Domains stem from the Behavioral Model for Vulnerable Populations.

Analyses

To study the intervention's effectiveness on factors strongly associated with housing outcomes, we will use the generalized linear mixed model (GLMM) to analyze differences between baseline and 6-month assessments for the intervention vs. control groups. We hypothesize that the intervention group will have more improvements in social skills; more appropriate service use (primary care, ambulatory mental health care, and substance use disorder treatment), and more improvements in social interactions and money management. We will analyze differences in housing retention outcomes, but view such as exploratory. In using GLMM for hypothesis testing, we will control for baseline differences between the two groups (predisposing, enabling, and need factors) by including them as covariates, and also determine if these covariates are associated with increased or decreased treatment effects on our core outcomes.

In power analyses, we assume a sample size of n = 26 in the intervention group and n = 17 in the comparison group, accounting for drop out of 12.5% over 6 months (consistent with the HUD-VASH exit rate) [4]. For analyses of differences between the intervention and control groups−in the degree of change from baseline to study exit−this design provides sufficient power (>0.80) to detect treatment effects as small as f = 0.20 [34], with the assumption that scores at baseline and study exit are correlated with r = .50. Cohen [34] describes f = .20 as a medium effect (f = .10 is a small effect, f = -0.25 is a medium effect, and f = 0.40 is a large effect). If the two groups have identical scores at baseline, this effect size is equivalent to a group difference at the end of the study of d = 0.8. Overall, the study is sufficiently powered to detect medium effects and to provide the effect size estimates needed to plan larger follow-up studies if the treatment effect is clinically relevant, but smaller than what this study can reliably detect.

Of note, a meta-analysis of social skills training [10] for persons with schizophrenia described a medium (f = 0.13) mean effect size on community functioning, e.g., social relationships, and a large weighted mean effect size (f = 0.25) on social skills. This study is thus powered to detect changes in social skills; as it is underpowered to detect changes in community functioning outcomes (like housing retention) we view such analyses as exploratory and focus on studying the intervention's effects on factors strongly associated with this outcome.

Aim 3: reflecting and evaluating

We will use mixed methods to examine contextual factors and stakeholder perspectives that are likely to influence future implementation of the intervention in routine HUD-VASH care. We will administer the Perceived Characteristics of the Intervention Scale (PCIS) to all staff (n = 40) on the two HUD-VASH intervention teams. The PCIS is a 20-item instrument that captures perceptions of an intervention in 9 domains: its relative advantage vs. alternatives, compatibility with existing values/experiences, complexity, potential for use in a trial, ability to achieve observed results, potential to be modified, concerns that need to be addressed to accomplish implementation, degree to which information about the intervention can be codified and transferred across contexts, and available supports, e.g., training or supervision [35].

Next, we will engage HUD-VASH leadership (n = 5) in individual, semi-structured interviews. Data from HUD-VASH staff (n = 20), including the interventionists, will be collected in 3–5 focus groups. We will use criterion sampling [36], using staff disciplines as the criterion, to recruit a breadth of staff. We will also recruit a purposive sample of intervention group participants (n = 15) for individual, semi-structured interviews, maximizing the sample's variation on age and gender. Of note, though these sample sizes are typical for thematic saturation in qualitative analyses in implementation science studies; [36] it is possible that the proposed sample size is inadequate for saturation. If needed, additional purposive sampling will ensue.

The interview and focus group guides will be informed by the CFIR and results of the PCIS. In the domain of intervention characteristics, will seek staff/leadership perspectives on the quality and validity of evidence linking the intervention to our desired outcome. We will assess the intervention's perceived complexity, along with barriers and facilitators to its adoption and feasibility (both in the context of this hybrid trial and with regards to sustainability in routine care). We will seek input about ways to change the intervention to increase its fit in HUD-VASH. Veterans will describe their satisfaction with the intervention, their perceptions of the intervention's utility for housing retention, any challenges they faced with it, and any suggestions they have for improvement.

In the domain of inner setting, staff/leadership will describe receptivity of the HUD-VASH teams to the intervention. We will explore perceptions of the degree to which the intervention melds with current processes and practices. We will inquire about factors or efforts needed to support the intervention's sustainability in routine care, e.g., budget increases, gathering input about ways to obtain these resources.

In the domain of characteristics of involved individuals, we will assess stakeholder attitudes, beliefs, and knowledge about the intervention. We will ask staff/leadership about their perceived capabilities to implement and use the intervention. Veterans will be asked if they would recommend the intervention to peers.

We will calculate means and standard deviations of PCIS domains; these data will inform the qualitative interview and focus group guides.

All qualitative data will be audio-recorded and professionally transcribed. We will use ATLAS.ti [37] for data analyses, employing the CFIR Codebook [38] as a top-level codebook. We will supplement these codes with codes that derive from our interview guide and emergent themes from the data. We will compare transcripts among stakeholder groups and use analyses to characterize barriers of and facilitators to implementation of the intervention.

Discussion

Prior research substantiates the positive health, housing, and psychosocial outcomes of persons with SMI engaged in supported housing [1,[39], [40], [41]]. Yet, few interventions address challenges faced by the many persons who enroll in supported housing, but struggle to attain or maintain housing [[4], [5], [6]]. Particularly for individuals with SMI, there is a pressing need to effectively implement innovative services that complement supported housing services and address mutable but overlooked factors predicting retention in independent housing and mental health. To our knowledge, this is the first study that uses a systematic approach to adapt social skills training to the setting and context of supported housing, with the aim of improving housing retention. Surprisingly few effective psychosocial interventions employed in settings for persons with SMI have been adapted for supported housing initiatives; even fewer have employed hybrid trial designs [26] to study their effectiveness and implementation.

This study will use literature review, key informant interviews, and an expert panel employing the RAND/UCLA appropriateness method [20], in sequence, to combine and tailor effective social skills training interventions to the setting and context of supported housing. By employing an effectiveness-implementation trial [26], under real-world effectiveness conditions, we will glean valuable information about this intervention's effectiveness on factors strongly associated with housing retention for persons with SMI in supported housing. Moreover, guided by the CFIR [42], we will gather important information relevant to this intervention's implementation and sustainability in routine care. Applying this widely accepted framework will allow us to standardize much of the data collection and analyses relevant to our implementation aim. By testing and implementing a tailored social skills training intervention in the nation's largest supported housing program, we hope to advance the science surrounding interventions that can help vulnerable persons with a history of SMI and homelessness retain housing.