Health and social outcomes in the Housing First model: Testing the theory of change.

Background: Homelessness continues to grow globally. The Housing First (HF) model offers immediate access to housing and support services without preconditions and has a growing body of evidence documenting its effectiveness at ending homelessness. HF has a robust theory of change that hypothesizes how unique program components (i.e., immediate access to housing, separation of services from housing, client choice, etc.) drive positive social and health changes over time. We advance the understanding of how HF causes client improvement by empirically testing this program's theory of change.
Methods: Using a unique longitudinal quantitative data from the large Canadian At Home/Chez Soi Housing First trial we used path analysis to test the theory of change for Quality of Life, Crisis related events or service utilization, and Recovery.  Program pathways and health and social outcomes were measured at enrolment, 6-, 12- and 24-months post-enrolment. Findings: Most hypothesized pathways were confirmed with path analysis.  Confirmed pathways for two outcomes- Quality of Life (QOL) and Recovery - were similar. Health and social consultations at enrolment, health status at 6- and 12-months post enrolment, and social connectedness at 12-months were important predictors of the 24-month outcomes of Quality of Life and Recovery, but not for Crisis related events or service utilization. Interpretation: This analysis directly responds to recent calls for more empirical evidence about intervention mechanisms. Ensuring linkages to health and social service consultations for clients, supporting clients' engagement with family and community, and enabling clients to improve or maintain good health will drive better longer term client outcomes within Housing First. Funding: Funding Mental Health Commission of Canada.

Evidence before this study

To review the literature published up until Dec 2021 on tests of the Housing First theory of change for outcomes of recovery and quality of life we searched Web of Science for all studies with the terms “Housing First” and “theory”.  Two papers were identified, one promoting theory-informed evaluations which was not a test of the Housing First theory of change.  The second paper, a meta-modeling of Housing First, was a test of the theory of change for the outcome housing tenure.  We did not find any papers testing the theory of change for any of the three outcomes of this study.

Added value of this study

Like many evidence-based interventions, too little focus has been given to empirically confirming program mechanisms and causal pathways of health and social interventions. To the best of our knowledge this is the first study to empirically test, using longitudinal data over 24 months of program participation, the theory of change for Quality of Life, Recovery and Crisis Related Events or Service Utilization. Program ingredients such as activities to increase social connectedness and intervention components that support improvements in physical and mental health within the first year of program participation are critical program strategies that improve program participants’ Quality of Life and Recovery; the latter intervention component also reduces Crisis related events or service utilization.

Implications of all the available evidence

The benefits of empirically testing a program's theory of change are to confirm hypothesized program pathways and identify strong drivers of successful outcomes that might warrant added emphasis while delivering the program to maximize program performance. Coupled with a strong theory of change and evidence that programs implemented with high fidelity drive better outcomes, this empirical test of the theory of change adds a new piece of the puzzle about causal mechanisms for program strategies. Yet, this type of analysis is rarely performed.  Available program evaluation data can and should be used to reveal critical drivers of program success.

Alt-text: Unlabelled box

Introduction

Estimates conservatively place the number of homeless people in OECD countries at 1.9 million in 2019 with actual numbers up to 20% higher. This represents a costly humanitarian tragedy and a violation of the principle of housing as a human right. Compared to the traditional or “housing ready” approach to housing and homelessness, the increasingly prevalent Housing First model offers immediate access to housing and support services without preconditions, and has been found to be both successful at ending homelessness and cost-effective.2, 3, 4, 5

Housing First approach is founded on a rights-based philosophy, which provides clients with immediate access to permanent housing and mental health support services, based on the expectation that behavioral change will be more successful in the longer-term if it is encouraged within a safe and supported environment., At heart, the Housing First model is a consumer-driven approach which emphasizes client choice and control within a trauma-informed, harm reduction framework. Clients are provided immediate housing and mental health support services, often Assertive Community Treatment (ACT) or Intensive Case Management (ICM). The former supports those with high and complex needs (e.g. diagnosed psychotic or bipolar disorders), whilst ICM serves all other clients. The goal of this intensive and client-centered support is to promote recovery with the aim of ending homelessness and facilitating better health and social outcomes. Housing First, like many evidence-based mental health interventions, benefits from a rich and growing body of research supporting its effectiveness yet few studies have empirically demonstrated the causal mechanisms of its program strategies.,

The Housing First program has a robust theory of change as briefly described. Access to housing and mental health support services soon after program enrolment triggers a cascade of events (Figure 1).,, Upon enrolment, service teams create collaborative housing and care plans and facilitate access to health services and income benefits, all guided by the choices and preferences of program participants and supported by strong working alliance with support workers. With increased exposure to supports, income and services over time, the theory of change predicts an increase in participant illness management and subjective well-being and that participants will be able, with support, to successfully continue to be housed.

Figure 1

Process of change facilitated by enrolment into a housing first intervention.

Housing and mental health stability are hypothesized to further enable clients to engage in personal interests including employment, education and social or community participation over time. Correspondingly, quality of life improves, recovery is supported and utilization of high cost services such as hospitalization, incarceration or emergency department visits are expected to decline. It is also recognized that the recovery process is not linear and not all participants will progress equally toward improved outcomes.

Studies and systematic reviews of Housing First programs have consistently demonstrated high levels of housing stability and program retention across a variety of countries and populations with a mix of findings on non-housing outcomes, such as quality of life, loneliness, social isolation, health care utilization and criminal justice outcomes in diverse service contexts.,, Systematic reviews and studies of multiple programs based in the USA and Canada have demonstrated that high fidelity to key program components is associated with better participant outcomes including housing stability, quality of life, and community integration., These diverse findings suggest that there is a need to pay greater attention to the underlying program theory of Housing First to understand how the intervention works, for whom and under what conditions for outcomes beyond housing tenure.

And while this highlights the importance of providing adequate ‘doses’ of the program's key ingredients, there have been no studies to date that empirically test, quantitatively and longitudinally, how participants engage with and benefit from specific program “ingredients” to advance their process of recovery within a Housing First program. In the present study, we leverage data from the At Home/Chez Soi pragmatic field trial for individuals who have severe mental illness and are homeless that demonstrated high levels of program fidelity to empirically test hypothesized pathways based upon the program's theory of change.

Methods

The study protocol for the At Home/Chez Soi (AH/CS) study - a Canadian five-site randomized trial of Housing First - has been described extensively in previous publications., A total of 2148 participants were enrolled in AH/CS from October 13, 2009 through June 30, 2011. Of these, 1158 were randomized to receive the housing intervention (HF) plus ICM (n = 689) or ACT (n = 469) supports, depending on their level of mental health support needs. Participants had a baseline and four follow-up interviews at 6, 12, 18 and 24 months after randomization. Exclusions due to deaths, withdrawals or losses during follow-up (4.8%) left a sample size of 1103.

Diagrams depicting theory of change pathways

The authors, including the originator of the Housing First program (ST), drew path diagrams based upon underlying program theory, program logic models, and knowledge of the program, to represent the hypothesized inter-relationships between variables at early program participation (0–6 months) and at 12 months, with the 24 month outcomes of interest.  These pathways were the best approximation of the Housing First theory of change given the data available. These pathways formed the basis of the models tested through path analysis (a special case of structural equation modeling (SEM)) for three outcomes, separately (see Appendix 1 Figure 1, Figure 2, Figure 3 for the models to be tested).

Figure 2

Unstandardized beta coefficients and odds ratios from the path analysis model* of theory of change for Quality of Life at 24 months follow-up for the At Home/Chez Soi study. Dashed lines represent a pathway to a binary outcome and solid lines represent paths to continuous outcomes.

Figure 3

Unstandardized beta coefficients and odds ratios from the path analysis model* of theory of change for Recovery at 24 months follow-up for the At Home/Chez Soi study. Dashed lines represent a pathway to a binary outcome and solid lines represent paths to continuous outcomes.

Variables and statistical analyses

Three outcomes were evaluated at 24 months.  Quality of Life was measured via the Quality of Life Inventory-20 total scores where participants were asked how they felt (from terrible to delighted) about family, finances, leisure time, living arrangements, safety, and socializing with higher scores representing better quality of life., We used the Recovery Assessment Scale total scores to capture the extent to which our participants felt they had gained control, meaning and purpose in their lives. Higher scores represent better perceptions around recovery. For crisis related events or service utilization we created a composite of events that occurred in the past 6 months. We summed the following yes/no events for the past 6 months: Event 1- at least one calls to a crisis line, 911 or other health line + Event 2- at least one hospital emergency room visit + Event 3- at least one episode of being detained by the police or being taken anywhere by the police (other than a police cell) (e.g., hospital, shelter, or a residence) + Event 4- at least one arrest. Higher scores indicated more crisis oriented events in the past 6 months. Variable names and descriptions of the other variables in our path models are provided in Table 1. In several instances we created composite variables using factor analysis, in part, to promote statistical efficiency.

Table 1

At Home/Chez Soi housing first variables in the theory of change path analysis model for baseline, 6, 12 and 24-month data.

Interview Point	Domain	Variable Description
Baseline	Hope for Change	Six items representing personal confidence and hope for the future from the Recovery Assessment Scale.20 Higher scores represent greater hope.
6 Months	Access to Public Benefits	Total income in the past 3 months (Canadian dollars) expressed in 1000 s.
	Working Alliance with Service Worker	Participant Working Alliance Scale total score capturing the degree to which the client perceives a good working relationship with the worker the client works closely with and the degree to which the client agrees with therapy goals.29 For participants who indicated they had no contact with any worker between enrolment and the six-month interview (<5%), they were assigned the lowest possible scale score (Never in all questions) under the assumption that all participants in the Housing First treatment arm are assigned a case manager or see a health or social service provider immediately after enrolment and, therefore, lack of contact might reflect a “lack of alliance” with the worker. Higher scores represent better perceptions of the working relationship with a service provider.
	Health or social service consultation	If the participant sought professional consultation from a health provider (e.g., doctor, nurse, psychiatrist) or social services provider (e.g., housing worker, a social worker, justice workers) in the past month then they received a "yes" for this composite variable.
	Subjective Well-Being	Short Form-12 Mental Component Summary score,30 with higher scores representing better well-being.
	Participation in Meaningful Activities.	Composite: Participant is sometimes/often/almost always involved in meaningful activities that are satisfying to them (Item 13 of the Multnomah Community Ability Scale) or worked at any job in the past 3 months or started a training program or courses in the past 3 months
12 Months	Social Connectedness	Binary Composite: For participants with a moderately extensive/extensive/very extensive social support network that includes family (Item 12 of the Multnomah Community Ability Scale) or participant with someone that they can share sensitive personal information with (Item 21 Quality of Life Inventory-20 Scale)18 they were assigned a 1. Others were assigned zero.
	Composite of poor physical or mental health	Composite:  Three measures, EQ5D Overall Visual Analog Scale19 reverse coded, Colorado Symptom Index scores,31 and Global Assessment of Individual Needs Short Screener (GAIN-SS) scores32 on substance use problems in the past month  were combined via principal component analysis.
24 Months	Quality of Life	Quality of Life Inventory-20 total scores,18,19 where participants asked how they felt (from terrible to delighted) about family, finances, leisure time, living arrangements, safety, and socializing with higher scores representing better quality of life.
	Crisis related events or service utilization	Composite: Crisis oriented service utilization or events that occurred at least once in the past 6 months. The sum of the following yes/no events: Event 1- In the past 6 months, at least one calls to a crisis line, 911 or other health line + Event 2- In the past 6 months, at least one hospital emergency room visit + Event 3- In the past 6 months, at least one episode of being detained by the police or being taken anywhere by the police (other than a police cell) (e.g., hospital, shelter, or a residence)  +  Event 4- In the past 6 months, at least one arrest.
	Recovery	We used the Recovery Assessment Scale total scores to capture the extent to which our participants felt they had gained control, meaning and purpose in their lives.20 Higher scores represent better perceptions around recovery.

The path analysis was conducted by fitting generalized structural equation models (GSEM) without latent variables. We tested the theory of change, separately, for each of the three 24-month outcomes of interest. GSEM allows for modeling of both exogenous (modeled as independent variables only) and endogenous (modeled as a dependent variable at least once) variables in the pathway. In our analyses, exogenous variables are income and hope for change at the time of enrolment. All other variables act as either intermediate or mediator variables or outcomes.

In GSEM, it is possible to model continuous (normal or non-normal), count, ordinal or categorical (binary, multinomial) outcomes using the appropriate probability distribution. For the continuous outcomes of quality of life and recovery, we considered the normal distribution and linear regression models after checking normality of residuals and report unstandardized beta coefficients with 95% confidence intervals. For the count outcome of crisis-related events or service utilization, we considered the negative binomial regression model and report unstandardized rate ratios and 95% confidence intervals. For binary variables (such as social participation or health or social service consultation), we considered the binomial distribution and logistic regression models, and report unstandardized odd ratios and 95% confidence intervals.

Missing data were handled using multiple imputation by chained equations (MICE). We created 20 imputed datasets following the recommendation to set the number of imputations equal to the average percentage of missing data (Appendix 2, in our study (16.0%) and performed GSEM in each imputed dataset. The imputation model included all analysis variables, as well as age at enrollment, indicator of having moderate needs, male gender, ethno-racial indicator, indigenous indicator as well as some of the auxiliary variables (Appendix 2 and 3).  Reasons for missing data included skipping an interview, and refusal to answer a question.

Multiple imputation and pooling of the GSEM estimates across imputations according to Rubin's rules were conducted using Stata 15 (StataCorp. 2017. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC). Syntax codes are included in Appendix 3. All statistical tests were two-sided and statistical significance was defined if p-values were 0.05 or less.

Ethics

All participants provided informed consent and Ethics Board approval was received from each of the relevant university/site REBs.

Role of the funding source

The study was funded by the Mental Health Commission of Canada and had no role in study design, data collection, data analysis, data interpretation, or writing of this manuscript.

Results

Description of program experiences and distribution on analytic variables

The characteristics and experiences of treatment group participants across the five Canadian sites by follow-up time period are presented in Table 2. Within the first six months of enrolment, the average past 3-month income of participants was $2300 (CAD) likely reflecting a high level of access to public benefits.  The majority were highly connected to health or social services as evidence by 71% reporting engagement with mental health or addictions treatment and 80% participating in meaningful activities or employment or training program.  Working alliance at the 6 month point reflects a reasonably strong collaboration and agreement between clients and providers with clients perceiving, on average, that they often agree with their workers.  These patterns are consistent with the high ratings on fidelity assessments for At Home/Chez Soi At 12 months, 85% had a family member or confidant in their social network. At 24 months, the mean quality of life score was 89 (range 20–140).  The mean total score for recovery at 24 months was 85.2 (range 38 - 110). For crisis related events, approximately half of the sample had no such events or service use while 5% had 3 or 4 types of service utilization.

Table 2

Sample characteristics of At Home/Chez Soi program participants in the treatment arms for program theory analyses.

	N	% or Mean (Standard Deviation)
Baseline
Age at enrolment	1103	40.7 (11.1)
Men	1103	66.2
Ethno-racial	1103	24.6
Indigenous	1103	22.6
Intensive Case Management (ICM)_Arm	1103	59.7
Hope for Change	1013	24.9 (5.2)
6 months
Income for the past 3 months (per 1000)	1065	2.3 (1.5)
Working Alliance	781	61.3 (19.2)
Health or Social Service Consultation	997	70.9
Participation in meaningful activities	1071	79.7
Subjective well-being	693	40.3 (12.7)
12 months
Socially connected	955	84.9
Poor physical or mental health*	866	−0.01 (0.99)
Reversed EQ5D overall health score	935	35.4 (21.1)
Colorado Symptom Index total score	892	32.6 (11.4)
Substance use problems in the past month (GAIN SS) total score	938	1.5 (1.8)
24 Month Outcomes
Quality of life total score	924	88.8 (22.6)
Crisis related events/service utilization	993
0	507	51.1
1	269	27.1
2	163	16.4
3	47	4.7
4	7	0.7
Recovery assessment scale total score	808	85.2 (12.0)

Path analysis models

Table 3 presents unstandardized beta coefficients, odds ratios and rate ratios with 95% confidence internals from our path analysis models quality of life, recovery and crisis related events or service utilization at 24-months after randomization. Figures 2 through 4 show the parameter estimates on the Theory of Change paths which visually present the direct and indirect pathways for the 24 month follow-up period. We present our 3 models separately below.

Table 3

Unstandardized Parameter Estimates and 95% confidence intervals from Path Analysis Models Testing of the Housing First Theory of Change for Quality of Life, Recovery and Crisis Related Events (N = 1103).

Path Model	Quality of Life	Recovery	Crisis Related Events or Service Utilization
0–6 months
Working Alliance
Public Benefits	0.41 (−0.71,1.52)	0.45 (−0.52,1.43)	0.41 (−0.576,1.38)
Hope for Change	0.32 (0.08,0.57)	NA	0.32 (0.08,0.57)
Constant	51.47 (44.56,58.37)	59.41 (56.77,62.05)	51.47 (44.87,58.06)
Health or Social Service Consultation^
Working Alliance	1.00 (0.99,1.01)	1.00 (0.99,1.01)	1 0.00(0.99,1.01)
Hope for Change	1.00 (0.97,1.03)	NA	1 0.00(0.97,1.03)
Constant	2.18 (1.01,4.75)	2.16 (1.32,3.52)	2.18 (1.00,4.76)
Subjective Well-Being
Public Benefits	0.70 (0.04,1.36)	0.81 (0.16,1.45)	0.70 (0.06,1.33)
Hope for Change	0.81 (0.65,0.98)	NA	0.81 (0.65,0.98)
Health or Social Service Consultation	−2.98 (−4.90,−1.05)	−3.00 (−5.09,−0.91)	−2.98 (−4.94,−1.01)
Constant	20.41 (15.95,24.86)	40.35 (38.18,42.52)	20.41 (15.94,24.88)
Participation in Meaningful  Activities^
Working Alliance	1.01 (1.00,1.02)	1.01 (1.00,1.02)	1.01 (1.00,1.02)
Hope for Change	1.01 (0.98,1.05)	NA	1.01 (0.98,1.05)
Health or Social Service Consultation	1.50 (1.04,2.16)	1.50 (1.04,2.16)	1.50 (1.04,2.16)
Constant	1.01 (0.41,2.47)	1.41 (0.76,2.64)	1.01 (0.41,2.48)
12 Months
Poor physical or mental health
Working Alliance	0.0020 (−0.002,0.0060)	0.0020 (−0.002,0.0060)	0.002 (0.002,0.006)
Subjective Well-Being	−0.036 (−0.042,−0.031)	−0.036 (−0.042,−0.031)	−0.0036 (−0.042,−0.031)
Constant	1.32 (1.06,1.62)	1.32 (1.02,1.62)	1.32 (1.02,1.61)
Socially Connected^
Health or Social Service Consultation	1.78 (1.21,2.61)	1.78 (1.21,2.61)	1.78 (1.21,2.61)
Constant	3.73 (2.78,5.02)	3.73 (2.78,5.02)	3.73 (2.78,5.02)
24 months
	Quality of Life	Recovery	Crisis Related Events+
Subjective Well-Being	0.22 (0.07,0.38)	0.10 (0.02,0.18)	NA
Poor physical or mental health	−7.49 (−9.31,−5.66)	−3.38 (−4.31,−2.45)	1.24 (1.15,1.34)
Participation in Meaningful Activities	1.05 (−2.46,4.56)	1.81 (−0.33,3.96)
NA
Socially Connected	8.73 (4.58,12.89)	4.88 (2.53,7.23)	1.15 (0.92,1.43)
Working Alliance	NA	NA	1.0 (0.99, 1.00)
Constant	71.62 (64.10,79.13)	75.34 (71.59,79.08)	0.73 (0.53,0.98)

results are Odds Ratios from logistic regression.

results are Incidence Rate Ratios from negative binomial regression.

Figure 4

Unstandardized beta coefficients, odds ratios and rate ratios from the path analysis model* of theory of change for Crisis Related Events at 24 months follow-up for the At Home/Chez Soi study. Dashed lines represent a pathway to a binary outcome and solid lines represent paths to continuous outcomes.

Quality of life (QOL) at 24 months

We present results moving from early program experiences (0–6 months) toward the 24-month outcome.  The baseline variable health or social service consultation launches three important indirect pathways to the outcome QOL at 24-months.  First, health or social service consultation is associated with 6 month subjective well-being (b=−2.98, 95% CI −4.90,−1.05) which in turn directly improves QOL (b = 0.2, 95% CI 0.07, 0.38); subjective well-being at 6 months also improves poor physical and mental health at 12 months (b=−0.04 (−0.04,−0.03) which has a direct path to QOL at 24 months (b = 7.49, 95% CI −9.31,−5.66) (Figure 2 and Table 3).  Health and social service consultations at baseline is significantly associated with improvements in 12 month social connectedness (OR 1.78, 95% CI 2.32,3.61) which in turn has a direct path to QOL (b = 8.73, 95% CI 4.58, 12.88).  These are three main pathways from baseline variables to QOL (Figure 2 and Table 3) with those going through social connectedness and poor physical and mental health at 12 months having the largest effects on improvements to QOL at 24-months.  Hope for change (b = 0.81, 95% CI 0.65, 0.98) and access to public benefits at 0–6 months (b = 0.70, 95% CI 0.04,1.36) were also found to improve subjective well-being, a key mediator toward QOL at 24 months as already noted.  Working alliance at baseline was hypothesized to have a larger role but many of those associations were not confirmed, specifically those with health and social service consultation and poor physical or mental health at 12 months. Only one leg of the pathway from Working Alliance to QOL mediated by participation in meaningful activities was confirmed (Figure 2 and Table 3).  Of the four direct pathways to QOL at 24 months—subjective well-being at 0–6 months, participation in meaningful activities at 6 months, social connectedness at 12 months, poor physical and mental health at 12 months—three were confirmed (Figure 2).

Recovery at 24 months

Starting from the period immediately after enrolment (0–6 months), health and social service consultation launches two significant pathways toward recovery at 24-months.  One of these pathways from health and social service consultation improves 12-month social connections (OR=1.78, 95% CI 1.21,2.61) and then it, in turn, has a direct pathway to recovery at 24- months (b = 4.88 95% CI 2.53,7.23).  A second mediated pathway from health and social connections at enrolment that passes through subjective well-being at 6 months and includes improvements in poor physical and mental health at 12 months (b=−0.04, 95% CI −0.04,−0.03) and then, in turn, directly improves recovery at 24 months (b=−3.38, 95% CI −4.31,−2.45).  The largest impacts on recovery from 12 month variables are from having high levels of social connectedness and low levels of poor physical and mental health (Figure 3 and Table 3).

For the model of recovery, it had been hypothesized that working alliance would have indirect effects on recovery but this was not confirmed with our path analysis model.  Working alliance was associated with improved participation in meaningful activities at 6 months (OR=1.01, 95% CI 1.00,1.02) but the next leg of the path to recovery was not confirmed (Figure 3, Table 3).

Crisis-related events or service utilization

The two key variables at baseline involved in significant pathways to the 24 month outcome crisis related events and service utilization are hope for change and health or social service consultation because of their significant association with subjective well-being at 6 months, respectively (hope for change b = 0.81, 95% CI0.65,0.98; health or service consultation b = 0.70, 95%CI (0.06,1.33) (Table 3, Figure 4). Subjective well-being at 6 months, as for the previous two 24-month outcomes discussed, has a mediating role in the path to crisis related events in its association with poor physical and mental health (b=−0.04, 95% CI −0.04,−0.03), and the direct path from 12-month poor physical and mental health to 24-month crisis related events (RR=1.24, 95% CI 1.15, 1.34). Social connectedness was not a confirmed pathway for this outcome as it was for the previous two.  The direct path from working alliance to crisis related events was also not confirmed in our analyses (Figure 4).

The funder, the Mental Health Commission of Canada, had no role in the design of the study nor the collection, analysis and interpretation of the data.  The MHCC had no role in writing this manuscript nor in the decision to submit this paper for publication.

Discussion

Our study contributes to the growing literature on empirical tests of program theories of change and uses a unique longitudinal quantitative data set on a large sample to examine pathways of change for diverse outcomes. Using path analysis, and data from the large At Home/Chez Soi pragmatic field trial, we were able to empirically confirm hypothesized direct and indirect relationships in the theory of change for Housing First.,, All of the variables examined in the model demonstrated indirect or direct relationships for two of the 24-month outcomes examined: quality of life (QOL) and recovery.  We were less successful in confirming hypothesized direct pathways for 24-month crisis related events and service utilization.

The findings from this set of analyses are not an outcome evaluation of the At Home/Chez Soi Housing First program, and consequently they complement, rather than build on, the number of studies examining the outcomes in this RCT.,, In particular, our findings contribute to the growing research on the mechanisms by which Housing First programs generate non-housing related outcomes. Several studies have examined the fidelity to the key program components of Housing First, illustrating that high levels of fidelity drive better outcomes. As fidelity measures do not identify the mechanisms or pathways by which program activities lead to specific outcomes, our findings offer complementary new insights on the longitudinal paths leading to distinct non-housing outcomes and they highlight the need to understand the contributing pathways for myriad long-term outcomes.  Thus, in addition to the Housing First theory of change, and the evidence from fidelity studies that adherence to the key components of the program designed to promote positive outcomes,, our findings here provide an additional piece of the puzzle to build a causal argument about how and why Housing First programs improve non-housing outcomes.

Over the first year of the program, health and social service consultation upon enrolment and health status at one year were central to significant paths for our three outcomes.  For all three of our models, health and social service consultation at enrolment was confirmed to be a significant part of multiple indirect pathways to the 24-month outcomes.  Counterintuitively, health and social service consultation was inversely associated with subjective well-being. One partial explanation is that not all health and social service needs were resolved via service engagement and could lead to diminishing psychological well-being. Physical and mental health status at 12 months was directly associated with all three of our outcomes and the magnitude of its impact on the outcomes was sizable. A one unit improvement in physical and mental health at 12 months yielded a 7-point change in QOL.  Two-year evaluation data for all five At Home/Chez Soi sites demonstrate a 16-point change in QOL from baseline to 24 months. Thus, seven points is almost half the size of the observed change in QOL over 2 years. Subjective well-being at six months was directly associated with both quality of life and recovery at 24 months although the magnitude of this direct contribution was relatively small.

Recovery approaches in mental health programs such as those used in Housing First seek to connect clients to meaningful daily routines around school and greater engagement with family and community. Being socially connected at 12 months was another important direct predictor of quality of life and recovery at 24 months confirmed by our path models. This finding builds on a rich evidence base about the buffering role of social connectedness among those with mental illness. A one-unit change in social connectedness at 12 months in our study, yielding an 8 point improvement in QOL, is equivalent to half the average change seen in QOL from baseline to 24 months as noted earlier.

The theoretical model for the third outcome – crisis-related events or service utilization–had the fewest hypothesized relationships confirmed. While it had been hypothesized that being socially connected or having a strong working alliance would act as buffers for crisis related events, these direct relationships were not confirmed in our models. Prior evaluations of interventions seeking to reduce emergency room visits or hospitalizations among homeless individuals failed to find evidence that continuity of care or linkage to community based services is effective in reducing crisis-oriented service use such as emergency room visits. The Housing First theory of change for crisis related events should be modified for future analyses. It is also possible that we should have separated criminal justice events from health services for crises instead of combining them as we had.  The program activities and pathways supporting and affecting health crises might differ from those involved in vulnerabilities and behaviours associated with criminal and justice processes.  Housing First evaluations report improvements in emergency room visits and hospitalization at 24 months while this was not observed in a systematic review of criminal justice outcomes like arrests. Housing First programs do not necessarily provide resources and supports for addressing criminogenic risks that would reduce crime.

Some other hypothesized pathways in the theory of change were also not confirmed.  While working alliance was hypothesized to have a key early role in several paths, many of those associations were not confirmed.  Our measure of working alliance was for the start of the program whereas it may have been important to track working alliance for later stages of the program as well.  We also only included the clients’ perspectives and did not include the workers’ ratings of working alliance. A more comprehensive assessment may have yielded different findings about working alliance.  We also hypothesized that hope for change early on would play a larger role in the recovery process, as it has been identified a key component in the path towards recovery. However, like working alliance, some but not all hope for change pathways were confirmed with our statistical models.

There are limitations to our analyses that should be considered. We relied on statistical significance to confirm direct and indirect associations in our models and had not, a priori, identified a magnitude of effect needed to confirm our hypotheses.  Immediate access to housing, which is a key treatment component for Housing First, was not an outcome for our path models and has already been examined in a prior meta-modeling study of how Housing First participants maintain housing tenure. Since all participants in the treatment groups secured housing there was very little variability in that variable. Our lack of data on actual process measures and participant engagement with program components over time may have left a gap in our ability to gain a full understanding of these pathways. The evaluation data were kept separate from program participation data and therefore process data were not available to the research team in the detail needed to include in this analysis. The pathways for the three 24-month outcomes examined in this analysis might differ substantially from those of other outcomes, such as employment, criminal justice involvement or housing stability, thus, the generalizability of these models for other outcomes might be limited.  Future analyses will have to examine the pathways for those outcomes. We did not include variables that are typically included in regression models for program evaluation such as site or gender or age.  Addition of these variable might change the parameter estimates we obtained. These variables were not part of our original theory of change models as the program was intended to meet all client's needs equally and fidelity to program key ingredients was high at all sites. Future research might develop models that include considerations of the variables. Our analyses were limited to the treatment group. Many of these same pathways might be applicable for the treatment-as-usual group of the At Home/Chez Soi program as the evaluation data demonstrated access to local services and similar improvements for both groups for several outcomes, and this too could be the subject of future research.

Our findings have implications for implementation of future Housing First programs and theory of change research. While securing and maintaining housing is an appropriate primary focus of these programs, enhanced efforts to engage clients during the first year of the program in internal or external initiatives to improve mental and physical health, including substance use issues would increase program impact for quality of life, recovery and crisis related events.  Enhancing clients’ connections to personal social support systems can also improve longer term quality of life and recovery outcomes. One outstanding question for future research related to these particular pathways is why working alliance improves participation in meaningful activities but not health status at one-year post-enrolment. Four of the five sites of At Home/Chez Soi were large cities and this may limit generalizability to smaller cities or more remote areas where configuration of services may differ given geographic considerations or where fewer clients are being served.  The benefits of services being more client-focused including clients having high levels of trust in program providers might be a focus of future research. Since Housing First is a client-centered program, thus, future examinations of program change theories should include and prioritize clients’ perspectives to add to our understanding of how, why and for whom programs work as this analysis represents the point of view of program planners.

Contributors

PO, ST, VS, JD, and RN conceptualized the paper.  OD and PO conducted the literature review for the Introduction.  ST, VS, and PO created the theoretical path models to be tested.   PO, RN, and JL designed, oversaw and implemented the analyses and had access to the underlying data.  All authors–PO, ST, VS, JD, RN, JL, NA– interpreted the findings, had access to the data and were involved in the decision to submit the paper for publication.  PO wrote the original draft of the paper and all authors reviewed and revised subsequent drafts.

Data sharing statement

The AH/CS survey dataset cannot be made publicly available due to the sensitive nature of the data and agreements and procedures governing the use of the dataset that were established by the study sponsor, the Mental Health Commission of Canada. However, anonymized participant data from the AH/CS study can be made available to investigators who complete the following steps: (1) present a study proposal that has received approval from an independent research committee or research ethics board; (2) provide a data request for review by the AH/CS data access committee; (3) following approval of the request, execute a data-sharing agreement between the investigators and the AH/CS data custodians. Study proposals and data access requests should be sent to Evie Gogosis (Evie.Gogosis@unityhealth.to), research manager for the Toronto site of the AH/CS study, and to Dr. Patricia O'Campo (patricia.ocampo@unityhealth.to), co-principal investigator of the Toronto site of the AH/CS study.

Funding

This research was funded by Mental Health Commission of Canada.

Declaration of interests

The organization where PO, VS are employed at the time of data collection received funds from the Mental Health Commission of Canada to conduct the research.  NA was employed by the research project during the time of data collection.  All other authors have no interests or conflicts to declare.