Recruitment and retention of homeless individuals with mental illness in a housing first intervention study.

BACKGROUND: Homeless individuals with mental illness are challenging to recruit and retain in longitudinal research studies. The present study uses information from the Vancouver site of a Canadian multi-city longitudinal randomized controlled trial on housing first interventions for homeless individuals. We were able to recruit 500 participants and retain large number of homeless individuals with mental illness; 92% of the participants completed the 6-month follow up interview, 84% the 24-month follow up, while 80% completed all follow-up visits of the study.
PURPOSE: In this article, we describe the strategies and practices that we considered as critical for successful recruitment and retention or participants in the study.
METHODS: We discuss issues pertaining to research staff hiring and training, involvement of peers, relationship building with research participants, and the use of technology and social media, and managing challenging situations in the context of recruitment and retention of marginalized individuals.
CONCLUSIONS: Recruitment and retention of homeless participant with mental illness in longitudinal studies is feasible. It requires flexible, unconventional and culturally competent strategies. Longitudinal research projects with vulnerable and hidden populations may benefit from extensive outreach work and collaborative approaches that are based on attitudes of mutual respect, contextual knowledge and trust.

Introduction

Homelessness has become one of the most pressing and complex social, health and political issues in many countries. Studies have consistently demonstrated particularly high prevalence rates of substance use and mental disorders among homeless populations in many Western countries including Canada [1], [2], the US [3], [4], [5], Australia [6], and Europe [7], [8], [9].

Research on how to address this important issue has increased over the past decades but is still fraught with a number of challenges. For example, recruitment can be difficult because potential participants may be withdrawn, socially alienated, distrustful of research projects, or absorbed with the demands of their current challenges. Researchers in this field need to develop effective outreach strategies in order to obtain adequately sized and representative samples [10]. Another issue common to longitudinal studies is participant attrition. Attrition can introduce bias if participants lost to follow-up differ from those retained in the study, particularly if the characteristics distinguishing completers versus non-completers are associated with the outcome measures [11], [12]. Factors such as residential instability, involvement in the justice system, cognitive impairment, and high rates of substance use makes retention of homeless individuals in longitudinal studies particularly challenging.

In recent years, researchers have identified a number of effective strategies to recruit and retain marginalized and hard-to-reach populations in longitudinal studies [13], [14], [15], [16]. Homeless individuals with mental illnesses overlap with several other at-risk groups, e.g., populations with high rates of injection drug use, HIV, and members of ethnic minority groups (e.g., [17], [18]). In 1996, Hough and colleagues [10] summarized successful strategies for the recruitment and retention of homeless mentally ill participants. However, the use of recently developed technologies such as e-mail, text messaging, and social media has received little attention.

The current paper, which was written by both researchers and field interviewers, describes the strategies and practices that we considered as critical for successfully recruiting and retaining homeless participants with mental illness for the Vancouver site of the At Home/Chez Soi Study, a Canadian housing intervention study. Furthermore, we will discuss a number of barriers and challenges faced in recruiting and retaining such participants. The paper may offer direction, recommendations, and inspiration to other researchers who are planning to conduct longitudinal studies with this population.

Summary of the At Home/Chez Soi study methodology

The At Home/Chez Soi Study is a randomized-controlled trial (RCT) examining the effects of different housing interventions for homeless adults with mental illness in five Canadian cities. Study design and sample size were determined by the At Home/Chez Soi National Research Team, which monitored activities at the different study sites. Details related to the RCT protocol and study procedures are described in detail elsewhere for both the multi-centre elements [19] and the Vancouver design [20].

Eligibility criteria in Vancouver included legal adult status (19 years and older), current mental disorder and being either absolutely homeless or precariously housed. Current mental disorder was determined using the Mini-International Neuropsychiatric Interview (MINI) [21] for the assessment of the following diagnoses based on criteria of the Diagnostic and Statistical Manual of Mental Disorders: major depressive episode, manic or hypomanic episode, post-traumatic stress disorder, mood disorder with psychotic features, and psychotic disorder. Where possible, mental disorder status was corroborated by physician diagnosis.

“Absolute homelessness” was defined as living on the streets or in an emergency shelter for at least the past seven nights with little likelihood of obtaining secure accommodation in the upcoming month. “Precariously housed” was defined as living in a rooming house, hotel, or other transitional housing; in addition, these individuals must have experienced at least two episodes of absolute homelessness or a single episode lasting at least four weeks during the past year.

In Vancouver, participants were recruited through referral from over 40 agencies available to homeless adults; the majority was recruited from shelters, drop-in centers, outreach teams, hospitals, community mental health teams, and criminal justice programs. We specifically targeted organizations that serve women, youth, aboriginal peoples, and gay/lesbian/transgender individuals in order to obtain a sample as diverse and representative as possible. All participants met face-to-face with a trained interviewer who confirmed study eligibility, explained procedures, and obtained informed consent. An honorarium of $5 was provided for the screening process. Institutional ethics board approval was obtained through Simon Fraser University and the University of British Columbia.

Eligible individuals completed a comprehensive baseline assessment including a variety of psychiatric, psychosocial, and health related measures and an overview of all measures used is published in detail elsewhere (see 19 and 20). Based on the results of the baseline assessment, participants' level of need was categorized as “high” or “moderate” by the following computer algorithm: [participants'] needs were categorized as “high” if they 1) met criteria for a current Psychotic Disorder or Manic/Hypomanic Episode on the MINI, and 2) received a total score of 62 or lower on the Multnomah Community Ability Scale (MCAS), a scale designed to measure daily functioning of individuals with severe mental illness [22], [23], and 3) reported at least one of the following: (a) two or more hospitalizations for mental illness during the last five years, (b) met criteria for current substance abuse or dependence on the MINI, or (c) reported arrest or incarceration in the past six months. All other study participants were categorized as having “moderate” needs. This categorization was grounded on clinical and functional considerations regarding individuals living with psychiatric illness: The presence or psychotic disorders and/or manic/hypomanic episodes, the presence of a concurrent substance use disorder, and repeated hospitalization due to mental illness were considered as indicators for high severity and chronicity of the participants' mental condition. Furthermore, a low score on the MCAS, the Multnomah Community Ability Scale, which measures daily functioning in the community context, as well as recent incarceration were considered as indicators of impaired or low psychosocial functioning.

It was assumed that a high severity and chronicity of the current mental illness and low psychosocial functioning would require (‘need’) more intense and/or more frequent psychosocial support, services and care.

Following the level of need categorization, participants were randomly assigned to receive either a Housing First intervention (i.e., an accommodation plus psychosocial support) or to usual care (i.e., participants received no active intervention through the study but were permitted to access any available form of care and housing by themselves). Participant flow through eligibility screening, need level assessment, and allocation to study arm is visualized in Fig. 1.

Fig. 1

Participant flow through eligibility screening need level assessment, and allocation to study arm.

All participants were followed for 24 months through four comprehensive and four brief follow-up interviews conducted at three-month intervals. Recruitment and randomization were conducted between October 2009 and June 2011. Because the final follow up assessment is still in progress, we used only data until the 21-month follow up assessment for the current study. The last 24-month follow up interview was conducted on March 25, 2013.

Recruitment and retention statistics

Of all five participating sites, Vancouver was a successful study site with timely recruitment (of N = 500 participants) and high attrition [27].

Table 1 describes the retention rates for each follow up at time point 6, 12, 18, and 24 months, where the full interview batteries were administered. The completion rate for each visit ranged between 83.7% and 92.4%. 79.6% attended all four visits, while only 4.6% of the participants did not attend any follow-up visit. Table 2 compares participants who had no follow up visit with participants who had at least one follow up visit on socio-demographic and clinical variables. Table 3A, Table 3B, Table 3C, Table 3D are showing comparisons of socio-demographic characteristics by 6, 12, 18, and 24-month follow up completion status respectively.

Table 1

Follow-up completion rate for study participants over 24 months by need level.

	High Need (n = 297)N (%)	Moderate Need (n = 200)N (%)	Total (n = 497)N (%)
Follow up visit completed
6 months	281 (94.6)	178 (89.0)	459 (92.4)
12 months	274 (92.3)	177 (88.5)	451 (90.7)
18 months	257 (86.5)	173 (86.5)	430 (86.5)
24 months	250 (84.2)	166 (83.0)	416 (83.7)
Number of completed FU visits
All four visits	240 (80.7)	155 (77.5)	395 (79.6)
Only three visits	18 (6.1)	15 (7.5)	33 (6.6)
Only two visits	19 (6.4)	12 (6.0)	31 (6.2)
Only one visit	10 (3.4)	5 (2.5)	15 (3.0)
No follow up visit	10 (3.4)	13 (6.5)	23 (4.6)
Reason for no (0) follow up visit
Death	5 (1.7)	4 (2.0)	9 (1.8)a
Withdrew	0 (0.0)	2 (1.0)	2 (0.4)
No contact	5 (1.7)	7 (3.5)	12 (2.4)

All 9 participants died within 7 months of enrolment. In addition, 20 participants died during the follow up period, but completed at least one follow up visit before death.

Table 2

Comparisons of Socio-demographic and mental disorder related characteristics by follow up completion status.

Variable	High Need (n = 297, 10 vs. 287)			Moderate Need (n = 200, 13 vs. 187)
	No FU visitn (%)/mean (sd)	At least one FU visitn (%)/mean (sd)	P valuea	No FU visit n (%)/mean (sd)	At least one FU visitn (%)/mean (sd)	P value
Sociodemographics
Age (years): mean (sd)	35.4 (14.8)	39.8 (11.0)	0.224	46.8 (9.5)	42.3 (10.5)	0.141
Male sex	9 (90)	209 (74)	0.462	10 (77)	131 (70)	0.760
Ethnicity
Aboriginals	2 (20)	42 (15)	0.817	3 (23)	30 (16)	0.682
Caucasian	6 (60)	164 (57)		6 (46)	104 (56)
Mixed/Other	2 (20)	81 (28)		4 (31)	53 (23)
Incomplete high school	7 (66)	172 (61)	0.745	5 (38)	96 (51)	0.405
Single/Never married	8 (89)	206 (72)	0.453	7 (54)	122 (65)	0.550
Have children (under18)	0 (0)	69 (25)	0.120	5 (42)	48 (26)	0.310
Currently employed	0 (0)	10 (3)	1.000	1 (8)	7 (4)	0.279
Homelessness
Absolutely homeless	10 (100)	222 (77)	0.125	8 (62)	148 (79)	0.165
Lifetime duration of homelessness (months)
mean (sd)	62.2 (60.1)	62.0 (67.3)	0.993	35.4 (68.6)	61.5 (69.1)	0.273
DSM-IV diagnosis
Major Depressive Episode	3 (30)	92 (32)	1.000	6 (46)	98 (52)	0.777
Manic or Hypomanic Episode	2 (20)	66 (23)	1.000	1 (8)	28 (15)	0.697
Post-Traumatic Stress Disorder	1 (10)	62 (22)	0.694	4 (31)	62 (33)	1.000
Panic Disorder	2 (20)	57 (20)	1.000	1 (8)	44 (23)	0.305
Psychotic Disorder	8 (80)	203 (71)	0.729	5 (38)	47 (25)	0.329
Alcohol dependence	2 (20)	70 (24)	1.000	3 (23)	46 (25)	1.000
Substance dependence	4 (40)	179 (62)	0.191	6 (46)	99 (53)	0.776
Suicidality (moderate or high)	2 (20)	91 (32)	0.730	3 (23)	72 (38)	0.378
Two or more mental disorders	5 (50)	143 (50)	1.000	5 (38)	87 (46)	0.775

p values (for both high and moderate need comparisons) were based on independent sample t-test with equal or unequal variance (for continuous variables) and Pearson chi-square test or Fisher's exact test (for categorical variables) as appropriate.

Table 3A

Comparisons of Socio-demographic characteristics by 6-month follow up completion status.

		High need (n = 16 vs. n = 281)			Moderate need (n = 22 vs. n = 178)
		6-month visit-non (%)/mean (sd)	6-month visit-yesn (%)/mean (sd)	P valuea	6-month visit-non (%)/mean (sd)	6-month visit-yesn (%)/mean (sd)	P value
Age (years)	mean (sd)	36.4 (13.5)	39.9 (11.0)	0.233	46.8 (9.0)	42.1 (10.6)	0.044
Gender	Female	1 (6)	75 (27)	0.079	4 (18)	54 (31)	0.230
	Male	15 (94)	203 (73)		18 (82)	123 (69)
Ethnicity	Aboriginals	4 (25)	40 (14)	0.507	4 (18)	29 (16)	0.862
	Caucasian	8 (50)	162 (58)		11 (50)	99 (56)
	Mixed/Other	4 (25)	79 (28)		7 (32)	50 (28)
Education	High School or Higher	4 (25)	111 (40)	0.234	12 (55)	87 (49)	0.616
	Less than high school	12 (75)	167 (60)		10 (45)	91 (51)
Marital status	Single (never married)	13 (87)	201 (72)	0.369	14 (64)	115 (65)	0.928
	Other	2 (13)	77 (28)		8 (36)	63 (35)
Have children (under18)	No	15 (100)	204 (75)	0.026	15 (71)	130 (73)	0.843
	Yes	0 (0)	69 (25)		6 (29)	47 (27)
Employment	Unemployed/other	16 (100)	269 (96)	1.000	21 (95)	171 (96)	1.000
	Employed	0 (0)	10 (4)		1 (5)	7 (4)
Housing status	Absolutely homeless	15 (94)	217 (77)	0.209	16 (73)	140 (79)	0.586
	Precariously housed	1 (6)	64 (23)		6 (27)	38 (21)
Lifetime homelessness (months))	mean (sd)	76.9 (100.5)	61.1 (64.7)	0.360	45.9 (66.1)	58.9 (76.0)	0.400

Bold values signify p < 0.05.

p values (for both high and moderate need comparisons) were based on independent sample t-test with equal or unequal variance (for continuous variables) and Pearson chi-square test or Fisher's exact test (for categorical variables) as appropriate.

Table 3B

Comparisons of Socio-demographic characteristics by 12-month follow up completion status.

		High need (n = 23 vs. n = 274)			Moderate need (n = 23 vs. n = 177)
		12-month visit-non (%)/mean (sd)	12-month visit-yesn (%)/mean (sd)	P valuea	12-month visit-non (%)/mean (sd)	12-month visit-yesn (%)/mean (sd)	P value
Age (years)	mean (sd)	38.3 (13.8)	39.8 (10.9)	0.614	43.5 (10.8)	42.5 (10.5)	0.649
Gender	Female	3 (13)	73 (27)	0.144	6 (26)	52 (30)	0.731
	Male	20 (87)	198 (73)		17 (74)	124 (70)
Ethnicity	Aboriginals	2 (9)	42 (15)	0.649	5 (22)	28 (16)	0.753
	Caucasian	13 (57)	157 (57)		12 (52)	98 (55)
	Mixed/Other	8 (35)	75 (27)		6 (26)	51 (29)
Education	High School or Higher	9 (39)	106 (39)	1.000	13 (57)	86 (49)	0.474
	Less than high school	14 (61)	165 (61)		10 (43)	91 (51)
Marital status	Single (never married)	19 (86)	195 (72)	0.143	13 (57)	116 (66)	0.395
	Other	3 (14)	76 (28)		10 (43)	61 (34)
Have children (under18)	No	21 (95)	198 (74)	0.026	14 (64)	131 (74)	0.281
	Yes	1 (5)	68 (26)		8 (36)	45 (26)
Employment	Unemployed/other	23 (100)	262 (96)	1.000	22 (96)	170 (96)	1.000
	Employed	0 (0)	10 (4)		1 (4)	7 (4)
Housing status	Absolutely homeless	19 (83)	213 (78)	0.587	16 (70)	140 (79)	0.299
	Precariously housed	4 (17)	61 (22)		7 (30)	37 (21)
Lifetime homelessness (months)	mean (sd)	75.9 (90.7)	60.8 (64.6)	0.442	38.5 (57.5)	59.9 (76.7)	0.198

Bold value signifies p < 0.05.

p values (for both high and moderate need comparisons) were based on independent sample t-test with equal or unequal variance (for continuous variables) and Pearson chi-square test or Fisher's exact test (for categorical variables) as appropriate.

Table 3C

Comparisons of Socio-demographic characteristics by 18-month follow up completion status.

		High need (n = 40 vs. n = 257)			Moderate need (n = 27 vs. n = 173)
		18-month visit-non (%)/mean (sd)	18-month visit-yesn (%)/mean (sd)	P valuea	18-month visit-non (%)/mean (sd)	18-month visit-yesn (%)/mean (sd)	P value
Age (years)	mean (sd)	40.9 (14.6)	39.5 (10.6)	0.557	42.2 (11.8)	42.6 (10.3)	0.851
Gender	Female	6 (15)	70 (28)	0.092	5 (19)	53 (31)	0.191
	Male	34 (85)	184 (72)		22 (81)	119 (69)
Ethnicity	Aboriginals	6 (15)	38 (15)	0.946	4 (15)	29 (17)	1.000
	Caucasian	22 (55)	148 (58)		15 (56)	95 (55)
	Mixed/Other	12 (30)	71 (28)		8 (30)	49 (28)
Education	High School or Higher	16 (40)	99 (39)	0.902	17 (63)	82 (47)	0.132
	Less than high school	24 (60)	155 (61)		10 (37)	91 (53)
Marital status	Single (never married)	28 (72)	186 (73)	0.851	17 (63)	112 (65)	0.858
	Other	11 (28)	68 (27)		10 (37)	61 (35)
Have children (under18)	No	32 (86)	187 (75)	0.111	19 (73)	126 (73)	0.985
	Yes	5 (14)	64 (25)		7 (27)	46 (27)
Employment	Unemployed/other	39 (98)	246 (96)	1.000	26 (96)	166 (96)	1.000
	Employed	1 (3)	9 (4)		1 (4)	7 (4)
Housing status	Absolutely homeless	33 (83)	199 (77)	0.471	17 (63)	139 (80)	0.043
	Precariously housed	7 (18)	58 (23)		10 (37)	34 (20)
Lifetime homelessness (months)	mean (sd)	60.0 (78.1)	62.3 (65.2)	0.845	59.4 (141.4)	57.1 (58.9)	0.884

Bold value signifies p < 0.05.

p values (for both high and moderate need comparisons) were based on independent sample t-test with equal or unequal variance (for continuous variables) and Pearson chi-square test or Fisher's exact test (for categorical variables) as appropriate.

Table 3D

Comparisons of Socio-demographic characteristics by 24-month follow up completion status.

		High need (n = 47 vs. n = 250)			Moderate need (n = 34 vs. n = 166)
		24-month visit-non (%)/mean (sd)	24-month visit-yesn (%)/mean (sd)	P valuea	24-month visit-non (%)/mean (sd)	24-month visit-yesn (%)/mean (sd)	P value
Age (years)	Mean (sd)	39.7 (14.5)	39.7 (10.5)	0.967	41.6 (12.2)	42.8 (10.2)	0.560
Gender	Female	10 (21)	66 (27)	0.435	7 (21)	51 (31)	0.228
	Male	37 (79)	181 (73)		27 (79)	114 (69)
Ethnicity	Aboriginals	9 (19)	35 (14)	0.869	5 (15)	28 (17)	0.850
	Caucasian	25 (53)	145 (58)		18 (53)	92 (55)
	Mixed/Other	13 (28)	70 (28)		11 (32)	46 (28)
Education	High School or Higher	22 (47)	93 (38)	0.238	19 (56)	80 (48)	0.414
	Less than high school	25 (53)	154 (62)		15 (44)	86 (52)
Marital status	Single (never married)	32 (70)	182 (74)	0.563	22 (65)	107 (64)	0.978
	Other	14 (30)	65 (26)		12 (35)	59 (36)
Have children (under18)	No	36 (82)	183 (75)	0.329	23 (70)	122 (74)	0.615
	Yes	8 (18)	61 (25)		10 (30)	43 (26)
Employment	Unemployed/other	46 (98)	239 (96)	1.00	32 (94)	160 (96)	0.626
	Employed	1 (2)	9 (4)		2 (6)	6 (4)
Housing status	Absolutely homeless	37 (79)	195 (78)	0.912	23 (68)	133 (80)	0.110
	Precariously housed	10 (21)	55 (22)		11 (32)	33 (20)
Lifetime homelessness (months)	Mean (sd)	46.2 (41.9)	65.0 (70.4)	0.016	61.3 (127.1)	56.6 (59.5)	0.740

Bold value signifies p < 0.05.

p values (for both high and moderate need comparisons) were based on independent sample t-test with equal or unequal variance (for continuous variables) and Pearson chi-square test or Fisher's exact test (for categorical variables) as appropriate.

Student's t-test and Fishers exact test were conducted to compare continuous and categorical variables between groups respectively. We found no significant differences with respect to socio-demographic and clinical variables, except housing status (absolutely homelessness versus precariously housed) in the 18 months follow up interview and lifetime duration of homelessness in the 24 months follow up. The following sections highlight the specific strategies our team has used to complete timely recruitment and maintain high retention rates.

Description of recruitment and retention strategies

Selection of and work with the field interviewers

We hired interviewers who had extensive experience working with homeless individuals and community-based organizations, and who possessed strong interpersonal skills. Given that many baseline and follow-up interviews were conducted in the community, we hired staff who were self-motivated and felt comfortable working alone and in an unstructured and often chaotic work environment. Interviewers were required to have some existing knowledge of basic research concepts including confidentiality, bias, and standardized administration of research measures. We attempted to hire individuals, who were familiar with the culture, social justice and critical thinking to articulate the needs and experiences of the participants' population. For our research work we specifically hired field interviewers who were familiar with Vancouver's homeless population, knowledgeable regarding existing services for homeless individuals, and comfortable walking through poor neighborhoods and interacting with their residents. Importantly, staff were required to have the ability to assess and manage various issues such as agitation and aggression, psychotic symptoms, intoxication, suicidal ideation, and self-harm. We hired people who had calm yet assertive manners, and were non-judgmental and supportive towards even the most marginalized individuals.

Furthermore, many homeless people have experiences such as childhood abuse or prostitution [24], [25] that are difficult, painful, or shameful to recall, and we invested effort in finding interviewers who were able to address such topics with respect and sensitivity. Retrospectively turnover rate amongst the field interviewers was low (three research assistant was hired after the training as mentioned above; one research assistant was hired to replace a research assistant who left the team to move to another city) which was facilitated by giving staff freedom, being flexible with part-time and full-time needs and providing an end-of-project bonus.

Peer interviewers

One of the objectives of the At Home Study was to engage peers who have experiences living with mental illness and/or homelessness as members of various teams within the project, including both service provision and research. Two peers were recruited and hired as field research interviewers, with the intention that they would administer research interviews, but more importantly to emotionally support the participants during the interview process. Although there was genuine commitment to include peer interviewers in the field research process, this remained challenging throughout the project. Time-constraints and lack of resources were the biggest obstacles for involving the peer interviewers in all aspects of the daily field research work. Peer interviewers were hired on an hourly contract-basis, which limited their physical presence in the field research office and as a team member. Furthermore, the support and collaboration with peer interviewers was not initially considered in the time allocation and workload of the field team leaders. However, peer interviewers proved to be invaluable in the qualitative interviews: building rapport, creating a safe atmosphere, and validating the participants' emotions that emerged during the interview process.

Staff training

Standardized yet practical administration of measures/questionnaires:

Introduction of the interview and its different components; explaining – in detail yet in plain language – participants' role and rights in this process and how participants can execute these rights (e.g. whom to phone, where to go if they would like to discuss the study, withdraw from the study, or learn more about the study)

Training around potential sources of bias and how to clarify ambiguous responses

How to introduce each questionnaire (as an interview session comprised of several questionnaires that covered a variety of topics, and how to probe if a participant's reply seems vague or unreliable)

How to redirect participants' attention to the interview content when they get distracted and lose focus

How to assess if a participant is not unable to continue with a measure/interview for several reasons (e.g. due to high levels of distress, or unreliable responses and how to pause or end the session)

How to introduce measures that may be emotionally challenging (e.g. around trauma histories, or child welfare system involvement), and/or judgmental (e.g. around substance use, incarceration, psychotic symptoms).

How to debrief with participant who experienced emotional distress as a result of the interview

Extensive role-playing and debriefing

Entering data onto a secure site via laptop computer

Clinical Judgment:

Several instruments, such as the MINI and the MCAS, required some degree of clinical judgment in administration and scoring. Interviewers were trained how to interpret common response styles such as over- and under-reporting, and how to use probes in order to fully assess the presence of a symptom or behaviors. If participants did not verbally acknowledge the presence of symptoms, but behavioral symptoms such as disorganized behavior or psychotic ideation could be observed, the interviewers were instructed to code the item positively.

Role-playing with each other and a professional patient actor, training videotapes, and training manuals were all used to practice the MCAS and MINI application and exercising clinical judgment. We found it specifically important to repeat practice with different role-play partners and watch several training videos available.

Without following a rigid interpreter-reliability process (which was not part of the study protocol) we included extensive group discussion about different ratings amongst interviewer when using rating scales that relied on interviewers' judgment. This enhanced clinical understanding, clinical decision-making and awareness for nuances in selecting a certain rating using a certain measure.

Staff and client safety:

A safety protocol was developed with key steps in the event of aggressive, violent, and suicidal behaviors, severe intoxication, etc.

Furthermore, interviewers were encouraged to set firm boundaries with participants and to break confidentiality if needed (e.g. imminent risk of harm to self or others).

In addition to the training week, staff members also attended a three-day cross-site training event prior to recruitment, which provided a more general overview of the study as well as - initial and ongoing - training on specific measures, interviewing techniques, and the computer system that was used for data entry and storage, scheduling, and randomization.

Finally, the consent form and specific measures that had been adapted for use with homeless populations were pre-tested with a sample of 12 individuals from local shelters and drop-in centers [26]. This allowed interviewers to gain more familiarity and practice with the materials and trial certain approaches how to interact and administer interviews in realist-informed settings.

Three interviewers were hired after the study commenced and, therefore, did not participate in the initial training. New staff were paired with experienced interviewers and observed several interviews before conducting individual modules and then entire interviews with an observer before doing an interview alone. As the study progressed, informal training was conducted in team meetings where pertinent issues such as clinical judgment and challenging participants were addressed. One of two team leaders, both with extensive professional mental health training, was always available by telephone to discuss clinical questions and decision-making rules that arose during an interview. Ongoing training and support via webinar, teleconference and site visits were also provided by the National Cross-Site Study Team.

Recruitment of participants

Establishing a trusting and collaborative working relationship with existing service providers was critical to successful recruitment. Although the study generated considerable media coverage and public attention, we found that many service providers did not have a clear understanding of the study objectives and the target population. These misunderstandings, among others, created some skepticism and reservations toward the project among some service providers. Therefore, members of the Field Research Team and the broader Project Team visited numerous service agencies to communicate the study's objectives, procedures, interventions, inclusion criteria, and to answer questions and respond to feedback.

The majority of participants were recruited via referrals from existing services. One interviewer who had extensive experience working in the community service sector was designated as the primary contact person and was responsible for answering questions from referral agents, providing telephone information and screenings, and scheduling each initial interview. This individual was also the primary contact for participants who dropped into the Field Office throughout the study. The Field Office itself was not located in the inner city but easily accessible for the participants. The importance of continuous relationship building with the community service sector as well as with participants cannot be over-emphasized.

Data collection

The initial screening interview included the collection of specific personal information to facilitate locating the individual in the future. Personal information included nick-names or aliases; services, organizations, and other places frequented on a regular basis (e.g., shops, supervised injection site, street corners, and bottle depots), phone numbers and email addresses, the names and contact information of friends or family, and a brief physical description of the person. Personal information including current address (housing) was updated with each contact. A photograph of the participant was also taken with their consent.

Given the nature of the At Home/Chez Soi project, participants in the intervention arms (High and Moderate Need) were housed through the At Home/Chez Soi project which was based on the ‘Housing First’ approach (in brief: providing fast access to non-conditional housing, not related on certain pre-requisites or behaviors e.g. abstinence from substances), therefore if participants needed to move due to any reason they would be provided with another housing option by the project. Moves and transitions were recorded as part of the project; but will not be discussed in detailed here, as participants' (mostly planned) house moves in the intervention arms did not pose any difficulties to successful follow-up.

For the administration of all measures, including tracking the due dates for follow up interviews and tracking attempts to make contact with participants, we used a database designed to support multiple staff members in conducting follow up. Any written information about participants was entered into the computer system which was password protected. Hand-written notes were shredded or secured in a locked filing cabinet. A log of tracking and contact attempts was created for each participant with date, time and staff initials. Tracking history notes included the contact method used (e.g. phone, letter), date and time of contact attempt, the result of the contact attempt, and suggestions to improve future contact strategies.

Retention of participants

Retention of participants was where the structure of the field research team and the attributes of the individual interviewers became critical to the success of the project. Given that many of the participants were recruited from community based organizations in Vancouver's Downtown Eastside, knowledge of the neighborhood and its multitude of services as well as a level of comfort in working within an often chaotic urban setting was paramount. Several of the interviewers had worked previously in Vancouver's supervised injection site, and knew many of the people in the study and a mutually trusting relationship with site staff. Furthermore, participants often recognized them, and reported feeling comfortable and accepted despite their lifestyle. Thus, we were able to reach even those participants whose movements and level of disconnect from other services would have made them challenging to retain. Often interviewers would attempt to make contact through several avenues to pass messages to difficult to reach participants, and “tracking notes” in the participant's profile page with suggestions to improve future efforts to make contact were continuously updated. Also included in these tracking notes were hints about how to make a connection in difficult cases, and avoid alienating or agitating participants. For example, we would highlight a female participant's request to only be contacted by female interviewers, or another participant's paranoia about being interviewed indoors.

In rare cases, in order to ensure interviewer safety, we would include suggestions to conduct interviews in a public arena or in a location where the participant felt at ease. In making an effort to be flexible, persistent and sensitive to the needs of our study subjects, we ensured that the participants felt comfortable, respected, and accommodated. By consciously avoiding creating a situation where a subject's difficult or aggressive behavior would have made them feel like they “burned a bridge”, that could undermine rapport and lead them to exclude themselves from further follow up, we took steps to safeguarded their continued involvement in the research. In some cases, the flexibility of research staff translated to travelling significant distances to the participant's neighborhoods or workplace, and working in the evening or early morning to ensure contact. Also critical to maintaining relationships in the community and with participants was a consistent effort to be punctual for appointments in institutions (e.g., hospital, residential treatment facilities, corrections facilities), and being respectful of the rules of different administrative environments.

In Vancouver, our intervention arms included services from several existing organizations, and relationship building with intervention staff was crucial to communication with participants. Letters and verbal messages regarding follow-up interviews were very often transmitted through case managers. To illustrate an example, at a congregate housing facility one research interviewer was appointed as the contact person for the majority of the follow-ups. The residents became accustomed to being tracked and interviewed at their home, and a consistent schedule for follow-up was established. The contact person built rapport with residents through being visible at meal times, in groups, or in the garden, and going for coffee or for a walk. The residents became aware of the schedule of the on-site interviewer, and frequently dropped in to the interview room to see when they were next due, or make contact. Communication with staff regarding upcoming interviews proved effective for those who were harder to reach. Over time, the ongoing consistent presence of the interviewer and the tight knit community created a culture of participants being proactive in scheduling their own interviews. The importance of this continuous contact and rapport with residents – but also the incentive paid to the participants - cannot be emphasized enough. In addition, building relationships with staff proved essential to effectively following those residents who had moved out for reasons ranging from incarceration to moving to their home communities, or for participants with issues of acute addiction, mental illness, or aggression. The interviewer would quickly learn from staff of any circumstances impeding follow-up, check in with case managers, and adjust plans to accommodate the participant. In many cases, when a resident had severed contact with the intervention team, the case manager and researcher would work collaboratively to track down the individual and re-establish contact. In several cases the ongoing engagement or re-engagement with the research team led to subsequent re-engagement with the clinical intervention teams and enabled them to work with the clients on their housing and health challenges.

That said, it was also crucial to maintain the distinction between research staff and intervention staff, and to uphold participant confidentiality at all times. This proved beneficial to participant retention in that the interviewer was perceived as being separate from systems of care that were not always perceived to be supportive. The perception of the interviewer as a “safe audience” also helped to ensure more honest and open feedback.

Use of technology in recruitment and retention of participants

In the ongoing process of maintaining contact and communicating upcoming interviews, we employed various technologies, sometimes several modes at once. Telephone, email, texting, social media (e.g. Facebook) and voice over IP services (e.g. Skype) were all employed to good effect, yet every effort was made to protect participants' confidentiality: as telephone contact numbers could very rapidly go out of service (e.g. phones being in and out of service depending on the available funds of the participants), and computers with internet service were available free of charge in a number of facilities and public institutions in the city, email was often the most effective method of maintaining contact. Texting proved to be the most reliable way to contact a participant whose cellphone plans offered free texts but charged by the minute for calls. In one case where a participant moved to Toronto and had no phone, his interviews were conducted via Skype. One individual, in his effort to get clean, purposefully disconnected his phone and neglected his email in an effort to avoid triggers and unhealthy contact with drug using acquaintances. The one mode of communication he maintained during this period was his social media account. In contacting him through this we experienced the unexpected benefit of enhancing rapport and trust, as he was then able to share pictures of his family, himself as an adolescent, and his estranged son with whom he was re-establishing contact. This kind of connection not only helped retain him in follow-up, but instilled a sense of rapport and trust that enabled him to broach difficult subjects that he would previously have declined to address. We were aware that contacting people through social media might interfere with confidentiality, so we did not contact participants through social media accounts on a large scale, but instead used this method as an individualized approach with one participant as per his wish. By doing so, we were very cautious in avoiding any communication around the terms ‘homelessness’ and/or ‘mental illness’, instead we would remind him of his ‘meeting’ or ‘interview’.

For the most difficult to reach individuals, we found that outreach, dropping in to homeless services, or stopping by parks or alleys that participants were known to frequent, knocking on doors of participant's rooms, and transmitting messages by word of mouth was the only hope of retention. Of particular help were the photos, which helped newer interviewers in identifying and tracking participants with whom they were not familiar and who had not been photographed at baseline. As another approach we put up posters in the areas where homeless people are known to be, with a generic reminder to call the study office and check if he/she is a participant of the At Home Study and if they are due for a follow-up appointment, as often times people might not remember that they are part of an ongoing research project that runs over several years.

Incentives

Probably the most effective tool for retaining participants' interest in the study was the use of cash incentives for continued contact. Participants were paid honoraria ranging from $20 to $40 for participating in the interviews, based on an average of $20 per hour, given the fact that some follow-up interviews took longer than others. Added incentives of being taken for coffee or lunch assisted greatly in creating an informal atmosphere, and were critical to engagement in some cases. We also offered bus tickets for those participants who were willing to travel to the field office, and light refreshments at the start of each interview to impart a sense of hospitality and appreciation.

Conclusion

The specific characteristics of homeless individuals necessitate the use of intensive and unconventional strategies to recruit and retain them in longitudinal scientific studies. Many of the procedures described in this paper appear quite individualized and time-consuming. Based on our experience, the intensity of the overall effort is more important than a specific procedure per se. Many study participants will be reached by the first attempt. However, researchers should be prepared to adopt multiple strategies of different intensity to retain individuals in the study.

We believe that one of the crucial factors for creating and maintaining ongoing relationships with our study participants was our staff's contextual knowledge of the environment, cultural norms, conditions, and circumstances of the population and Vancouver's Downtown Eastside. The statistical data describing our retention rates demonstrate that these culturally sensitive procedures have been highly successful. But besides the statistics, we also felt that the behaviors of our study participants indicated the appropriateness of our strategy. For example, evidence of sex work, criminal behavior, drug use or drug dealing could certainly have been an impediment to access, especially when inviting a researcher into personal space (apartment or hotel room), but because of the experience of many of the interviewers in the neighborhood and the rapport established over years of contact, they were invited into rooms where a stranger would have been turned away. Furthermore, we did not have a single incident in the Field Research Office over the course of the study; neither assaults, nor theft, nor breaking into the field office, or the like, except some occasional verbal aggression of the more marginalized participants. Our active and outreaching (rather than office-based only) approach and our good relationship with staff from other services and systems (e.g., forensic) was also essential to the success of the project. Further, our hiring strategy to have individuals on the field research team with exceptional interpersonal skills likely benefitted that we are able to engage even the most marginalized individuals, e.g. participants with absolute (street/shelter) homelessness and with prolonged lifetime homeless histories, which showed different retention characteristics and therefore are likely in need of very flexible and individual retention strategies as described above. Our main conclusion is that longitudinal experimental research projects with vulnerable and hidden populations benefit from extensive outreach work and collaborative approaches that are based on attitudes of mutual respect, contextual knowledge and trust.

Funding

This research was funded by a grant to Simon Fraser University from Health Canada and the Mental Health Commission of Canada.

Trial registration

Current Controlled Trials ISRCTN57595077 and ISRCTN66721740. ISRCTN57595077 - Vancouver at home study: housing first plus assertive community treatment versus congregate housing plus supports versus treatment as usual.

Conflict of interest

The authors declare not conflict of interest.