Recruiting under-represented populations into psychiatric research: Results from the help for hoarding study.

This study compares the effectiveness of approaches used to recruit a diverse sample for a randomized clinical trial for Hoarding Disorder (HD) in the San Francisco Bay Area. Of the 632 individuals who inquired about the study, 313 were randomized and 231 completed treatment. Most participants heard about the study via flyering (N = 161), followed by advocacy groups (N = 113), word of mouth (N = 84), health care professionals (N = 78), online (N = 68), and media (N = 11). However, those that heard about the study via advertising methods, such as flyers, were less likely to complete the study, p = .01, while those recruited via advocacy groups were most likely to be randomized, p = .03. No source proved more effective in recruiting underrepresented groups such as men, p = .60; non-whites, p = .49; or Hispanics, p = .97. Advertising recruited the youngest individuals, p < 0.001, and word of mouth was most likely to recruit unemployed, disabled, or retired individuals, p = .01. Thus, results suggest an ongoing multimodal approach is likely to be most effective in both soliciting and retaining a diverse sample. Future studies should compare recruitment methods across greater geographical regions too, as well as in terms of financial and human costs.

RCT Entities:   Population Interventions Outcomes

Introduction

The success or failure of any randomized clinical trial (RCT) is largely dependent on the recruitment of a sample representative of the pertinent population. Because recruitment methods can disrupt a study's timetable, consume staff hours, and reduce the ability of a study to detect significant differences in treatment [1], recruitment can be the most challenging part of a clinical research study [2]. One particularly difficult challenge is recruiting and retaining groups typically underrepresented in research, such as males, non-white populations, older individuals, individuals of low socioeconomic status, and individuals of low occupational status [2]. These difficulties often result in a lack of diversity among clinical research participants that impedes the generalizability of the studies in question, and slows the eradication of health disparities [3].

To allow for more beneficial, timely, and cost-effective research, it is valuable to examine which recruitment approaches offer the most extensive outreach to underrepresented target populations of interest, especially to those who may be more likely to complete the study. However, few studies address recruitment strategies and retention of various populations in experimental trials and most are solely descriptive [4]. The need for more research on the successful recruitment of under-represented populations is particularly important for disorders that are also under-represented in the clinical trials/treatment literature, for example, psychiatric disorders such as hoarding disorder (HD).

Hoarding disorder is a relatively new psychiatric diagnosis, appearing for the first time as a distinct disorder in the Diagnostic and Statistical Manual of Mental Disorders, 5th ed. (DSM-5). As such, HD presents unique challenges to researchers, as many affected individuals are less likely to recognize their symptoms as part of a medical and/or psychiatric condition warranting treatment or future research. Furthermore, limited insight is common among those with HD, making it even less likely for them to seek treatment [5]. Nonetheless, HD is a prominent condition affecting 2–5% of the general population [6] and approximately 6.5% of individuals over the age of 65 [7], and has a profound public health impact [8,9]. Thus, recruiting representative samples of individuals into treatment studies of HD is crucial for the later generalization of the findings, and for the design of subsequent implementation studies.

The goal of this study was to determine which of a variety of recruitment strategies utilized in a large behavioral treatment study in HD was the most successful. In particular, we aimed to determine: (1) which method(s) of recruitment generated the most initial interest in the study; (2) which method(s) of recruitment resulted in the highest number of participants enrolled and completing the study; and (3) which method(s) of recruitment were most effective in soliciting and retaining under-represented groups in research.

Methods

Study design

This study was the result of a collaborative partnership between the Mental Health Association of San Francisco (MHASF) and researchers at the University of California, San Francisco, and was funded by the Patient-Centered Outcomes Research Institute (PCORI). The full protocol for this randomized-controlled trial comparing psychologist-led group cognitive behavioral therapy to peer-led community based group treatment for HD can be found in “Comparison of a peer facilitated support group to cognitive behavior therapy: Study protocol for a randomized controlled trial for hoarding disorder” [10]. As this was a community-based study, the criteria for participation were deliberately broad and inclusive. Individuals were eligible to participate if they were 18 or older, resided in the greater Bay Area, met clinical diagnostic criteria for HD, and were able to participate in a group setting. Participants with dementia and those who had participated in behavioral therapy for HD in the past year were excluded. Individuals with psychosis, substance use disorders, other psychiatric disorders, and mild cognitive impairment were included if they met the above inclusion criteria. Treatment was provided without cost, and participants were reimbursed for completing the research assessments prior to and following treatment.

Potentially interested individuals were asked to complete a set of initial screening questionnaires to confirm the presence of hoarding behaviors. During this screening, potential participants reported age and gender and completed three questionnaires that assessed hoarding symptoms. Participants who met initial screening criteria were invited for a clinical interview, where they reviewed and signed a written consent form. As part of the initial interview, information on gender, race, ethnicity, marital status, employment status, and insurance status were collected, and research staff assessed the presence and degree of hoarding symptoms as well as rated individuals’ insight into their hoarding symptoms as either fair to good, or poor. Those that were interested and eligible for participation following the clinical interview were randomized to one of the two treatment arms, and completed pre- and post-treatment assessments. The study was approved by the Institutional Review Board of the University of California, San Francisco.

Recruitment methods

The recruitment goal was to enroll and randomize a sample of 300 participants representative of the larger San Francisco Bay Area into the treatment study over the span of two years. Based on prior data collected by MHASF, a 15% drop-out rate was estimated. To recruit eligible participants, the research team implemented a multifaceted approach. Media advertisements regarding the study were placed with local radio stations and newspapers in communities in the Bay Area. Advertisements were also placed on multiple websites, including the MHASF website, hoarders.org, and clinicaltrials.gov (NCT02040805). Referrals were solicited by sending flyers to or directly contacting treatment providers throughout the Bay Area, including the University of California, San Francisco, the Northern California Kaiser healthcare system and the San Francisco Veteran's Administration Medical Center, the San Francisco Hoarding Task Force, among others. Study information was also distributed via listservs, including the MHASF listserv, and the International Obsessive Compulsive Disorder Foundation (IOCDF) local support group chapter's listserv. Information was distributed directly to potential participants by study personnel via the MHA's ongoing-drop-in support groups for HD, the annual International Conference on Hoarding and Cluttering organized by MHASF, and outreach events conducted by MHASF's peer facilitators. Finally, the research team distributed flyers on a regular, bi-weekly, recurring basis at a wide variety of locations including coffee shops, laundromats, libraries, community centers, senior living centers, and apartment buildings.

The flyer, aimed towards individuals who had problematic hoarding problems but who might not have recognized either the term “hoarding” or the diagnosis “hoarding disorder”, targeted individuals who identified as being “overwhelmed with clutter” (although the term “hoarding disorder” was mentioned) and described the study, eligibility criteria, treatment, and compensation (Fig. 1).

Fig. 1

Flyer used in recruitment.

Source classification

Participants reported how they learned of the study upon initial contact and screening. Responses were then classified into one of the following four categories: health care professionals, advocacy groups, advertising, or word of mouth. Advertising was then further divided into three subsets including flyers, online, and media (newspaper or radio). If a response could belong to more than one category, the most direct source was consistently classified as the primary referral source to better identify how participants are being reached and to whom advertising should be directed. For instance, if an individual was informed via word of mouth by someone who initially saw a flyer, word of mouth was classified as the primary source, with flyer being secondary. If an individual was informed via a flyer in a medical office or advocacy organization, flyer was classified as the primary source, with the distribution method being secondary. Additionally, to identify the most common advertising locations, flyer locations and individual websites were noted whenever possible. Flyer locations included health clinics, advocacy groups, community and senior centers, and community businesses, such as coffee shops, grocery stores, etc., while online sources included advocacy group websites, listservs, clinicaltrials.gov, or unspecified sources such as Google, Eventbrite, hoarders.org, Craigslist, etc.

Analysis

Outcomes of interest included initial contact by the participant to the research team and completion of screening questionnaires, eligibility/completion of baseline clinical interview, randomization into the treatment study, and completion of the treatment study and post-treatment assessments. Raw data were first examined to determine which source of recruitment had the largest outreach in terms of the number of individuals who contacted the research team. To determine possible associations between recruitment sources and the other outcomes of interest, three independent chi square tests were conducted for the four primary sources and 1) eligibility, 2) randomization and inclusion, and 3) study completion. Next, a series of models were estimated and tested to determine whether any sources were significantly more effective than others in recruiting various underrepresented groups. Chi square tests were conducted for recruitment sources and gender, ethnicity, race, occupational status, and insight. To determine whether a specific source was particularly effective for older individuals or individuals of lower educational status, one-way ANOVA tests were conducted.

Results

Demographic characteristics

Fig. 2 shows the participant flow from initial interest through study completion. Over 600 individuals indicated initial interest in the study, 476 were screened, 323 were randomized into one of the two treatment arms, and 231 completed the study.

Fig. 2

Participant retention and attrition.

Seventy-three percent of the 476 individuals who were screened were women, and 10% percent of those screened self-identified as Hispanic or Latino. The mean age of those screened was 58.5 years (21–92, SD = 11.8), and 63.0% identified as white, 8.0% identified as African American, 12.1% as Asian, and 17.3% as mixed race or other. Fig. 3, below, illustrates the age and race frequency of screened individuals compared to those residing in the Bay Area according to the most recent census data [11].

Fig. 3

Comparison of ages and races of Bay area population and screened sample.

The mean level of education was 15.3 years (range 9–20, SD = 2.3), while 42.9% were privately insured, 49.7% publicly insured (e.g., Medi-Cal/Medicaid or Medicare) and 7.4% had no insurance or reported being inadequately insured. Approximately 29.6% of screened individuals were employed, 25.4% were unemployed, and 38.1% were disabled or retired, while 7.0% reported other occupational status.

Initial inquiries

Of the 632 individuals who inquired about participating, 515 (%) individuals reported how they heard about the study. Most reported hearing about it through advertising, most commonly flyers (N = 161), followed by advocacy groups (N = 113), word of mouth (N = 84), health care professionals (N = 78), and online (N = 68) (Fig. 4). Media sources such as radio and newspaper ads recruited the fewest individuals (N = 11). Only 3.5% of the individuals reported both a primary and secondary source, with the most common being word of mouth (N = 14) and flyers (N = 7). Of the 67 individuals who specified where they had seen flyers, 60% reported seeing a flyer in a community center or senior center, as opposed to health clinics (17%), advocacy groups (5%), or community businesses (14%) such as coffee shops, or laundromats. Among the 71 individuals who reported finding study information online, 25 specified that they found it via a listserv, 6 via an advocacy group website, and 1 via clinicaltrials.gov, with the remainder not providing the specific website.

Fig. 4

Referral sources of those indicated interest.

Recruitment methods and retention

Fig. 5 shows the percent of participants by recruitment source for each study stage. A total of 413 individuals were determined eligible for the study, and there were no differences between recruitment sources in terms of eligibility, p = .49. However, advocacy groups recruited the largest proportion of individuals who were ultimately randomized into the study, p = .03. Although those who found out about the study through advertising via flyers, online, newspaper, or radio were more likely to make initial contact, these individuals were significantly less likely to complete the study than were those recruited from other sources, p = .01.

Fig. 5

Percentage of individuals from each source at each stage of the study. Columns indicate the percent of potential participants in each stage broken down by type of recruitment.

Recruitment by demographics

As noted above, the sample was predominantly women, was racially and ethnically diverse, and was older and more highly educated than the general population of the Bay Area according to the 2010 U.S. census [11]. Table 1, below, describes the demographics of the screened individuals by source of referral. Of the recruitment methods used, no specific approach recruited significantly greater proportions of men, p = .60; non-whites, p = .49; or Hispanics, p = .97. Similarly, no specific method recruited a greater proportion of individuals with relatively poorer insight into their hoarding symptoms, p = .81. However, two sources were significantly associated with the recruitment of two populations of interest. First, a one-way ANOVA showed a significant difference between the mean ages of those recruited by various sources, p < 0.001. A post-hoc Tukey test showed that advertising via flyers recruited significantly younger individuals than advocacy groups, p = .02, or word of mouth, p = .02.

Table 1

Demographics of screened individuals by recruitment source. Note: Parentheses indicate within-column percentages unless otherwise specified. Data for gender and age were collected on all individuals (N = 515). All other variables were collected for randomized participants only (N = 323). Missing/not available reflect the number and percent of those with data for the given variable who did not indicate a referral source.

	Recruitment SourceNumber (% of total in each referral category)				Missing/Not Available	P-Value
	Health Care Professional (N = 78)	Advocacy Groups (N = 113)	Advertising (N = 240)	Word of Mouth (N = 84)
Gender (N = 515)
Male N = 152 (29.5)	21 (28.3)	26 (23.6)	67 (29.3)	26 (31.7)	0 (0)	.59
Race (N = 323)					79 (24.5)	.55
White	34 (70.8)	47 (63.5)	71 (58.2)	30 (61.2)
Black	3 (6.3)	4 (5.4)	12 (9.8)	3 (6.12)
Asian	1 (2.1)	10 (13.5)	17 (13.9)	8 (16.3)
Other	10 (20.8)	13 (17.6)	22 (18.0)	8 (16.3)
Ethnicity (N = 323)
Hispanic/Latino	5 (10.2)	7 (9.1)	13 (10.5)	6 (11.8)	79 (24.5)	.97
Mean Age (SD) N = 515	59.6 (9.8)	60.2 (10.3)	56.4 (12.4)	60.6 (12.2)	0 (0)	.001
Mean years of education (SD) (N = 323)	15.4 (2.4)	15.0 (2.3)	15.5 (2.3)	15.0 (2.3)	4 (1.2)	.38
Insurance status (N = 323)					5 (1.5)	.52
Privately insured	27 (50.0)	28 (35.0)	62 (46.6)	24 (47.1)
Publicly insured	25 (46.3)	46 (57.5)	60 (45.1)	24 (47.1)
Not adequately insured	2 (3.7)	6 (7.5)	11 (8.3)	3 (5.9)
Employment status (N = 323)					10 (3.1)	.09
Employed	13 (24.5)	22 (27.2)	50 (39.1)	9 (17.7)
Unemployed	13 (24.5)	20 (24.7)	31 (24.2)	13 (25.5)
Disabled/Retired	24 (45.3)	35 (43.2)	35 (27.3)	25 (49.0)
Other	3 (5.7)	4 (4.9)	12 (9.4)	4 (7.8)
Insight					23 (7.1)	.71
Fair to Good	45 (90)	65 (84.4)	109 (87.9)	41 (82)
Poor	5 (10)	12 (15.6)	15 (12.1)	8 16.3

Secondly, word of mouth recruited more individuals of lower employment status, including those who were unemployed, disabled, or retired, p = .01.

Conclusions

Discussion

This study uses data from a recently completed clinical trial to identify the recruitment methods that were the most effective for engaging and retaining a diverse group of participants in psychiatric treatment outcomes research. We found that the most effective approach in attracting interest was repeated flyering, followed by targeted outreach by an advocacy group that was known for its work in the disorder of interest (in this case, hoarding disorder). The variety of approaches used allowed us to engage over 600 potential participants, screen nearly 500, and treat over 300 in a period of less than two years. We were also able to recruit a diverse group of participants, and our sample was fairly representative of the larger population of the San Francisco Bay Area, with one exception—we were not successful in recruiting equal numbers of males and females into the study, but as with many other clinical trials, had a predominance of women participating.

Interestingly, advertising via traditional media (radio and newspaper), one of the most expensive but least time-intensive methods, had the smallest outreach, while, advertising via flyers achieved the largest outreach. One possible reason for this is that we posted flyers in a variety of community-based locations (such as coffee shops, community centers, senior centers, and apartment complexes) and continued to re-post throughout the course of the study, while the traditional media ads were played or printed for a limited number of weeks. With limited resources for advertising, we hypothesized (and data has supported) that spending the person-time to repeatedly flyer was more likely to be successful for the population of interest. We also noted that those recruited from advertising were less likely to complete the study than those recruited via other sources, even though there were no differences in eligibility. A lack of active motivation to pursue and complete treatment may have played a role in this difference. Unlike several of the other methods (word of mouth, clinician referral, and advocacy group outreach), advertising does not involve direct person-to-person contact, which may decrease the feeling of accountability felt by the potential participant to follow through with treatment.

Both fortunately and unfortunately, examination of the methods’ success in recruiting underrepresented groups yielded few concrete solutions regarding better including these populations in research. Although no approach outperformed any other in targeting men, non-whites, or Hispanics, word of mouth proved most effective in recruiting unemployed, disabled, and retired individuals. This is especially worth noting for a population with HD, as the majority are older individuals, and are disproportionately disabled or unemployed [9]. Fortunately, the approaches we used resulted in a diverse participant pool, one that was, in almost all respects, representative of the general population in the relevant geographic area. These results suggest that an ongoing multimodal approach, with a few key modifications to target specific populations of interest, is likely to be needed in recruiting a diverse group of participants for psychiatric research. Successful recruitment is likely to include low-cost and easily accessible advertising such as flyers or brochures distributed in areas where the population of interest is known or thought to frequent or reside and may be addressed towards both the affected individual and his/her family or friends. Thus, future studies may consider comparing the effectiveness of different flyers or brochures and how they recruit different demographic populations. Investment of time and money in recruitment is an important component of a successful clinical trial, but thoughtfulness is required regarding the prioritization of the many possible avenues currently available.

Limitations

Although this study contributes important information regarding successful recruitment into a psychiatrically focused treatment study, it has several potential limitations. First, the study focused on a large, fairly urban, geographical region, the San Francisco Bay Area, and may not be generalizable to other geographical regions or to more rural settings. Additionally, this study did not directly assess the amount of time and/or money invested in any specific recruitment method and thus does not provide suggestions as to what method of recruitment may be the most timely or cost-efficient. Future studies involving individuals with HD in different settings may find that different approaches work better, and additional work should be done to directly examine the costs of recruitment, both financial and human.

Disclosures and acknowledgments

Dr. Mathews has received grant funding from the NIH and travel reimbursement and speakers' honoraria from the Tourette Association of America (TAA), as well as honoraria and travel reimbursement from NIH for serving as NIH Study Section reviewer. Dr. Delucchi receives research support from the NIH and honoraria and travel reimbursement from NIH for serving as NIH Study Section reviewer. Dr. Mackin receives research support from the National institute of Mental Health, National Institute of Aging, the Hillblom Foundation, Janssen Pharmaceuticals (research grant), and the Alzheimer's Association. Dr. Mackin has also received travel support from the National Institute of Mental Health for Workshop participation. Dr. Tsoh receives research support from NIH, PCORI and the California Tobacco Related Research Program, and honoraria and travel reimbursement from NIH for serving as NIH Study Section reviewer. All other authors report no conflicts of interest.

Funding/support

This work was supported through a Patient-Centered Outcomes Research Institute (PCORI) Assessment of Prevention, Diagnosis, and Treatment Options Program Award (#CE-1304-6000). The funder had no role in the design and conduct of the study; collection, management, analysis and interpretation of the data, preparation, review, or approval of the manuscript, or decision to submit the manuscript for publication.

Disclaimer

All statements in this report, including its finding and conclusions are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee.