Recruitment cost and outcomes for an arthritis work disability prevention randomized clinical trial: The Work It study.

BACKGROUND: Despite the recommendations to increase recruitment of participants into clinical trials, investigators face costly challenges in trials investigating work disability interventions for people with arthritis and rheumatological conditions. This study aims to evaluate the recruitment costs and outcomes from a randomized controlled trial of an arthritis work disability prevention program conducted between 2011 and 2015, to inform planning and monitoring recruitment in similar studies.
METHODS: Data were obtained from enrollment and financial records pertaining to recruitment costs for each recruitment approach employed. The cost for each recruitment method was calculated for total cost and cost per number of participants screened, eligible, and enrolled in the trial. Then the yield of each possible recruitment method was also determined based on the ratio of the number of randomized participants divided by the number of people contacted through each recruitment method. Finally, the.
RESULTS: Recruitment rate was lower than projected. Community advertising, specifically newspapers, was the most successful method of recruitment in terms of numbers, but social media, specifically Craigslist, was the least costly method used to recruit. Some social media approaches, including Facebook and LinkedIn, yielded few if any participants. Recruitment efforts used successfully in the past are not always effective.
CONCLUSIONS: Costs to recruit large numbers of people with arthritis into clinical trials are high. Investigators are encouraged to monitor recruitment efforts and evaluate the costs and outcomes of their strategies throughout the study period. Close consideration to recruitment costs should be considered as part of the research fiscal resources prior to and during the study period for long-term outcomes like work disability. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT01387100, date: 06/01/2011.

Background

Randomized controlled trials (RCTs) are widely accepted as the gold standard in evaluating the effectiveness of health-related interventions [1]. One of the significant obstacles investigators encounter while conducting RCTs is poor recruitment [[2], [3], [4], [5]]. A systematic review found that nearly two-thirds of 114 trials reviewed did not meet recruitment goals, and over half of the studies failed to finish on time [4]. Other reviews of controlled trials reported that between 30 and 55% of studies recruited the required sample size in the needed timeframe [6,7]. Failing to recruit the required sample size for an RCT can result in making the study statistically underpowered to detect a meaningful effect of the tested intervention, possible termination of the study, and substantial financial costs.

Despite recommendations to increase recruitment of participants into clinical trials [[8], [9], [10], [11]], investigators still face this costly challenge resulting in difficulty answering their clinical and scientific questions. This problem persists at a greater level when outcomes require a more extended study period, such as work disability. Thus, while clinical trials are critical to identifying effective interventions and strategies for chronic diseases, challenges remain to recruit adequate sample sizes. Particularly, trials evaluating the efficacy of novel interventions to prevent work disability, a significant problem among people with chronic rheumatological conditions, require large sample sizes due to the complexities in measuring work disability and the relatively rare occurrence of this event over a short period. Researchers, however, have little empirical evidence to inform budget and guide recruitment approaches for these types of studies [7,11,12].

Arthritis and related rheumatological conditions commonly manifest in pain, tissue damage, and activity limitations and RCTs commonly evaluate outcomes of pharmacological and non-pharmacological treatments. Minimizing and preventing work disability is vital among people with arthritis since 20–40% are unemployed due to their health within ten years of being diagnosed [[13], [14], [15], [16]]. RCTs examining interventions for arthritis work disability outcomes need a high number of person-years to examine ‘events’ which are typically rare over short-term periods. The need for a large sample size requires aggressive and efficient recruitment approaches when the study begins. In 2019, Kakumanu et al. found that the cost of recruiting a single participant in a community-based trial was CAD 88 (about USD 70) per participant [17]. This makes early planning and adequate and intentional allocation of funds to support the breadth of recruitment strategies essential to ensure effective recruitment. However, to our knowledge, there are no studies that examine recruitment efforts and their costs among people with arthritis who are at risk of work disability. Knowledge about the yield of recruitment from specific sources and costs and outcomes per participant recruited from specific approaches would be helpful in planning and implementing recruitment efforts. To address this gap in the literature, we evaluate the recruitment costs and outcomes from an RCT examining the efficacy of an arthritis work disability prevention program conducted between 2011 and 2015. The trial titled “Efficacy of a Modified Vocational Rehabilitation Intervention” is also known as the “Work It” study as a means to provide supportive information for planning and monitoring recruitment in similar studies.

Methods

Study recruitment protocol

The “Work It” Study is a randomized controlled trial of people with self-reported arthritis or related rheumatic conditions at risk of work loss. The detailed methods of the trial protocol and findings are published elsewhere [18,19]. This clinical trial was approved by the Institutional Review Board for Boston University, and all participants provided written informed consent. A sample size of 350 was initially proposed with a 2-year timeline. Based on previous studies implemented by the investigative team, we anticipated we would enroll 15 people per month.

The initial inclusion criteria were: a) age between 23 and 63 years, b) self-report of doctor-diagnosed arthritis or a rheumatological condition (osteoarthritis, rheumatoid arthritis, lupus, fibromyalgia, scleroderma, ankylosing spondylitis, psoriatic arthritis, juvenile arthritis, Reiter's syndrome, polydermatomyositis, vasculitis, and gout), c) self-reported concern about the ability to remain employed (assessed with the following question: “Do you have any concern about your ability to remain employed now or in the next few years due to your health condition?” A ‘yes’ response indicated concern, d) currently employed at least 15 h per week, and e) living or working in eastern Massachusetts. Exclusion criteria were: a) on worker's compensation or disability leave at the time of the telephone screening, b) planning to stop Work in the next two years, c) participating in other research intervention studies related to employment, or d) unable to speak and understand English. At the beginning of the second year of the study, eligibility criteria were modified: i) self-report of chronic back pain or chronic fatigue were added as acceptable diagnoses, ii) age range was expanded to 21–65, and iii) the geographical recruitment area was expanded to all of Massachusetts.

Sampling frame and recruitment approaches

Clinical and community approaches were planned at the beginning of the study based on a previous clinical trial with successful recruitment by the investigators [20]. The following approaches were proposed and implemented during years one and two of the recruitment period.

Year 1

Medical approaches

Rheumatological Practices: Nine rheumatological practices from across the Greater Boston metropolitan area agreed to assist with recruitment by displaying brochures in waiting areas and treatment rooms and recommending the study to patients who may be eligible. A tri-fold flyer was developed with photographs representing teaching, industry, and manufacturing professions, along with a brief description of the purpose of the study, eligibility criteria, and how to contact the study team if interested. Based on the proposed methods in the grant proposal, the investigators offered rheumatology practices $500 to support staff time for recruitment efforts for the study. Some practices accepted payment; some waived or reduced the compensation. Rheumatology practices continued recruitment efforts in year 2; however, they all waived the compensation fee for year 2 of the study. Occupational and physical therapy practice sites were added to the health care practice recruitment procedures; however, these practices were not compensated financially.

Community approaches

Flyers: Flyers with the text “Are you employed and have arthritis or another rheumatic condition and have concerns about your ability to remain employed due to your health?” were developed and printed for wide distribution. Flyer printing costs were $1674. Flyers were placed in multiple locations throughout greater Boston and suburbs, such as libraries, community centers, supermarkets, public notice boards, cafes, laundromats, hospitals, clinics, colleges, etc. In addition, flyers were handed out at commuter rail stations at rush hour and farmer's markets during lunch hour.

Newspaper advertisements. Newspaper advertisements primarily occurred in a community newspaper with wide distribution to people using public transportation to and from Boston, the “Metro”. The Metro advertisements ran for 16 months. Study staff consulted advertising staff at the Metro to devise an optimal advertising strategy given the constraints and resources of the recruitment budget. Initially, an advertisement ran twice monthly. Study staff consulted the Metro staff to determine the optimal strategy for study recruitment through this source. The recommended plan was advertising four times per month for four additional months and then decreasing advertisements to twice a month as readership became more familiar with the study. Total newspaper advertisement costs amounted to $12,738, which resulted in the most expensive recruitment method in total dollar amounts but also generated the highest levels of traffic and interest. Advertisements were also published once in other local newspapers that had fewer readerships. A print advertisement ran in the Boston Globe Sunday paper on one Sunday with an online advertisement posted for one million impressions over one week.

Community Website Postings (Craigslist): Multiple listings were posted weekly to a community website, Craigslist. The postings were listed in two different sections: the “jobs” category ($25 payment per post was needed to post in this section totaling $4655 in costs paid to Craigslist) and the volunteer category (free) throughout greater Boston area North Shore, South Shore, and Worcester. The “job” category postings were rotated through the different job types from month to month.

Foundations and Support Groups: Professional foundations (e.g., the Arthritis Foundation, Lupus Foundation, and Scleroderma Foundation) and arthritis-related support groups were contacted by telephone and email and given information on the study. Interested groups were mailed packets of flyers and pamphlets to be given out at their meetings. Support groups included in this approach were Lupus, Scleroderma, Arthritis, Fibromyalgia, Massachusetts Teacher's Association, New England Black Nurses Association, and Church Congregations. Study staff attended several support groups and discussed the study when invited by the support group. Information provided to support groups ran from September 2011 through December 2013. These support groups were included in the “other” recruitment category, alongside human resource departments and emails sent to hospital nursing staff with costs totaling $849.

Social Media: Social media was attempted but not utilized heavily. Twitter and Facebook were used early during recruitment, but readership with social media strategies was low at the beginning of the initiation of the study. One paid posting was listed per the study period on both LinkedIn and on Facebook.

Year 2

With the exception of the social media recruitment strategies, all of the above approaches were continued in year 2 of recruitment. In addition, the following approaches were added to broaden our sampling efforts:

Medical Registry: A medical registry was accessed from the hospital affiliated with Boston University and used to contact participants about the study. This registry is for people who are interested in participating in medical research. The registry was relatively new, and as such, 343 participants with arthritis and related rheumatological diagnoses were identified and contacted, either by letter or by email (participants’ preference), with detailed information about the research study, including a study brochure. Total costs for the medical registry approach amounted to $341.

Physician Letters: Two rheumatologist physician practices mailed letters (n = 650) directly to patients. The letters contained information about the study from the physical and the principal investigator as well as a study brochure. A self-addressed and stamped postcard could be returned if interested, and costs totaled $1780 for this approach.

Direct Mail: A data management company using direct mail marketing was employed, costing a total of $4009 for this approach. The company identified users with self-reported arthritis who were thought to be employed living in zip code areas we specified.

Snowballing: Participants in the study were mailed a small postcard with the study information and were asked to give the card to someone they knew who might be interested in the study.

Opt-in approaches were used for all recruitment strategies; Interested participants contacted the study staff by telephone or email and were screened over the phone.

Analytical approaches

The yield of each possible recruitment method was also determined based on the ratio of the number of randomized participants divided by the number of people contacted through each recruitment method. The costs of each recruitment method were evaluated by calculating the cost per participant screened, eligible, and enrolled. Costs for all recruitment strategies were obtained from grant expenditure records (e.g., receipts and invoices) and were checked twice to assure accuracy in coding. Costs that non-grant funds supported (e.g., small printing jobs) were established based on the number of printed copies and the established printing charge. Staff time was not included in the cost calculations for this study.

Results

Six hundred fifty-two people were screened over two years; 493 participants were eligible to participate in the study (76%). Reasons for ineligibility were: unemployment, employment less than 15 h per week, a medical condition that did not meet eligibility criteria, and no concern about the ability to remain employed due to their health condition. Of the eligible participants, 319 people returned their consents (64%), and of these, 293 people completed their baseline interview. A total of 287 people were randomized into the study; 6 people were not eligible at baseline and were not randomized into the study. Fig. 1 shows the flow diagram of the recruitment procedures for the study.

Fig. 1

Flow chart of the “Work It” recruitment procedures.

After six months of recruitment, the rate of enrollment and randomization was only six participants per month (46% of the proposed 15 participants per month), revealing that the initial recruitment methods were insufficient. (See Fig. 2). As a result, the initial eligibility criteria were modified. A staff member was dedicated solely to support recruitment efforts, and more recruitment methods were introduced in year two.

Fig. 2

Proposed and Actual Recruitment Timelines for the “Work It” sized Controlled Trial.

Study participants were enrolled through community and medical sources, with 15% of participant's recruitment through clinical approaches and 85% of participants recruited through community approaches. In terms of the specific approaches, advertisements in newspapers resulted in the greatest number of participants (n = 78; 27% of the total enrolled sample), and online advertising websites (e.g., Craigslist) (n = 65; 23% of the total enrolled sample) resulted in the following highest number of participants. However, both of these approaches resulted in a lower proportion of participants enrolled and randomized per number screened than other methods used in this study. (See Table 1). In contrast, physician letters to patients and direct marketing approaches resulted in lower total screened participants, yet higher percentages of enrolled and randomized per person screened. (See Fig. 3). The yield for using a medical registry was 8.3%, physicians' letters were 3.4%, and direct marketing was 1.4%. The yield for other methods (e.g., flyers, community websites) was unavailable as the number of people reached could not be determined. (See Table 2).

Table 1

Recruitment methods breakdown.

Recruitment Yield	Medical Registry	Letters from phys.	Brochures in offices	Newspapers	Internet	Direct mail marketing	Snowball/friend	Other community	Flyers
N (%)
Screened	21	38	22	196	176	47	21	63	68
Screened eligible	15 (71)	28 (74)	14 (64)	143 (73)	139 (79)	38 (81)	17 (81)	53 (84)	45 (66)
Randomized/enrolled	9 (60)	26 (93)	10 (71)	78 (55)	65 (47)	27 (71)	8 (47)	36 (68)	28 (62)
Enrolled per month activea	2.3	4.3	0.6	4.3	2.8	3	1.3	1.3	1.3
Active months	4	6	18	18	23	9	6	27	22

Adjusted rate of subjects enrolled per month of active recruitment method. See methods section for more detail.

Fig. 3

Participants per recruitment method.

Table 2

Recruitment costs breakdown by method.

Method	Total Cost ($)	Cost per screened ($)	Cost per screened eligible ($)	Cost per randomized ($)
Medical Approaches
Rheumatological Practices	1750.00	79.55	125.00	175.00
Medical registry	341.00	16.24	22.73	37.89
Letters from physicians	1780.00	46.84	63.57	68.46
Community Approaches
Flyers	1674.00	24.62	37.20	59.79
Newspapers	12,738.00	64.99	89.08	163.31
Internet Ads (Craigslist)	4655.00	26.45	33.49	71.62
Social media	36.74	NA	NA	NA
Direct mail marketing	4009.00	85.30	105.50	148.48
Snowballing/Friend	0.00	0.00	0.00	0.00
Othera	849.00	13.27	15.72	23.58
Total	27,832.74	42.69	46.94	96.98

Other: contacting local disease-specific support groups, foundations (e.g., Arthritis foundation), contacting human resource departments, and email sent from hospital to nursing staff).

The total recruitment cost of the Work It study was US$27,832.74 (see Table 2). The least expensive recruitment method per person enrolled was the use of a medical registry ($16.24 per participant screened; $37.89 per participant randomized) and letters sent directly from physicians to participants ($46.84 per participant screened; $68.46 per participant randomized). The recruitment category, “other,” was the least expensive in terms of cost per screened and randomized ($13.27 and $23.58, respectively); however, this category includes a combination of approaches, making it difficult to perform specific cost calculations by subcategory (e.g., participants recruited through foundations). In contrast, the most expensive method of recruitment was displaying informational brochures in waiting areas of doctors’ offices ($79.55 per participant screened, $175.00 per participant randomized), and distribution of flyers in prominent public areas ($88.68 per participant screened, $215.36 per participant randomized).

Discussion

Recruitment for the Work It RCT on work disability recruited 82% of the proposed sample size over 20 months. Recruitment costs and outcomes per participant differed across approaches. The proposed recruitment timeline for the study was two years. Our early recruitment efforts were not meeting our anticipated numbers, so sampling frames were expanded over the two years, recruitment methods were critically reviewed for optimal outcomes, and changes were made to the study staff.

The initial Work It study recruitment methods, based mainly on recruitment through rheumatologists' offices, Craigslist, and newspaper advertisements, were planned based on recruitment outcomes from a previous intervention study with a similar sample size and timeline conducted by the study investigators [21]. In this study, recruitment through rheumatologist's offices was very successful, with receptionists and physicians letting patients know about the study and patients calling the study number if interested. A similar approach was used in the Work It study, including paying rheumatology offices; however, the enrolled sample from this approach in our study was very low. Because few participants were recruited in this manner and nine practices were paid, costs per participant recruited were high. Costs for this approach would have been even higher if the physicians' required financial support for the second year of recruitment, which they all deferred because of the low yield from this approach. The Health Information Portability and Accessibility Act Legislation (HIPAA) could explain some of the challenges in recruiting from physician offices, though many studies have experienced difficulty recruiting from physician offices in this manner.

On the other hand, two medical approaches—a medical registry and letters mailed directly to patients by a physician—were strategies that achieved the highest enrollment yield showing similar trends in the literature [17]. These approaches may have allowed a more direct targeting of people who would be eligible for and benefit from the study, and therefore, a more significant number of people enrolled per screened. Both of these strategies were cost-effective as well, making them appealing approaches for use in similar studies. Nonetheless, we would not have met our sample size requirements relying on these recruitment approaches alone.

Community approaches, particularly newspaper advertisements, Craigslist postings, direct marketing, and flyers, resulted in the largest numbers of enrolled participants, yet these approaches were also the most expensive. Direct marketing yielded many enrolled participants in other studies [22,23], though the costs per person enrolled were not reported. In our study, direct marketing was one of the more costly approaches. Newspapers were also expensive, though, for our study, it was successful. Our flyering approach was expensive and did not result in large numbers, mainly since this approach was used throughout the recruitment period.

As with numerous studies encountering reduced recruitment outcomes, we expanded our sampling frame by adding more approaches throughout the recruitment period. In addition, similar to other studies, we expanded our eligibility criteria and hired staff explicitly dedicated to supporting recruitment efforts [7,9,11]. While ideally the appropriate recruitment approaches would be implemented at the beginning of the study, our study also shows that close monitoring and thoughtful oversight of recruitment approaches throughout the recruitment period is necessary to achieve optimal outcomes.

When recruiting for a randomized controlled trial, the nature of the intervention and follow-up period could influence people's decision to enroll in a study. For the Work It study, the intervention had a minimal burden on the participants. It only required a one-time in-person meeting followed by follow-up phone interviews lasting approximately 15 min. Furthermore, the participants required little travel as they were assigned to interventionists based on their work/home location. Follow-up data collection interviews were conducted over the phone, making the study less burdensome for participants. However, a longer follow-up period (2 years for the Work It study) and a small compensation amount (USD 80 over two years) could have negatively affected the recruitment rate. Additionally, although the Work It study intervention does not involve disclosure of the participants' health conditions to their employers, some participants may have opted out of the study due to fear of negative repercussions if their employer learned of their health conditions and/or the difficulties they face in the workplace.

Numerous factors are identified in the literature that could help or hinder recruitment efforts. Factors that can facilitate recruitment include 1) support from a clinical research methods center, 2) paid staff at recruitment sites, 3) dedicated recruitment staff on the study, 4) subject involvement in establishing recruitment protocols, 5) establishing expected recruitment numbers based on pilot trials, 6) staff support and close recruitment monitoring, 7) multicenter studies, 8) trial incentives, 9) active treatment as a control, 10) fully-funded study, 11) short duration of follow-up, 12) high community interest in the study, 13) experienced study investigator and steering committee, and 14) opt-out approaches (i.e., the recruitment staff contacts the potential participant and the participant states he/she is not interested in the study) [[4], [5], [6]].

Study recruitment staff report high levels of competition for study subjects, time strain between clinical care and recruitment efforts, and recruiter perceptions of subject benefit and burden can impact recruitment outcomes [24]. However, the empirical evidence examining any of these specific factors is limited and what is available is mixed. For example, incentives to study participants increased recruitment in one study [25] but had no impact in another [4].

While providing valuable information to plan recruitment methods in RCTs examining similar interventions, our study has several limitations. First, the yield could not be calculated in entirety for all of the recruitment methods, as it was impossible to know the total number of people it reached (e.g., newspapers). Second, the method of data collection resulted in a recruitment method category called “other,” which consisted of a pooled set of recruitment strategies; hence, we could not obtain the cost and outcomes of each particular method within this pooled group (e.g., number of screened eligible via contacting disease specific support groups). Third, this study was conducted in the United States within a specific disease population, making some of the challenges faced in this study (e.g., HIPAA regulations) not generalizable to other countries. Fourth, the costs of staff time were not included in the calculations, making a comprehensive estimate a challenge.

This is the first study to evaluate the cost outcomes of recruitment strategies implemented across medical and community settings for a randomized controlled trial for people with work limitations due to their arthritis (the Work It study). This study provides information that may be used by future investigators to appropriately plan and budget for recruitment approaches. Overall recommendations for this type of study are: 1) both clinical and community recruitment approaches are likely necessary, 2) adequate funds need to be budgeted in the grant to support the costs of varied approaches, 3) close monitoring of recruitment outcomes is critical, and 4) strategic changes in the recruitment approaches may need to be considered promptly in order to achieve adequate sample size.

Ethics approval and consent to participate

Ethical approval for this study was approved by the Institutional Review Board for Boston University and all participants provided written informed consent. The trial was registered at ClinicalTrials.gov under the number NCT01387100. Dates of trial: June 2011–March 2016.

The funding body has a role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

National Institute on Disability, Independent Living, and Rehabilitation Research grant number 90RT5009–01. Date: 10/1/10-9/30/17.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Authors' contributions

RA, JK, IH, MV contributed to the data analysis. RA, JK, MV, and IH had major contributions to the write-up of the paper and the development of the tables and graphs. All authors read and approve the final manuscript.