Recruitment challenges in stroke neurorecovery clinical trials.

There are multiple available treatments to enhance stroke rehabilitation, although few interventions have confirmed significant clinical improvements on motor function in pivotal Randomized Clinical Trials. Development of large Randomized Clinical Trials is limited by several barriers and low enrollment rate is considered an important factor. Consequently, most of the evidence comes from small sample size studies, often leading to limited conclusions. According to the National Institute of Health (NIH), about 80% of clinical trials in the United States do not achieve their timelines, increasing research costs and postponing regulatory approval of new therapies. Given that the success of a Randomized Clinical Trial is dependent on enrolling an adequate number of subjects, effective strategies to enhance recruitment rates are highly desirable. In addition, given the resources and time limitations, it is important to understand which strategies are most cost-effective. In this manuscript, we summarize and discuss nine recruitment strategies used in an NIH R21 sponsored clinical trial, including medical records review and online advertising, among others. In addition, we developed an index to compare the time spent benefit of each approach and guide the allocation of the recruitment efforts. For this trial, online advertising and referral from health care professionals other than physicians were the strategies with greater time-benefit, leading to the largest number of stroke subjects enrolled.

Introduction

Recruitment is a major determinant of a successful clinical trial. Despite the importance of recruitment and adequate enrollment in clinical research, numerous stroke trials still fail to achieve their initial target sample size, which can negatively affect the validity of study results [1]. Poor recruitment often delays the development, evaluation and further approval of new rehabilitative treatments to diminish stroke related disabilities [2]. In fact, previous clinical trials in stroke have presented a substantially low recruitment yield. A recent systematic review published in 2017 has reported that recruitment efficiency in stroke trials has not increased over the past 25 years and, if changed, has decreased [3].

Although some clinical trials involving stroke populations have shown a satisfactory recruitment yield, greater than 80% [4], the wide majority of stroke clinical trials report low recruitment yield. A recent study of Koh et al. [5] reported a recruitment yield of only 4%, with a total of 25 enrolled subjects out of 618 screened contacts. Likewise, a stroke trial published in 2012 enrolled a total of 41 participants out of 532 screened individuals, accounting for a recruitment yield of approximately 8% [6]. Additionally, a meta-analysis on medical intervention for acute ischemic stroke has highlighted the trial entry criteria, the number of study centers and the type of intervention as important predictors of recruitment efficiency [7]. Interestingly, the inclusion/exclusion criteria of a trial has been identified as a determinant of a sufficient subject accrual for more than thirty years [8].

Clinical investigators in the stroke field need to overcome recruitment barriers, as to avoid insufficient enrollment rates, reduction of statistical power and an increase of costs. In fact, enrollment failure or delay can lead to meaningful additional costs, but recruitment itself is already considerably expensive [9,10]. When it comes to stroke trials assessing new rehabilitative methods, such as Non-Invasive Brain Stimulation (NIBS), recruitment obstacles are even more significant, since proposing an alternative to the conventional medication/therapy treatment might seem daunting to physicians and patients in nonacademic settings.

The implications of poor recruitment in stroke and NIBS trials justify the need of identifying strategies to overcome predominant barriers. A recent study comparing recruitment in two trials of invasive and non-invasive brain stimulation has identified factors that may affect the enrollment rate of stroke patients, such as inclusion/exclusion criteria [11]. Despite the simplicity and safety of NIBS, the authors of this mentioned study highlighted that NIBS trials had significantly lower enrollment rates in comparison to trials of Deep Brain Stimulation. This consideration may be due to the heterogenous evidence regarding the efficacy of NIBS for stroke rehabilitation [12]. Besides this report, previous studies in stroke and NIBS have not discussed specific nor efficient recruitment strategies in this field, which are essential elements of a clinical trial recruitment plan.

Therefore, using our experience with recruitment in an R21 NIH sponsored clinical trial of stroke and NIBS, in addition to our experience in longitudinal clinical trials in NIBS, we reviewed and analyzed recruitment methods used in our trial as to determine their efficiency. Sharing experiences with recruitment is critical to increase success of future clinical trials.

Methods

Study background

The clinical trial for which the recruitment strategies are discussed in this manuscript was designed to evaluate the effects of transcranial magnetic stimulation (TMS) combined with a Selective Serotonin Reuptake Inhibitor (SSRI) on upper limb motor recovery in post-stroke individuals. This study was designed as a parallel, randomized, double-blind, placebo-controlled, single-center clinical trial. We chose to analyze this particular study due to our large experience in recruiting stroke survivors from June 2015 to December 2017. Inclusion criteria included individuals who had suffered an ischemic cerebrovascular accident (CVA) within the past two years that caused hemiparesis or hemiplegia (Fugl-Meyer = ≥ 11 < 56). Participants were required to be over 18 years old and able to provide informed consent. Methodological details can be found at ClinicalTrials.gov (identifier: NCT02208466). This trial was reviewed and approved by Partners Human Research Committee, the Institutional Review Board of Partners HealthCare.

The study was divided into a screening visit, baseline visit, 10 daily stimulation sessions (from 3rd through 12th visits with the 13th visit including post-intervention assessments), a weekly stimulation session for 8 weeks (14th to 21st visit). In the last visit (22nd), post-intervention assessments were conducted (Fig. 1).

Fig. 1

Study visits overview.

An important point to highlight is that this study could be considered as a complex clinical trial as it involves the administration of a drug and of non-invasive brain stimulation, as well as multiple visits to the research center. Therefore, this trial holds multiple challenges for recruitment, which will be discussed throughout this manuscript.

The recruitment strategies applied in this clinical trial varied along the two-year period, as inefficient approaches were eventually replaced by other strategies that were potentially more effective. Despite the transition between recruitment methods, all of the nine strategies applied in the trial are discussed in this study, and all were IRB approved. Additional information on how each strategy was applied in the trial is presented.

Recruitment strategies

Below we discuss and categorize the recruitment methods used in our clinical trial.

Physical flyers circulation

Flyers were broadly disseminated throughout the entire trial duration, mostly around the Greater Boston area, which has a population of approximately 8,099,575, according to the 2014 US Census estimate. Target places included stroke outpatient clinics, support groups, affiliated hospitals, university bulletin boards, and public outdoor areas. The average distance to each posted location was about 5.5 miles. The estimated amount of time spent on this recruitment strategy was compromised as the research team responsible for this task went to each site using public transportation. The flyers were distributed once per week and the researchers spent approximately an hour per day to visit about 5 different sites. Between 20 and 100 flyers were distributed in each site per visit. Only participants that mention the physical flyer as recruitment strategy were considered in this method.

Referral from physician

Neurologists and Physical Medicine and Rehabilitation (PM&R) physicians from Spaulding Rehabilitation Hospital (SRH) and Massachusetts General Hospital were approached in person or contacted by email. Information regarding the clinical trial was presented and discussed with them about once per month, with an average presentation duration of 2 h. In addition, the physicians were often asked to disseminate the trial details among doctors from other facilities within the Partners Healthcare Network, as to enhance trial awareness and improve recruitment. Most of the time was spent sending emails to a large number of physicians from pre-established hospital directory lists. The task of sending and answering emails required about an hour and a half of dedication per week.

Referral from other health professionals

Physical and occupational therapists are the health professionals that are most in contact with stroke survivors, due to the high intensity and frequency of rehabilitation programs. In this strategy, a research assistant was responsible for constantly sending emails with electronic versions of flyers to these health professionals, as to remind them about trial eligibility criteria and recruitment needs. The emails were sent at least once per week and the therapists were asked to forward the trial information to other departments in their facility. This recruitment strategy required about an hour of the research team's dedication per week, as every aspect or query needed to be well explained and clarified.

Dissemination through outpatient centers

Outpatient centers that admitted brain injury and stroke patients were also a target for trial dissemination. Every month, research staff contacted the site manager or clinical supervisor of the center by phone in order to seek cooperation for trial recruitment. If requested, study electronic information was also sent by email. An estimated 4 h per month were spent in this strategy due to the large amount of potential outpatient centers in the nearby hospital areas.

Targeted campaign in support groups

We contacted support groups for stroke continuously during the first year. The major goal in contacting and visiting these support groups was to raise awareness on stroke rehabilitation techniques, especially NIBS. Support groups were found through local hospitals and the stroke institute at SRH. A presentation was given in support groups by our team members, once per month or twice every two-month. Each visit used to last for about 2 h, although most of the time was spent for driving to the meeting places as some of them were about 2 h away from the hospital facilities.

Newspaper advertising

Advertising the clinical trial in newspapers was a strategy implemented in the initial period of recruitment, from November 2015 to October 2016. The ads were published in local newspapers, such as the Boston Metro, in very specific study periods, which helped us identifying participants recruited from this strategy. These newspapers have a high rate of distribution in the Boston area and the ads were usually posted once per week for 4 weeks in a row. These newspapers claim to reach about 29.3% of people living in Suffolk County, Massachusetts. The advertisement contained basic information about the study and the contact information of our leading research coordinator. The same standardized information was provided for the callers. The main goal of this type of advertisement was to disseminate information about the trial to a broader population. During each week, most of the time was spent by answering calls from interested individuals, scheduling new visits, and updating the ads information, which required an estimated time of 4 h per week.

Medical records review

From the onset of recruitment until the end, we conducted a review of medical records of patients admitted to post-stroke rehabilitation services within the Partners Healthcare Network. Two investigators were responsible for reviewing the medical history of recently admitted patients to our stroke unit. Due to the large amount of patients admitted daily, the review was conducted at least three times per week, with a total time of 4 h spent on each day. During this review, trained research assistants searched for pre-screening inclusion and exclusion characteristics in patients as to identify potentially eligible participants. Cases were then discussed with the attending physician who requested permission for our team to approach the patient. The majority of time spent on this method was to analyze patients’ medical records.

In person approach to brain injury clinics

A research assistant was responsible for attending weekly clinics that admitted patients with different neurological deficits, including stroke. If the clinician suggested the trial for a patient who was interested, our research assistant was called to provide more information. The clinic appointments lasted half a day each and occurred once per week during either the morning or afternoon shifts, requiring an estimated time of up to 5 h per week.

Online advertising

Online advertising through Google AdWords was introduced as a recruitment strategy in April 2017 in order to reach a broader population of potentially eligible post-stroke individuals. With online ads, the target population was not only the patients themselves, but also friends and relatives that may disseminate the trial information among stroke survivors. Our google advertisement actively appeared within the google search window of individuals within a 250 mile radius from our facility that searched on Google for “stroke rehabilitation”, “stroke therapy”, or “stroke research”. If interested, these individuals had access to submit contact information on our institutional webpage. Research staff mostly spent time contacting patients who subscribed, by using emails or phone calls, which demanded an average of 2 h per day of the research team's recruitment efforts. Telephone and email scripts were used to standardize information and promote adequate recruitment.

All interested individuals or patients admitted to the hospital stroke department, regardless of the recruitment strategy, were submitted to pre-screening. This process consisted of filling out a pre-established checklist with some essential information, including the type of stroke suffered, time since stroke and the existence of motor function disability. Pre-screened patients identified as potentially eligible were asked to schedule a consent visit, in which a full screening would be performed, with questionnaires and motor function assessments.

Time effort-benefit analysis

We have developed an index to allow us to establish the time spent benefit of each recruitment strategy. Our index corresponds to a ratio between the number of enrolled individuals and the total number of hours spent on each corresponding recruitment method. For determining the amount of time spent on each strategy, we estimated the number of hours per week required by each method and multiplied it by the number of weeks for which the strategy was applied.

In addition to the index calculation, we estimated the trial overall conversion ratio (number of patients needed to screen as to have one randomized) and the recruitment yield of each strategy, which corresponds to the proportion of randomized participants among pre-screened subjects. We believe that such measures will provide useful information about the strategies’ efficiency.

Results

In our phase II clinical trial, we started recruiting participants in June 2015, although the collection of recruitment data was only initiated in November 2015. For this reason, the results provided here refer to the time period between November 2015 and December 2017. Along the recruitment process, 1247 individuals were pre-screened. Subjects submitted to pre-screening who fulfilled the eligibility criteria were added to the complete screening process. Of these pre-screened individuals, 279 were identified as potentially eligible and the remaining ones were immediately excluded. Among the potentially eligible individuals, 44 agreed to be fully screened and to provide initial consent, which was mandatory before performing the Fugl-Meyer scale evaluation that confirmed the validity of the complete study inclusion criteria. Only 27 out of 44 individuals had eligibility confirmed and could be randomized. The remaining 17 consented subjects were found to be ineligible due to very low motor function (82.3%, n = 14), medication interaction with SSRI (11.7%, n = 2), or loss of interest (5.8% = 1). As a result, the overall yield rate considering all prescreening was 2% and the trial overall conversion ratio was about 10:1, meaning that at least 10 potentially eligible individuals had to be pre-screened to randomize one. A recruitment flow chart is shown in Fig. 2.

Fig. 2

Recruitment flowchart.

Reasons for ineligibility after pre-screening varied considerably among individuals. The most common reasons included an occurrence of stroke within more than two years (49%), a stroke of hemorrhagic etiology (11%), and absence of motor impairment (11%). These three characteristics were defined as exclusion criteria to our clinical trial. Other reasons for ineligibility after pre-screening and their respective proportions are illustrated in Fig. 3.

Fig. 3

Reasons for individuals not being eligible after pre-screening. *Other exclusion conditions refer to aphasia, congestive heart failure, cancer, cognitive impairment, multiple sclerosis, previous craniotomy, and locked-in syndrome.

(color figure).

By making our time effort-benefit analysis, it was evident that a few strategies, such as medical records review and referral from physicians, demanded a larger number of hours per week to recruit at least one individual. On the other hand, strategies such as referral from other health professionals and flyers circulation showed that fewer hours were needed for recruiting a single participant. As the strategies applied in our clinical trial varied along the recruitment period, we present a chart that shows enrollment rate vs. recruitment strategies in different trial stages (Fig. 4).

Fig. 4

Patients recruited between November 2015 and December 2017 related to applied strategies. Orange line represents the cumulative number of patients recruited in the two-year period. Colored areas represent the period in which each strategy was applied. Strategies: A – Physical flyers circulation; B – Referral from physicians; C – Referral from other health professionals; D – Dissemination through outpatient centers; E – Targeted campaign in support groups; F – Newspaper advertising; G – Medical Records Review; H–In person approach to brain injury clinics; I – Online advertising.

(color figure).

Following our index, an apparent negative correlation between the index of a strategy and the number of hours it demanded during the trial recruitment process became clear. The index values varied from 0.2 to 4.2. With online advertising, investigators were able to recruit the largest number of individuals (44%, n = 12) with a total of 280 h spent; therefore, this strategy was that with the largest index (4.2) and consequently the strategy with the best time effort-benefit. The medical records review demanded the largest amount of time throughout the trial (960 h) and did not allow recruitment of many participants (7.4%, n = 2), resulting in the lowest time effort-benefit analysis among all strategies (0.2).

The recruitment yield considering the number of enrolled individuals from the total of potentially eligible ones considerably varied between each strategy. The referral from other health professionals was the method with the highest recruitment yield, 66%, meaning that we were required to only identify 2 potentially eligible subjects as to recruit at least one. The online advertising strategy presented a recruitment yield of 21.4%, although several potential participants contacted through this strategy were not actually eligible after screening or had no interest in participating despite being the ones establishing the initial contact. The targeted campaign in support groups and newspaper advertising had null index and recruitment yield as no individuals were recruited from these strategies. For additional information in this regard, please refer to Table 1.

Table 1

Summary of each strategy's success during the trial recruitment period.

Recruitment Strategy	Individuals randomized	Estimated time spent (hours)	Index	Recruitmen Yield (%)
Physical Flyers Circulation	2	96	2	33
Referral from physicians	1	192	0.5	33
Referral from other health professionals	4	92	4.1	66
Dissemination through Outpatient Centers	2	192	1	29
Targeted Campaign in Support Groups	0	48	0	0
Newspaper Advertising	0	160	0	0
Medical Records Review	2	960	0.2	1.3
In Person Approach to Brain Injury Clinics	4	400	1	22
Online Advertising	12	280	4.2	21.4
Total	27	2420	1.1	9.6

Discussion

In this manuscript, we discuss recruitment strategies and their application in a clinical trial involving stroke subjects and NIBS. The low enrollment yield of our phase II clinical trial, despite the strong efforts dedicated to the recruitment process, possibly reflects the extra challenges faced when testing complex and innovative approaches such as tDCS and TMS, as mentioned by Potter-Baker et al., in 2016 [11]. Recruitment in clinical trials is a dynamic process, which requires constant evaluation and improvement of applied strategies, as well as team integration and feedback from staff [13]. Along the two-year period, we believe that our enrollment rate increased as a consequence of careful planning in recruitment methods and a constant re-evaluation of study strategies and accomplishments. The remarkable progress of our recruitment was certainly influenced by several factors such as effortful training of the research team and strong familiarity with the stroke population, both essential aspects for successful recruitment according to Hadidi et al. [14]. Therefore, we understand that frequent analysis of recruitment performance and continual adjustment of the initial study plan are important ways of preventing unnecessary costs and enhancing recruitment efficiency in stroke trials.

The data obtained with the index calculation developed in this study indicate that online advertising and referrals from other health care professionals are the strategies with higher time effort-benefit analysis. With online advertising, we have recruited the largest number of individuals likely due to the possibility of reaching a broader population and to the easy access of individuals to the ads through the internet. Taylor-Pilliae et al. has also reported a great effectiveness of paid advertisements in a recent study with stroke survivors, which was responsible for 21.4% of enrolled individuals [15]. In contrast, another study with post-stroke patients has discussed that media advertising was not an effective strategy to recruit the target population [12]. In fact, online advertising makes it possible to recruit individuals independently of their attendance to healthcare services, thus enhancing also the external validity of the study. Despite the effectiveness of paid advertisements, we did not explore all types of ads, such as television, magazine or radio, which have shown to significantly increase the number of individuals aware of the trial [15,16]. The application of these other advertisements would add extra costs to recruitment, although they should be considered in view of the success observed in previous trials [10].

An aspect worth highlighting is that to use online advertisement, investigators should prepare the research team to deal with potential subjects from diverse cultures and nationalities, which means they need to acquire cultural competency [17,18]. When research staff are well prepared to contact individuals from different places, the accessible population is broader, admitting higher chances of recruitment. In our trial, this prerequisite was assured as we developed a Spanish version of the protocol and foreign language speakers were part of the research team, allowing the inclusion of individuals from different countries besides the United States of America.

The use of online advertisement also demands research staff training on adequate collection of subjects contact information and management of operational systems. These were essential requirements applied in our trial to ensure appropriate manipulation of internet ads, which has also been mentioned in a recent stroke study [19]. In line with this view, Gomes et al. has reported that clinical trials applying telephone and/or in-person interviews should plan in advance a content-analysis technique, as to be able to extract useful and correct information from interviews [20]. In this mentioned study, it has also been reported that the acquisition of inadequate phone numbers was one of the biggest challenges in their recruitment process, which may be a direct consequence of an inadequate conduction of patient interviews or a poor research staff training in this regard.

The recruitment yield analysis of each method showed that direct patient-clinician encounter is one of the most accurate strategies likely due to the pre-existent two-way trusting relationship between post-stroke patients and their own treating physician. Referral from other healthcare professionals also allowed recruitment of several stroke subjects, which emphasizes the importance of developing partnerships in research, especially between professionals, outpatient centers and/or larger institutions with diverse skills and mutual interests [21,22]. The collaboration of physicians in our trial has improved communication with participants and enhanced their level of confidence on the study protocol. Mistrust and fear of potential subjects in regard to safety are common recruitment barriers reported in the literature, especially in clinical trials that apply NIBS as an intervention [18,23]. As to overcome this obstacle, we believe that increasing community awareness about available rehabilitative therapies for stroke through conversations with healthcare professionals is a useful strategy to motivate stroke survivors to take part in research studies.

The circulation of physical flyers had a significant impact in our recruitment process and was also considered a strategy with high time effort-benefit anlaysis, possibly due to its simple application and clear illustration of the trial. Berge et al. has reported that presenting streamlined and understandable study information to potential participants is a promising method to ensure that recruitment meets the target [24]. In line with this finding, we believe that recruitment flyers are required to be self-explanatory, with sufficient details and accessible illustrations as to arouse readers’ interest and make them comprehend study goals. Especially in trials of stroke, a complex neurological disorder often misunderstood by patients, presenting clear material about NIBS effects on stroke may increase knowledge of the condition and reduce mistrust of participants [21,24,25].

A crucial challenge we had to overcome in our clinical trial was the lack of resources for transportation of participants to our facilities. This is a major recruitment issue mentioned in other stroke trials, especially due to the natural evolution of stroke that causes multiple disabilities on affected individuals [20,26]. A study of Polese et al. has reported that lack of transportation was one of the most frequent reasons for refusals, which accounted for 20% of their potential participants [19]. When investigators choose to apply medical records review as a recruitment strategy, transportation issues can highly interfere in the success of this method, as registered patients often have mobility deprivation and difficulties to reach healthcare services without support. With a goal to overcome this barrier, part of the transportation costs were covered by the trial.

Recruitment is highly time-consuming and study participants often have difficulties to arrange appointments or arrive to study visits on time, mostly due to busy routines or work schedules. Berge et al. has highlighted that several research centers have little time dedicated to participants, usually limited to week days and working hours, which may make study subjects give up on their participation [24]. Considering that most stroke patients have moderate to severe disabilities and are not able to drive, they often depend on relatives or neighbors who cannot provide a lift at any time. As to overcome this issue, we provided a wide flexibility for study appointments to potentially eligible individuals, by expanding schedules to beyond working hours, also on weekend days.

An important factor to consider is the increase of recruitment rate over time. In our case, the last year of the trial (final 1/3 period) recruited about 61.3% of subjects. Several factors may explain this significant change. The most important explanation is the training of the research team and learning the most effective approaches to reach the study population, structure and local health care community. In fact, we did notice this effect when we adapted the initial study eligibility criteria to reach a larger number of patients from our accessible population. Hotter et al. has discussed the importance of modest modifications to strict selection criteria for expanding chances of recruitment and increasing enrollment rates [27]. Another explanation for an increase in our recruitment rate along the trial is an improvement in the research team's communication and feedback from coordinators. This consideration agrees with a report of Blanton et al. which affirmed that presenting bar graphs of the research progress periodically may bring benefits to the recruitment process [28].

Even though this manuscript presents relevant information regarding recruitment strategies in clinical trials, the results should be viewed in light of some limitations. First, these conclusions are based on a specific measure we developed that takes into consideration the hours spent and the number of recruited individuals in each recruitment approach. It is important to highlight that the number of hours presented in Table 1 comes from an estimation performed by the research staff, and therefore is likely to present variation in relation to the true number of hours dedicated for each activity. In addition, the evaluation of these methods may differ if we decide to analyze the cost (money spent) demanded by each strategy. However, as salaries of personnel would be similar in the different strategies, the hours spent is also a surrogate for the financial cost. In this study, we did not include the analysis of expenses with research staff as most of the recruitment tasks were performed by unpaid volunteers, which is not the reality in other trials. In addition, although we have presented information about the time effort-benefit analysis of each recruitment method, it is true that different clinical trials may admit distinct results from each strategy. Accordingly, the success of recruitment in stroke and NIBS trials is directly related to the study population (children, young adults and/or elders), study region (metropolis or rural area; developed or developing countries) and study eligibility criteria.

Conclusions

The results of this manuscript are useful to guide future investigators in organizing the recruitment process of a stroke clinical trial, as well as to diversify the applied methods by using evidence-based information, in order to achieve enrollment goals. A multi-targeted approach seems to be the ideal method to admit large recruitment yields, as well as to achieve the pre-determined sample size goal faster. The combination of multiple methods of recruitment will not by itself ensure a large number of recruited individuals, but a constant evaluation of the results of each technique contributes to improve the process. Finally, investigators need to consider the learning curve, as any project presents with its particular challenges, and thus take this into account when planning recruitment costs and time.

Funding

Grant support: NIH R21 grant (R21HD079048-01A1).