A pilot randomized trial of five financial incentive strategies to increase study enrollment and retention rates.

BACKGROUND: Enrollment and retention difficulties remain major barriers to conducting clinical trials. Financial incentives may promote clinical trial enrollment, however delivery methods to maximize enrollment, maximize retention, and minimize cost remains uncertain.
METHODS: We conducted a single-blind, web-based randomized controlled trial of five financial incentive strategies on enrollment and retention rates in a longitudinal study of advance directives among community-dwelling older adults. Participants were eligible to receive a fixed total financial incentive, but the disbursement amounts at each study timepoint (baseline, 2-weeks, 4-weeks, and 6-weeks) differed between study arms. At each timepoint, participants completed a different advance directive. We conducted an intention-to-treat analysis for the primary and secondary outcomes of enrollment and retention.
RESULTS: 1803 adults were randomized to one of five incentive strategies: constant n = 361; increasing n = 357; U-shaped n = 361; surprise n = 360; self-select n = 364. Overall, 989 (54.9%) participants elected to enroll in the advance directive study. There were no differences in enrollment rates between the control (constant 53.5%) and any of the four intervention study arms (increasing 54.3%, p = 0.81; U-shaped 57.3%, p = 0.30; surprise 56.9%, p = 0.35; and self-select 52.2%, p = 0.73). There were no differences in retention rates between the control (constant 2.1%) and any of the four intervention study arms (increasing 5.2%, p = 0.09; U-shaped 3.9%, p = 0.23; surprise 2.4%, p = 0.54; self-select 2.1%, p = 0.63).
CONCLUSIONS: Financial incentive programs for trial enrollment informed by behavioral economic insights were no more effective than a constant-payment approach in this web-based pilot study.

Introduction

Participant recruitment and retention have been referred to as “the most difficult and challenging aspect of clinical trials.” [1] Indeed, over 40% of National Cancer Institute-sponsored clinical trials were never completed, typically due to difficulties with trial enrollment [[2], [3], [4], [5]]. Even when investigators successfully reach their target enrollment, they rarely do so on schedule [5,6]. Moreover, slow recruitment leads to costlier studies [7], and under-enrollment may result in studies that are underpowered or aborted early resulting in wasted financial resources [8] and raising ethical concerns for exposing patients to risks with reduced likelihood of benefit [9]. These issues extend beyond enrollment as low retention rates result in data missingness that risks both a decrement in power and introduces selection bias despite randomization [10].

Few interventions have been shown to address patient-reported barriers to trial participation, including uncertainty associated with randomization, potential side effects of experimental drugs, time demands, and costs [11]. Such barriers result in additional resources expended for recruitment and may compromise scientific rigor [[12], [13], [14]]. Evidence for financial incentives to promote participation in clinical trials is promising and most participants consider the practice acceptable [[15], [16], [17], [18]], but constrained research funds and ethical concerns about potential inducement limit their use.[19,20] Insights from behavioral economics may offer innovative approaches to deliver financial incentives for trial enrollment that improve their effectiveness without increasing costs [21].

For this pilot study we sought to evaluate the impact of five innovative financial incentive strategies on clinical trial enrollment and retention rates. We conducted a web-based randomized clinical trial (RCT) to test the hypotheses that a U-shaped payment strategy across the four data collection timepoints would maximize enrollment rate, while an increasing payment strategy would maximize retention, and both would be more effective than a constant payment strategy. We further hypothesized that allowing participants to control their own payment strategy or be attracted by the element of surprise would improve enrollment rates when compared to a constant incentive strategy. The goal of this study was to identify promising incentive strategies to be tested in a real-world clinical trial.

Methods

Setting, design and participants. We conducted a five-arm, single-blind RCT comparing five financial incentive strategies aimed at increasing clinical trial enrollment rates in a web-based, longitudinal study of healthcare advance directives. We used the web-based research survey platform Qualtrics, (Provo, Utah, USA) to recruit community-dwelling adults >50 years of age from across the United States. An older population was preferred since completion of advance directives is considered most relevant for older adults or those with a chronic life-limiting illness [22]. Potential study participants were identified by their active member status in a Qualtrics panel and invited via email to participate in a “health-related research study.” The invitation email did not include payment information. By clicking on the embedded study web link participants were automatically enrolled in the financial incentives trial and taken to the Qualtrics research site to learn more about the advance directives study. The electronic informed consent document included standard institutional internal review board-approved language about the study objective, benefits, and risks of participation to most closely represent real-world conditions. Participants were not prospectively informed about the financial incentive trial and were unblinded to their assigned payment strategy at the time of consent for participation in the advance directive study. This approach enabled an unbiased evaluation of participants’ true trial enrollment decisions based on financial incentives rather than relying on hypothetical decision-making, which may not reliably reflect real life decisions. Reminder emails were sent at each study timepoint to participants who enrolled in the advance directive study.

Intervention. Participants were randomized with 1:1 allocation to receive one of five payment strategies: constant, increasing, U-shaped, surprise, or self-select (Table 1). The study groups differed by disbursement amounts at each study timepoint, but the total potential $1.00 incentive was equal across study arms. This amount was chosen because it was consistent with the standard compensation for surveys of a similar length conducted among the targeted Qualtrics panel, and pilot testing showed a ceiling effect (98% enrollment rate) with higher incentives that more closely resembled amounts used in typical biomedical survey research. For each group, payment at each timepoint was contingent upon completion of the survey, and electronic payments were made immediately thereafter. Participants were considered “lost to follow-up” after failure to complete one study timepoint.

Table 1

Financial incentive strategies.

Financial Incentive	Baseline	Week 2	Week 4	Week 6	Total Study Payments	Behavioral Rationale
Constant	$0.25	$0.25	$0.25	$0.25	$1.00	Control condition
Increasing	$0.10	$0.20	$0.30	$0.40	$1.00	Sunk cost bias and loss aversion [23,24]
U-shaped	$0.40	$0.10	$0.10	$0.40	$1.00	Larger incentives more effective than smaller incentives [25]
Surprisea	($0.20)	($0.10)	($0.30)	($0.40)	$1.00	Lottery-based incentive [26]
Self-select	participants chose preferred incentive arm				$1.00	Giving control enables intrinsic motivation [27]

Participants in the surprise arm were informed of the total potential incentive amount, but not the disbursement amounts at each timepoint.

The constant payment strategy served as a control group. We hypothesized that an increasing payment strategy would lead to similar enrollment rates but higher retention rates, while a U-shaped payment strategy would maximize enrollment but might risk lower retention due to the natural tendency for people to discount rewards too far in the future [28], such as the higher last payment. Payments in the surprise arm were blindly allocated in random order in an attempt to leverage the element of surprise in augmenting enrollment. Similarly, participants in the self-select arm were asked to choose their preferred payment strategy among the four options in an effort to offer a sense of control.

Outcomes and data collection. The primary outcome of enrollment rate was defined as the proportion of eligible individuals who provided informed consent to participate in the advance directive study after effectively being “approached” for recruitment upon clicking on the study web link in the email invitation, which did not contain any study specific information in order to minimize potential bias. The secondary outcome was retention rate, defined as the proportion of enrolled participants who completed the entire study. Other subject data collected included sociodemographics, self-reported past medical history, prior experience with the intensive care unit, and palliative care.

Statistical analysis. We assumed a 60% enrollment rate in the constant payment arm. A total sample size of 1800 participants would achieve >80% power to detect an absolute 10% difference in the proportion of patients who enroll between the control arm and each intervention arm at an of 0.05 based on a chi-square test [29]. Summary data are reported as N(%) unless otherwise indicated. We performed bivariate analyses of subject characteristics by enrollment status using chi-square or t-test for categorical and continuous variables, respectively. Due to data collection limitations on the Qualtrics platform among participants who declined to participate in the advance directive study, we were unable to assess differences in subject characteristics across study arms. We performed unadjusted intention-to-treat analyses of enrollment and retention rates between the control and each intervention arm using a chi-square test or fisher's exact test as appropriate. Given the exploratory nature of this study we did not adjust for multiple comparisons [[30], [31], [32], [33]]. All analyses were performed using Stata v14.2 (StataCorp, College Station, Tx) and R 3.5.0. The University of Pennsylvania Institutional Review Board deemed this minimal risk research eligible for review exemption.

Results

We randomized 1837 Qualtrics panel members to receive a financial incentive strategy (constant n = 367; increasing n = 368; U-shaped n = 367; surprise n = 368; self-select n = 367) for participation in the advance directive study (Fig. 1). Post-randomization subject exclusions included 34 for self-reported age <50 years and 3 for completely missing data, for a total of 1800 participants included in analyses.

Fig. 1

Assessment for eligibility and randomization. a3 participants were dropped from the intention-to-treat analysis for completely missing data.

Participants were predominantly Caucasian (84.8%), female (63.5%), and had a mean age of 60.8 years (standard deviation 7.8). Most participants assigned to the self-select incentive strategy chose a constant payment (66.8%), with fewer opting for the U-shaped (15.3%), surprise (16.3%) or increasing (1.6%) payments. Overall, 989 (54.9%) participants enrolled in the advance directive study. Participants who enrolled tended to be younger and female gender (Table 2). Among enrolled participants, there was an imbalance of co-morbid lung disease(p = 0.03); other characteristics were similar across the five study arms (Table 3).

Table 2

Participants characteristics (intention-to-treat populationa).

Characteristic	Enrolled (n = 989)	Declined (n = 811)
Age, mean (±SD)b	60.0 (±7.5)	61.8 (±8.0)
Gender	n (%)	n (%)
Femaleb	654 (66.1)	487 (60.3)
Race/Ethnicity
White	831 (84.1)	694 (85.6)
Black	58 (5.9)	44 (5.4)
Hispanic	48 (4.9)	36 (4.4)
Other	51 (5.2)	37 (4.6)
Employment
Not employed	510 (54.4)	404 (53.4)
Gross household income
≤$39,999	342 (36.2)	267 (35.7)
$40,000–74,999	309 (32.7)	213 (28.5)
≥$75,000	294 (31.1)	268 (35.8)
Marital status
Married	519 (52.5)	462 (57.0)
Education completed
College or beyond	446 (45.7)	368 (47.7)

The intention-to-treat population includes all participants who underwent randomization to the five study groups and met age eligibility criteria and had any available data.

P < 0.01; all other characteristics showed nonsignificant differences with P > 0.05.

Table 3

Participant characteristics by financial incentive arm among those who enrolled (N = 989).

Characteristic	Constant (n = 193)	Increasing (n = 194)	U-shaped (n = 207)	Surprise (n = 205)	Self-Select (n = 190)
Age, mean (±SD)	59.6 (±7.5)	61.1 (±7.6)	59.9 (±7.5)	59.9 (±7.6)	59.7 (±7.3)
Gender	n (%)	n (%)	n (%)	n (%)	n (%)
Female	134 (69.4)	128 (66.0)	130 (62.8)	135 (65.9)	127 (66.8)
Race/Ethnicity
White	165 (85.5)	159 (82.4)	172 (83.1)	170 (82.9)	165 (86.8)
Black	8 (4.1)	17 (8.8)	13 (6.3)	12 (5.9)	8 (4.2)
Hispanic	11 (5.7)	9 (4.7)	10 (4.8)	9 (4.4)	9 (4.7)
Other	9 (4.7)	8 (4.1)	12 (5.8)	14 (6.8)	8 (4.2)
Employment
Employed	85 (46.2)	79 (43.4)	89 (44.7)	88 (45.1)	87 (48.9)
Gross household income
≤$39,999	65 (35.1)	67 (35.8)	80 (40.4)	67 (33.7)	63 (35.8)
$40,000–74,999	53 (28.6)	67 (35.8)	64 (32.3)	64 (32.2)	61 (34.7)
≥$75,000	67 (36.2)	53 (28.3)	54 (27.3)	68 (34.2)	52 (29.5)
Marital status
Married	104 (53.9)	100 (51.5)	109 (52.7)	104 (50.7)	102 (53.7)
Education completed
College or beyond	87 (45.5)	81 (42.4)	91 (44.4)	108 (53.7)	79 (42.0)
Comorbidities
Cancer	15 (7.8)	13 (6.7)	8 (3.9)	15 (7.3)	13 (6.8)
Cardiac disease	14 (7.3)	20 (10.3)	15 (7.2)	12 (5.9)	13 (6.8)
Kidney disease	6 (3.1)	9 (4.6)	6 (2.9)	6 (2.9)	5 (2.6)
Liver disease	4 (2.1)	3 (1.5)	8 (3.9)	3 (1.5)	1 (0.5)
Lung disease	5 (2.6)	10 (5.2)	19 (9.2)	8 (3.9)	8 (4.2)
Mental health disorder	15 (7.8)	13 (6.7)	18 (8.7)	13 (6.3)	18 (9.5)
Neurologic disease	6 (3.1)	12 (6.2)	9 (4.3)	6 (2.9)	12 (6.3)
Serious illness experience
ICU patient	7 (3.6)	18 (9.3)	17 (8.2)	15 (7.3)	14 (7.4)
ICU visitor	110 (57.0)	119 (61.3)	117 (56.5)	118 (57.8)	113 (59.5)
Death of loved one	139 (72.0)	154 (79.4)	144 (69.6)	143 (69.8)	142 (74.7)

In intention-to-treat analyses, there were no differences in enrollment rates between the control (constant 53.5%) and any of the four intervention study arms (increasing 54.3%, p = 0.81; U-shaped 57.3%, p = 0.30; surprise 56.9%, p = 0.35; and self-select 52.2%, p = 0.73) (Fig. 2). There were also no differences in retention rates between the control (constant 2.1%) and any of the four intervention study arms (increasing 5.2%, p = 0.09; U-shaped 3.9%, p = 0.23; surprise 2.4%, p = 0.54; self-select 2.1%, p = 0.63) (Fig. 2). The median time to complete each advanced directive was 4 min and 49 s (IQR: 2 min 50 s–7 min 9 s).

Fig. 2

Enrollment and retention rates by financial incentive strategy.

Discussion

Clinical trials represent the highest standard for evidence-based medicine, yet slow and under-enrollment continue to pose threats to the cost and scientific rigor of such critical research efforts. In this large randomized pilot study, we found that none of the four financial incentive strategies informed by principles of behavioral economics increased enrollment or retention rates in this clinical research study when compared to a constant incentive strategy. The enrollment rate in the constant (control) payment arm was moderate and consistent with our expectations. However, we were surprised to find no improvement in enrollment rates with the U-shaped, surprise or self-select strategies.

There are several potential explanations for the study's null findings. First, and perhaps most relevant for future research, financial compensation for time and effort is just one of several considerations for individuals deciding whether to enroll in a clinical trial. Thus, different approaches to offering actuarially equivalent incentives may be immaterial if barriers such as perceived study risks or distrust of biomedical research are highly operative [34]. Second, the enrollment rate being similar regardless of initial payment amount may highlight a ceiling or floor effect of enrollment in research studies implemented on this web-based platform. Although the direction of the effect cannot be determined from these data, we attempted to avoid both by scaling the payments to the historical compensation plan used among this Qualtrics member panel. However, beyond the median completion time of 4 min and 49 s, it is unknown how the cognitive effort for completing different versions of advance directives compared to other studies offered to this panel. We purposefully chose a high cognitive effort task to mimic real-world clinical research and ensure future applicability of effective interventions. The overall payments in our study may have been perceived by eligible participants as relatively too low, thereby negating the intended effects of different payment paradigms. Third, although the total payment was scaled appropriately for this panel, it is possible that the absolute difference between individual disbursement amounts at each timepoint was not sufficiently large to influence decision-making. Finally, just as people respond differently to health-related behavioral interventions [35], it is possible that certain types of individuals were more responsive to their assigned financial incentive strategy than others [36,37]. For example, finding that younger and female participants were more likely to enroll overall merits further investigation.

Another interesting finding was the preferred option of a constant incentive among individuals assigned to select their payment strategy. Traditional economic theory might have predicted that a higher incentive early on, as with the U-shaped approach, would promote clinical trial enrollment [25]. The failure to detect such an effect may again be attributable to an overall ceiling or floor effect, but could also be attributed to ordering effects since the constant incentive option was listed first [38].

The third finding was that contrary to our expectation retention rates in this web-based study were not superior for the increasing or any other interventional payment strategy compared with control. However, the overall very low retention rate precludes reasonable inferences from such data. We suspect this result was primarily related to a loss of interest or unanticipated competing tasks. Given the increasing focus on long-term outcomes after serious illness or hospitalization, poor participant retention in longitudinal research may be as problematic as slow or under-enrollment. While it is likely that some decision-making processes overlap between enrollment and follow-up, there are additional barriers to consider when designing behavioral interventions to improve study retention, including competing tasks, time constraints, loss of interest, or decline in health status over time. Using non-financial behavioral economic interventions at follow-up timepoints, such as appealing to one's altruistic motivations or moral duty, using social norms, or providing peer-based comparisons [[39], [40], [41], [42], [43]], may be more effective at improving retention than financial incentives alone.

There are several important limitations of this study. First, the use of a web-based platform may limit generalizability to real world clinical trials. Although the optimal scenario would have been to test different financial incentive strategies for enrollment in an actual clinical trial, few real-world trials provide the sample size necessary to test several promising approaches at one time. Second, the responses of community-dwelling adults may not predict how patients would respond. We limited the eligible age criteria to >50 years in an effort to identify people more likely to have the intrinsic motivation to complete an advance directive, yet the rates of chronic disease in this cohort were low. Third, the financial incentives used in this study were lower than is typical for most biomedical research studies and may have been too low to motivate behavior change. As mentioned above, the payments were chosen intentionally to avoid a ceiling effect on enrollment by relatively over-incentivizing this Qualtrics panel and all arms were eligible to receive the same total amount. Finally, we were unable to evaluate heterogeneity of treatment effects or participant characteristics independently associated with enrollment due to our inability to access stratified sociodemographic data among those who declined to enroll.

Conclusions

Financial incentives may be an important tool in improving participation in clinical trials, but the most cost-effective way to offer incentives remains unknown. In this pilot web-based RCT, we found that applying insights from behavioral economics to modify the schedule of financial incentives for participation in a clinical trial did not increase enrollment or retention over time. Future research should focus on understanding which types of patients are less likely to enroll or follow-up, and why, in order to develop novel behavioral interventions that can be targeted to address such barriers.

Funding

The project described was supported in part by Grant Number UL1TR001878 from the National Center for Advancing Translational Science through the Institute for Translational Medicine and Therapeutics (ITMAT) at the University of Pennsylvania (KRC). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Science or the National Institutes of Health.

Author's contributions

KRC, KNY, EC, SB, and SDH contributed to the study design and protocol. DCK, KRC and SDH drafted the manuscript. KNY, SB, and EC provided critical feedback and revisions. All authors have provided final approval of the study protocol.