Effectiveness of online education for recruitment to an Alzheimer's disease prevention clinical trial.

INTRODUCTION: Low awareness of Alzheimer's disease (AD) clinical trials is a recruitment barrier. To assess whether online education may affect screening rates for AD prevention clinical trials, we conducted an initial prospective cohort study (n = 10,450) and subsequent randomized study (n = 351) using an online digital tool: AlzU.org.
METHODS: A total of 10,450 participants were enrolled in an initial cohort study and asked to complete a six-lesson course on AlzU.org, as well as a baseline and 6-month follow-up questionnaire. Participants were stratified into three groups based on lesson completion at 6 months: group 1 (zero to one lesson completed), group 2 (two to four lessons), and group 3 (five or more lessons). For the subsequent randomized-controlled trial (RCT), 351 new participants were enrolled in a six-lesson course (n = 180) versus a time-neutral control (n = 171). Screening and enrollment in the Anti-Amyloid Treatment in Asymptomatic AD (A4) clinical trial were reported via the 6-month questionnaire and are the primary outcomes.
RESULTS: Cohort: 3.9% of group 1, 5% of group 2, and 8.4% of group 3 screened for the A4 trial. Significant differences were found among the groups (P < 0.001). Post hoc analyses showed differences in A4 screening rates between groups 1 and 3 (P < 0.001) and groups 2 and 3 (P = 0.0194). There were no differences in enrollment among the three groups. RCT: 2.78% of the intervention group screened for A4 compared to 0% of controls (P = 0.0611). DISCUSSION: Online education via the AlzU.org digital tool may serve as an effective strategy to supplement clinical trial recruitment.

INTRODUCTION

Rapid and cost‐effective recruitment for clinical trials is fraught with challenges. Nearly two billion dollars is spent annually in the United States on patient recruitment, and most clinical trials require timeline extensions due to recruitment difficulties. These significant challenges also exist in the field of Alzheimer's disease (AD), as it is estimated that over 500,000 volunteers need to be screened to enroll 70,000 participants for >150 planned or ongoing clinical trials. Due to ineffective or inefficient recruitment practices, investigators face increased costs and fail to meet study deadlines, which ultimately delays the regulatory approval of new therapeutics. , , In addition, insufficient recruitment results in missed opportunities for patients who may benefit from the interventions administered in these clinical trials. Identifying strategies that improve recruitment would, therefore, benefit trialists, clinical researchers, and patients. Potential barriers to efficient recruitment include a general lack of public awareness about clinical trials , and recruitment methods that have not kept pace with advances in communications and other technologies.3 Therefore, online educational strategies that are free, widely promoted, and available to the general public may facilitate recruitment in a rapid and cost‐effective manner. ,

Alzheimer's Universe (http://www.AlzU.org) is a free online digital tool and educational portal with evidence‐based resources for people at risk for AD; it was created to educate the public about AD clinical trials and risk‐reduction strategies. The conceptual framework aims to educate users about AD in order to influence awareness, beliefs, willingness, and intentions to ultimately drive behavioral change. In a prior randomized study, participation in AlzU.org significantly improved users’ knowledge about AD, as well as several measures of behavioral intent such as willingness to participate in an AD prevention clinical trial. However, it was unclear whether self‐reported measures of increased interest would translate into real‐world behavioral change of screening and enrolling in an AD prevention clinical trial.

In this article, we present two sequential studies investigating the impact of AlzU.org on screening and enrollment into the “Anti‐Amyloid Treatment in Asymptomatic Alzheimer's disease” (A4) AD prevention trial. We first conducted a prospective cohort study to investigate the relationship between AlzU.org lesson completion rates (low, medium, and high “dose”) and A4 screening and enrollment. We then conducted a randomized controlled trial (RCT) with new participants to investigate the influence of AlzU.org on A4 screening and enrollment.

METHODS

Overview of AlzU.org

The AlzU.org online digital educational tool has been described previously. The course tested included six lessons ranging from 6 to 16 minutes in length (64 minutes total): (1) introduction to AD, (2) stages of AD, (3) AD risk factors, (4) AD diagnosis, (5) AD management (prevention and treatment), and (6) overview of evidence on nutrition and exercise for AD. Lessons were made to be highly interactive, including multi‐media videos, interactive webinars, voiceovers, animated graphics, and periodic user assessments. Content was designed using the assertion‐evidence approach, along with best practices for online education.

RESEARCH IN CONTEXT

Systematic review: Authors searched ClinicalTrials.gov and World Health Organization's International Clinical Trial Registry Platform to identify online education intervention studies for recruitment to Alzheimer's disease (AD) prevention clinical trials. Search terms were “online education OR e‐intervention OR digital tool” and “clinical trial recruitment.” No completed studies using an online education tool for clinical trial recruitment were identified.

Interpretation: The use of online public education may be useful for the improvement of clinical trial recruitment strategies, and may be especially useful for AD prevention clinical trial recruitment.

Future Directions: Further study of the effectiveness of online digital tools in large, randomized study cohorts warrants consideration. In addition, this approach may serve as a reproducible model that may be applied to areas of clinical research outside of AD prevention.

All lesson content was created by a multi‐disciplinary team of AD healthcare professionals, including four neurologists (two with a sub‐specialization in AD and two with graduate training in medical education), two instructional designers, and a professional graphic designer. Each lesson was reviewed initially by six laypersons from each of our respective recruitment age groups via focus groups and refined accordingly. Extensive beta‐testing was then conducted, where users watched each lesson and provided evaluative feedback via Likert scale ratings and open‐ended responses using Survey Monkey (surveymonkey.com). Modifications to the lessons based on user feedback were made. Independent peer‐review by a board‐certified neurologist, clinical neuropsychologist, and/or a memory disorders nurse practitioner was obtained.

Study design and participants

From June 15, 2015 to January 31, 2017, participants were recruited via internet marketing, social media, broadcast media, and other outreach initiatives to join AlzU.org and complete a six‐lesson course. Inclusion criteria were willingness to opt‐in to longitudinal study survey emails and age 65 to 85. Exclusion criteria were previous screening and/or enrollment in the A4 clinical trial before joining AlzU.org and a self‐reported diagnosis of AD. Institutional review board approval was obtained, and patient informed consent was obtained online (Protocol #1311014539).

After the initial prospective cohort study (June 15, 2015 to December 23, 2016), two refinements were made to AlzU.org to improve content accessibility and lesson completion rates/user retention: (1) lessons were updated to be compatible with mobile devices and (2) additional email reminders were sent if lesson content remained incomplete.

Between January 2, 2017 and January 31, 2017, participants were randomized (1:1) using a random number generator to either complete the six‐lesson course available on AlzU.org or a time‐neutral control course with general educational content on AD (Figure 1).

FIGURE 1

Consort diagram

Procedures

Upon joining AlzU.org, participants completed a pre‐course (baseline) comprehensive questionnaire covering demographics, as well as behavioral, lifestyle, and risk assessment measures including a validated scale evaluating the stages of change of behavioral intent related to AD prevention. After completing the baseline questionnaire, participants were redirected to the AlzU.org lesson menu. In an effort to guide users through a precise, stepwise, and developmental educational experience, only lessons 1 and 2 (as well as online cognitive assessments) were accessible when users first joined AlzU.org. Subsequent lessons and activities were unlocked following completion of the first two lessons. A three‐question multiple choice quiz was administered before and after each lesson to assess knowledge gains and promote retention of high‐yield content. The course could be completed at the user's convenience, and reminder emails were sent if lessons remained incomplete. Other user resources available included links to AD clinical trial screeners (including the A4 study), two online AD clinical trial registries, advocacy initiatives, four validated web‐based longitudinal cognitive assessments (Cognitive Function Test, Neurotrack, the Self‐Administered Gerocognitive Exam [SAGE] test, the Face Name Associative Memory Test ), and a nutrition and lifestyle tracker. Six months after joining AlzU.org, a follow‐up questionnaire was sent via an automated email, with email reminders at 190 and 200 days if not completed. The follow‐up questionnaire mirrored the baseline assessment and was designed to evaluate the longitudinal effects of AlzU.org course completion by asking specific questions related to screening and enrollment in the A4 clinical trial.

Randomization (randomized controlled trial)

Between January 2, 2017 and January 31, 2017, participants who joined AlzU.org were randomized to one of the two courses available. The intervention group completed a six‐lesson interactive web‐based course accessible via computer, tablet, and personal mobile devices. The control group completed one time‐matched interactive webinar lesson on AD statistics and public policy, and time‐matched video lessons, also accessible via computer, tablet, or personal mobile devices. The control educational content focused on AD information that would not be expected to change perceptions about AD clinical trials or other behaviors (eg, AD caregiving advice and the differences between age‐related cognitive decline and AD).

Outcomes

The primary outcome was rate of screening and enrollment into the A4 clinical trial six months after joining AlzU.org. In the cohort study, screening and enrollment rates were compared across the three lesson groups. In the RCT, screening and enrollment rates were compared between the intervention and control groups in participants who reported AlzU.org as the specific influencing factor for their decision.

Statistical analyses

Prospective cohort analyses

The analyses were pre‐specified to compare three “doses” of lesson exposure: group 1 (zero to one lesson completed; 0 to 7 minutes), group 2 (two to four lessons completed; 17 to 45 minutes), and group 3 (five or more lessons completed; 60 to 64 minutes). Analyses were completed using R version 3.5.1. The primary outcome was analyzed using chi‐square goodness‐of‐fit tests to explore differences in A4 screening/enrollment rates among lesson completion groups. Post hoc analyses were conducted to determine if there were significant differences in screening and enrollment rates between specific lesson completion groups.

Randomized‐controlled trial analyses

All analyses were pre‐specified as intention‐to‐treat (ITT), but utilized a modified ITT population because those who had already screened or enrolled in the A4 study were excluded from analyses. The primary outcome was analyzed using a Fisher's exact test to assess the influence of AlzU.org on A4 screening/enrollment between intervention and control groups.

RESULTS

Cohort study

Participants and recruitment

Between June 15, 2015 and December 23, 2016, a total of 36,267 unique users joined AlzU.org and completed the baseline survey, of which 10,450 met the inclusion criteria. The majority of participants were female (79.7%) and the mean age was 72.56 years (Table 1). A majority of participants received post‐secondary education, including an associate's degree (16.49%), bachelor's (24.08%), or post‐graduate degree (25.06%). Most users were Caucasian (80.89%). No notable differences were observed between participants who completed the follow‐up survey and those who were lost to follow‐up (Table 1).

TABLE 1

Cohort baseline characteristics

Variable	Subcategory	N = 10,450 completed baseline survey (% of total)	N = 2469 completed both baseline and post‐survey (% of total)
Age	Mean	72.56	70.49
Gender	Female	8329 (79.7%)	2046 (82.86%)
	Male	1467 (14.0%)	359 (14.54%)
	Prefer not to say	54 (0.51%)	5 (0.20%)
	No response	600 (5.74%)	59 (2.38%)
Education level	Elementary/primary	53 (0.51%)	6 (0.24%)
	High school/ secondary	2687 (25.7%)	500 (20.25%)
	Associates	1724 (16.49%)	370 (14.99%)
	Bachelors	2516 (24.08%)	670 (27.13%)
	Post‐graduate	2619 (25.06%)	825 (33.41%)
	Prefer not to say	216 (2.07%)	33 (1.34%)
	No response	635 (6.08%)	75 (3.04%)
Ethnicity (multi‐response item)	White/Caucasian	8453 (80.89%)	2169 (87.85%)
	Hispanic/Latino	494 (4.72%)	100 (4.05%)
	Black/African American	349 (3.34%)	41 (1.66%)
	Asian/Pacific Islander	234 (2.24%)	44 (1.78%)
	Middle Eastern/Arab American	35 (0.33%)	5 (0.20%)
	Native America/American Indian	95 (0.91%)	18 (0.73%)
	Other/mixed	172 (1.65%)	32 (1.30%)
	Prefer not to say	140 (1.34%)	17 (0.69%)
	No response	478 (4.57%)	63 (2.55%)
Referral source	Facebook	2977 (28.49%)	631 (25.56%)
	endALZnow.org	892 (8.53%)	438 (17.74%)
	NBC Nightly News	51 (0.49%)	13 (0.53%)
	CNN/CNN.com	397 (3.80%)	77 (3.12%)
	CBS This Morning	63 (0.60%)	13 (0.53%)
	NBC Today Show	105 (1.01%)	20 (0.81%)
	Dr. Oz	1763 (16.9%)	196 (7.94%)
	Maria Shriver	183 (1.75%)	29 (1.17%)
	The Alzheimer's Prevention & Treatment Diet book	273 (2.61%)	59 (2.39%)
	YouTube	22 (0.21%)	7 (0.28%)
	Newspaper	87 (0.83%)	36 (1.46%)
	Radio	19 (0.18%)	1 (0.04%)
	Internet site	1565 (14.97%)	511 (20.70%)
	Medscape.com	100 (0.96%)	33 (1.6%)
	Other	640 (6.12%)	335 (13.57%)
	No response	1071 (10.2%)	70 (2.84%)

Referral sources

A total of 9379 responses were received from participants (89.75%) regarding referral source to AlzU.org. The largest referral source (n = 2977, 28.49%) was Facebook.com, followed by the Dr. Oz Show (n = 1763, 16.9%), internet site (n = 1565, 14.97%), EndAlzNow.org (n = 892 or 8.53%), and CNN.com (n = 397, 3.80%). The remaining participants were referred through a large variety of other sources (Table 1).

Lesson completion rates

A total of 10,224 participants were included in lesson completion analyses at 6 months (n = 226 were excluded due to e‐mail bounce back deactivations): 7085 (69.3%) were in group 1 (completed zero to one lesson), 1912 (18.7%) were in group 2 (two to four lessons), and 1227 (12.0%) were in group 3 (five or more lessons). The referral source that led to the highest completion rate was EndAlzNow.org, with 27.7% completing five or more lessons and 27.5% completing two to four lessons.

Screening and enrollment rates

A total of 2469 participants (23.6%) completed the end‐of‐study questionnaire assessing the impact of AlzU.org on screening for the A4 clinical trial (“screen rate”): 1153 were in group 1 (46.7%), 623 in group 2 (25.2%), and 693 were in group 3 (28.1%). Forty‐five participants in group 1 (screen rate = 3.9%), 31 in group 2 (screen rate = 5.0%), and 58 in group 3 (screen rate = 8.4%) indicated that they had screened for the A4 study (Figure 2). A chi‐square goodness‐of‐fit test found differences in screening rates among the groups (P = 0.0002). Post hoc analyses showed differences in A4 screening rates between groups 1 and group 3 (P < 0.0001), and between groups 2 and 3 (P = 0.0194), with group 3 having higher screening rates in both cases.

FIGURE 2

A4 trial screening and enrollment rates by AlzU.org lesson completion group. Note: There were significant differences in screening between groups 1 and 3 (P < 0.001) and between groups 2 and 3 (P = 0.0194). There were no significant differences in enrollment between groups

Of the 134 participants who reported screening for the A4 clinical trial, 33 reported that they ultimately enrolled: 15 from group 1 (enrollment rate = 1.3%), 7 from group 2 (enrollment rate = 1.1%), and 11 from group 3 (enrollment rate = 1.6%) (Figure 2). A chi‐square goodness‐of‐fit test determined that there were no significant differences in enrollment among the groups (P = 0.2335).

Randomized‐controlled trial

Subjects and recruitment

Between January 2, 2017 and January 31, 2017, a total of 1564 users joined AlzU.org and completed the baseline survey, of which 351 met the inclusion criteria. Of these 351 participants, 171 were randomized to the control group and 180 to the intervention group (Figure 3). The mean age of all participants was 67.96 years. The majority of participants were female (77.8%) and Caucasian (86.61%). No differences were observed between the intervention and control groups (Table 2).

FIGURE 3

A4 trial screening and enrollment rates in intervention and control group. Note: We observed a trend in screening between the intervention and control groups (P = 0.061). There was no significant difference in enrollment between the two groups

TABLE 2

RCT baseline characteristics

Variable	Subcategory	N = 171 control (% of total)	N = 180 intervention (% of total)	N = 351 total (% of total)
Age	Mean	67.41	68.49	67.96
Gender	Female	136 (79.53%)	137 (76.11%)	273 (77.78%)
	Male	26 (15.20%)	38 (21.11%)	64 (18.23%)
	Prefer not to say	1 (0.58%)	0 (0%)	1 (0.28%)
	No response	8 (4.68%)	5 (2.78%)	13 (3.70%)
Education level	Elementary/primary	1 (0.58%)	0 (0%)	1 (0.28%)
	High school/secondary	39 (22.81%)	29 (16.11%)	68 (19.37%)
	Associates	22 (12.87%)	24 (13.33%)	46 (13.10%)
	Bachelors	40 (23.40%)	47 (26.11%)	87 (24.79%)
	Post‐graduate	58 (33.92%)	71 (39.44%)	129 (36.75%)
	Prefer not to say	0 (0%)	0 (0%)	0 (0%)
	No response	2 (1.17%)	2 (1.11%)	4 (1.14%)
Ethnicity (multi‐response item)	White/Caucasian	143 (83.63%)	161 (89.44%)	304 (86.61%)
	Hispanic/Latino	9 (5.26%)	7 (3.89%)	16 (4.56%)
	Black/African American	5 (2.92%)	4 (2.22%)	9 (2.56%)
	Asian/Pacific Islander	5 (2.92%)	4 (2.22%)	9 (2.56%)
	Middle Eastern/Arab American	3 (1.75%)	2 (1.11%)	5 (1.42%)
	Native America/American Indian	1 (0.58%)	1 (0.56%)	2 (0.57%)
	Other/Mixed	3 (1.75%)	2 (1.11%)	5 (1.42%)
	Prefer not to say	1 (0.58%)	1 (0.56%)	2 (0.57%)
	No response	8 (4.68%)	4 (2.22%)	12 (3.42%)
Referral sources	CBS This Morning	1 (0.58%)	1 (0.56%)	2 (0.57%)
	CNN/CNN.com	0 (0%)	2 (1.11%)	2 (0.57%)
	Internet site	21 (12.28%)	30 (16.67%)	51 (14.53%)
	endALZnow.org	25 (14.62%)	33 (18.33%)	58 (16.52%)
	Facebook	38 (22.22%)	39 (21.67%)	77 (21.94%)
	Maria Shriver	9 (5.26%)	5 (2.78%)	14 (3.99%)
	Medscape.com	0 (0%)	4 (2.22%)	4 (1.14%)
	NBC Nightly News	3 (1.75%)	0 (0%)	3 (0.85%)
	Newspaper	0 (0%)	2 (1.11%)	2 (0.57%)
	Other TV program	10 (5.85%)	11 (6.11%)	21 (5.98%)
	Radio	1 (0.58%)	1 (0.37%)	2 (0.57%)
	The Alzheimer's Prevention & Treatment Diet book	17 (9.94%)	15 (0.56%)	32 (9.12%)
	YouTube	2 (1.17%)	2 (1.11%)	4 (1.14%)
	No response	44 (25.73%)	35 (19.44%)	79 (22.51%)

A total of 272 (77.5%) of the 351 participants reported their referral source (Table 2). Facebook.com (n = 77, 21.9%) and endALZnow.org (n = 58, 16.5%) were the most commonly reported referral sources, and this was the case in both groups.

Within the control group, 66 participants completed zero lessons (38.5%), while 105 completed one lesson in addition to varying amounts (0 to 57 minutes) of matched duration video content. Within the intervention group, 58 participants completed zero lessons (32.2%), 11 completed one lesson (6.1%), 38 completed two to four lessons (21.1%), and 73 completed five or more lessons (40.6%).

Analyses were performed using a modified ITT analysis. Five participants in the intervention group screened for the A4 clinical trial, compared to none in the control group. Two participants in the intervention group enrolled in the A4 clinical trial, compared to none in the control group. Assumptions were not met for chi‐square goodness‐of‐fit, so Fisher's exact test was used to test the hypothesis that screening rates were equal between the experimental groups. There was insufficient evidence to reject this hypothesis (P = 0.0611). As there were no subjects from the control group that screened, no hypothesis test was run on enrollment rates (Figure 3).

DISCUSSION

This study demonstrates that AlzU.org, an online digital tool for AD education, may be a worthwhile strategy for improving screening rates for AD prevention clinical trials. The prospective cohort study demonstrated an association between lesson completion rate and A4 trial recruitment, with participants in the highest lesson completion group demonstrating the greatest A4 screening rates. However, no significant differences in enrollment rates were found between the lesson completion groups.

After completion of the cohort study, all online educational content was modified to improve content accessibility and lesson completion rates/user retention by updating lesson content to be compatible with mobile devices and adding additional email reminders if available lessons remained incomplete. The subsequent RCT was planned to similarly enroll 10,000 participants; however, due to the earlier than expected closure of recruitment to the A4 trial, participants could only be enrolled for 29 days. Although there were no significant differences in enrollment rates in the RCT, we did observe a trend in screening rates between intervention and control groups (P = 0.0611). One potential reason for the lack of significance was the relatively small sample size (n = 351) when compared against the cohort study (n = 10,450). Further study is warranted in larger populations to draw more definitive conclusions.

AlzU.org is a free online resource that has accumulated a source of potential trial‐ready participants who are intrinsically motivated to spend time completing online education about AD prevention and clinical trials. Since 2014, over 1.5 million unique visitors have used AlzU.org and engaged with educational content. AlzU.org holds promise as it is a free, web‐ and mobile‐compatible online resource with up‐to‐date information for the public. By providing information across all devices, and by using a broad and diverse outreach strategies, participants were easily recruited in a cost‐effective manner. The ubiquitous presence of social media likely also contributed to the effective recruitment to AlzU.org, as a large portion of the participants (≈50%) were recruited from online and social media (with the greatest number recruited from Facebook). Our previous studies have similarly found that social media advertising via Facebook led to rapid and cost‐effective study recruitment, and as such was an effective means of disseminating awareness of clinical trials. Although another prior randomized study demonstrated that AlzU.org can successfully provide knowledge about AD risk reduction and increase subjective willingness to participate in a clinical trial, this study demonstrates the potential of the AlzU.org digital tool to objectively increase screening rates in AD clinical trials in the real‐world.

This study has several limitations. One notable limitation is that all participants had access to some educational materials posted on the website, including a Clinical Trials information page and blog. These resources were available for all participants regardless of lesson completion, and RCT participants randomized to the control group may have also engaged with this information, potentially biasing the results. Future studies may yield more valid conclusions if better able to control for similar types of external variables.

Another limitation of this study is the use of self‐reported, online questionnaires as a means of reporting whether screening or enrollment in the A4 trial occurred. Because there was no verification of screening and/or enrollment, these results are subject to response bias. Furthermore, although efforts were made to increase response rate among all participants (ie, periodic email reminders, user experience consultants, user experience testing, in‐person focus groups, optimized questionnaire layouts, and web‐ and mobile‐accessible questionnaires), selection bias may be present due to low completion rates from intrinsically less‐interested participants. This may be represented by the higher response rates in questions at baseline in participants who also completed the post‐survey. Therefore, future studies addressing these limitations are needed.

Other limitations include the lack of diversity within the participant pool, as most participants (in both the cohort study and RCT) reported as female and Caucasian. Given that trial recruitment from underrepresented populations has been a well‐documented challenge, this is an important limitation that warrants further investigation. Future studies are necessary to evaluate the effectiveness of varied recruitment strategies, including online education, in underrepresented populations.

Despite the limitations presented, this study lends support for further study of the use of online educational interventions to promote screening and enrollment for AD clinical trials. By taking advantage of recent advances in communication practices and technology as a whole, we believe that online interventions can facilitate the recruitment process. Investigating the possible barriers that exist between the public and their willingness to participate in AD education, screening, clinical trials, and risk reduction is essential for the advancement of therapeutic interventions in the field of AD. It is also imperative to consider the effect of recruitment efforts on participant engagement and retention in clinical trials. Therefore, in addition to studying recruitment through AlzU.org, future studies evaluating participant engagement and retention in clinical trials are warranted.

CONCLUSION

Digital tools such as AlzU.org may present a promising framework to promote screening for AD prevention clinical trials. Furthermore, this approach may serve as a potentially reproducible model that may be applied to other areas of clinical research outside of AD prevention. Further study of online educational interventions for clinical trial recruitment is warranted to build upon these findings.