Recruitment strategies for a pragmatic cluster randomized oral health trial in pediatric primary care settings.

OBJECTIVES: To describe multilevel recruitment strategies for an ongoing clinical trial in pediatric primary care settings, and assess adoption and reach of these strategies via the RE-AIM framework.
METHODS: This study is part of a larger pragmatic cluster randomized clinical trial focused on the effectiveness of interventions on the practice, provider, and caregiver levels on dental utilization for Medicaid-enrolled 3-6 year old children. Pediatric practices were recruited according to the proportion of Medicaid-eligible children, geographic region, and County. In accordance with the RE-AIM framework, providers reached were those approached directly and consented, and those who participated in the intervention training adopted to deliver the intervention. Caregivers reached were those approached and consented at their child's well-child visit to participate in the trial.
RESULTS: Recruitment goals were met over a 21 month period, with an overall enrollment of 18 practices, 62 providers, and 1024 caregivers-child dyads. The majority of practices enrolled were small, suburban, and located in an urban county. The participation rates among approached providers and caregivers was 93% and 84% respectively. Enablers for recruitment was the one-on-one interaction with the provider and caregivers. Barriers to recruitment for caregivers included no-shows and cancellations at well-child visits. Adoption of intervention among providers was high, and caregiver reached were representative of the eligible target population.
CONCLUSIONS: Active approaches to recruitment, such as utilizing opinion leaders, in-person recruitment, and building relationships with practice staff, can result in successful enrollment and imp lementation of a multi-level intervention in pediatric primary care settings.

Introduction

Dental caries (tooth decay, cavities), the most common chronic childhood disease, affects 21.4% of children aged 2–5 years in the U.S., with a disproportionate number of minority and low income children affected [1,2]. Among this age group of children, low preventive dental utilization rates also exist for those enrolled in public insurance programs, such as Medicaid [3]. Reports indicate that preventive visits to primary care physicians among 0–6 year olds is 88% [4], while preventive dental use among Medicaid-enrolled, 3 to 5 year-old children is much lower at 48% [5]. To address the disproportionate burden of dental caries in low-income children, studies have been conducted in various environments, including schools [[6], [7], [8]] and community settings such as local health clinics and Head Start Programs [9,10]. Pilot programs have found integrating oral health into well-child visits (WCV) is not only logical and practical, but has also resulted in an uptake of dental referrals and early childhood caries prevention [11]. Further, Medicaid-enrolled children who received preventive oral health services at WCVs are 17% less likely to develop dental caries [12]. Therefore, in primary care settings, pediatricians and nurse practitioners have an opportunity to implement early oral health interventions such as oral health assessments or fluoride varnish application [13].

Few clinical trials exist that have recruited parents and children for an oral health study in primary care settings. But, clinical trials for addressing medical conditions (e.g. coronary heart disease, obesity) have successfully recruited participants at the practice, provider, and patient levels in primary care settings [[14], [15], [16], [17]]. The successful strategies at the practice and provider levels included academic detailing, peer recruitment, leveraging professional associations, conducting pilot studies, and engaging communities of interest [11,13,18,19]. One study found that 90% of pediatricians were of the opinion that they should examine their patients’ teeth and educate families about oral health, implying willingness to participate in and implement new oral health activities [20]. The common strategies in participant recruitment included simple eligibility criteria and consent process, incentives, active approaches, developing strong relationships with practice staff and providers, and providing practices with feedback regarding recruitment progress [11,15,18,19,[21], [22], [23]]. The one oral health observational study in primary care found face-to-face recruitment was the most successful in meeting enrollment goals [24].

The Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework is the most frequently applied implementation framework used for research within clinical, community, and other settings [25]. The RE-AIM framework draws on other evaluation frameworks such as Diffusion of Innovation [26] and PRECEDE-PROCEED [27] models but is also different in that it facilitates translation of research to practice, emphasizes on internal and external validity issues and representativeness, and standardized ways of measuring factors that evaluate public health impact and widespread application [25]. According to the RE-AIM framework [25], participant sample characteristics are important to evaluate representativeness (i.e. Reach) of the target population and provider/staff/setting willingness (i.e. Adoption) to establish external validity [28]. Several pragmatic trials addressing physical activity, obesity, blood pressure and weight loss in primary care have employed the RE-AIM framework to report on implementation and adoption [[29], [30], [31], [32]]. In this oral health pragmatic trial in primary care settings we are also using the RE-AIM framework to evaluate the implementation of multi-level interventions for widespread application in these settings [33].

Due to the paucity of literature on successful recruitment strategies for oral health clinical trials in the primary care setting, our objectives for this study are to: (1) describe the recruitment strategies at the practice, provider, and parent/caregiver levels for an ongoing cluster randomized clinical trial; (2) utilize the RE-AIM framework to report on adoption (providers) and reach (parent/caregivers) according to practice and socio-demographic characteristics; (3) assess the recruitment time length and parent/caregiver reached and not reached in practices; (4) investigate the characteristics of parent/caregivers that contribute to implementation barriers and enablers.

Methods

Study design

The present study is part of an ongoing cluster randomized clinical trial investigating the effectiveness of multi-level interventions at the practice (EMR incorporation of oral health questions) and provider levels (theory-driven based education and skills training) versus enhanced usual care (standard AAP-based provider training) to increase dental utilization among Medicaid-enrolled 3 to 6-year-old children. Participants are being followed for 24 months or completion of three well-child visits. This clinical trial utilizes the RE-AIM framework for evaluation and has been published previously [33]. The study sites are 18 primary care practices located in NE Ohio. Pediatric providers (Pediatricians/Nurse Practitioners) and patients (caregiver and their 3-6-year-old child) were recruited over a 21-month period from November 2017–August 2019. The study was approved by the Institutional Review Board of University Hospitals Cleveland Medical Center. The clinical trial has been registered in clinical trials.gov (NCT03385629). Written consent was obtained from providers and caregiver participants.

The eligibility criteria were as follows: (1) Practices were selected based on Medicaid eligibility of children (20–40% and >40%) and use of electronic medical records (EMR; (2) Providers were Pediatricians/Nurse Practitioners in these practices with a minimum of 2 patient-care days per week; (3) Caregivers of Medicaid-enrolled children attending a well-child visit at these practices were invited to participate. Inclusion criteria was: caregivers aged 18+, spoke English or Spanish, planned to stay at the practice for the duration of the study, child aged 3–6 and enrolled in Medicaid, free from serious medical/behavioral conditions that precluded participation in dental screening.

The present study utilizing the RE-AIM framework is reporting on the reach and adoption components of this framework as relevant to recruitment to assess external validity [34,35].

Recruitment strategies

Pilot Phase: A comprehensive recruitment strategy began during the proposal writing process, was revised during the pilot trial phase and was further refined and subsequently implemented in the main trial. During the pilot phase, an eight-member community advisory board (CAB) of neighborhood/community leaders, individuals from Head Start/schools, a community health center, county public health, a pediatrician, and a caregiver of a young child was assembled. The CAB assessed barriers and provided input to all aspects of the project including recruitment.

The pilot phase also included two focus groups conducted with providers and office managers at two pilot practices (not involved in the main trial). A mock patient walk-through video was also created to illustrate the minimal time commitment necessary for implementation of the study procedures. Two focus group sessions were conducted with caregivers of young children from the two pilot study practices. Feedback was provided regarding incentives and barriers that may facilitate or prevent participation in the study.

Additionally, these two practices were also used to delineate logistics including recruiting providers and caregivers from these practices to determine participation rates and time length of recruitment required for a larger main trial. In the pilot study, a total of 86 caregivers were recruited (95% participation rate) within a 3 month time frame. Based on these estimates and making conservative allowances for an intra-cluster (within-practice) correlation of 0.04, and a 25% drop-out rate, a sample size of 512 participants per arm (total n = 1024) was required to provide an estimated 80% power to detect the necessary difference in dental utilization rates between the two arms. A 9 month time frame was projected to recruit this required sample size for the main trial. Further, the practices were to be rolled out every nine weeks with an equal number of participants from each site to accomplish this sample size goal.

Specific recruitment strategies employed at each level (Fig. 1) for the main trial were as follows: Practices: To determine practices that would participate in the main trial, a project co-investigator (AH), the Medical Director of the Rainbow Care Connection (RCC), a pediatric Medicaid accountable care organization, took a proactive approach to reaching out to 26 practices that were currently part of the research network. An informational webinar led by the co-investigator (AH) and PI (SN) was organized for practices to learn about the dental project. Next, the PI and project staff met with interested Medical Directors and other pediatric providers from practices meeting the inclusion criteria. As an incentive, practices were given a $1000 facility use fee to compensate for study activities taking place at the practices. Eighteen (18) practices provided letters of support committing to participate in the main trial. Practices were randomized to one of the two arms using a restricted randomization scheme with the constraint that equal practices be assigned to each arm. This approach involved the computation of a balance score (for each candidate randomization) based on marginal differences in three key practice-level variables: % Medicaid-enrolled patients [20%–40% and >40%], ratio of patients to providers, and county [Cuyahoga vs. other]).

Fig. 1

Recruitment strategies employed at each level.

Providers: For the main trial, providers who met the inclusion criteria at each of the 18 practices were invited to attend a “lunch and learn” session with the PI and project staff regarding the study goals and logistics. Providers viewed the mock patient walk-through video and provided informed consent. Based on information available about the practices, the goal was to enroll all 67 providers from the participating practices.

Parent/Caregiver-Child dyads: For the main trial, roll-out of caregiver-child dyad recruitment began following completion of provider training at each practice. A “lunch and learn” session was first held with practice staff to view the mock patient walk-through video and to determine logistics that would work best for recruitment at each practice. A research staff and dental hygienist team were assigned to cover each practice during the recruitment period.

Recruitment and enrollment strategies proved to be effective in the pilot study were used for the main trial and included: (1) Practice staff (using a recruitment script) gave a brief overview of the dental study to those meeting the inclusion criteria when making appointment reminder calls, and caregivers were asked to come early to fill out the consent forms and questionnaires; (2) Medical assistants in some practices approached parent/caregivers during triage to introduce the study to show practice support of the project; (3) Caregiver preference was sought for questionnaire completion, i.e. via paper or tablet; (4) Caregivers were given a cash incentive, family tooth care kit, and small gift for the child at each WCV.

Data collection

Practice: The number of practices approached and providing letters of support was recorded at study initiation. Practices were characterized as: percentage of Medicaid enrollees: 20–40% and >40%; size of practice was calculated using the mean number of providers at each practice with small practices consisting of clinics with 1–3 providers and medium practices consisting of more than 3 providers based on prior literature [36]; geographic region (urban, suburban, rural) assessed by location in the county; and county (Cuyahoga, non-Cuyahoga).

Provider: A baseline questionnaire was completed by all providers regarding the following socio-demographic and other variables: age in years, gender (female or male), race (Caucasian, Black, other), ethnicity (Hispanic/Latino vs not Hispanic/Latino), medical degree (MD/DO vs MNP/DNP), formal oral health training (no training vs some training), and years of work experience.

Caregiver: A baseline questionnaire was completed by all caregivers regarding the following socio-demographic variables: age in years, gender (female, male), race (Black, non-Black), ethnicity (Hispanic/Latino, not Hispanic/Latino), marital status (single, married), education (less than a high school diploma/GED, high school diploma/GED or greater), employment status (employed, not employed), and language at home (English, Spanish).

Data analysis

Descriptive statistics (frequencies, means, standard deviation, rates) was used to describe the characteristics of the practice, provider, and caregivers. According to the RE-AIM framework, Reach was calculated as follows: The percent of providers reached was classified as the number consented among the total providers in the 18 practices; the percent of caregivers reached was classified as the number of individuals that were consented among those approached for participation in the practices. The providers who consented also agreed to implement the intervention activities for the main trial indicating adoption.

The average length of recruitment and number of caregivers reached was calculated based on practice level characteristics (practice size, % Medicaid population, geographic region, county). To understand the number of caregivers reached, those not reached (refused/ineligible, no-show, cancelled/rescheduled/missed visits) was calculated for each of the 18 practices. Caregivers reached in the study sample were compared to data from the 2014–2018 United State Census Bureau, 2018 Bureau of Labor Statistics, and Ohio Medicaid Assessment Survey data to evaluate sample representativeness to assess external validity [[37], [38], [39]]. In order to assess implementation barriers and enablers, the association between practice characteristics (practice size, % Medicaid population, geographic region, county) and caregiver socio-demographic variables (age, sex, race, marital status, education, and employment) was calculated using chi-square test. A alpha level of ≤0.05 was used to assess significance. All statistical analyses were performed using SAS software (Version 9.1, SAS Institute, Inc., Cary, NC).

Results

The study recruited 18 practices, 62 providers, and 1024 required caregivers over a 21 month period (November 2017–August 2019). The recruitment strategies employed for the study are given Fig. 1.

Table 1 indicates that the majority of the practices were small, had 20–40% Medicaid patients, most were suburban, and resided in Cuyahoga county. Of the eligible providers approached, 94% (63 out of 67) were reached, as they provided consent and agreed to adopt project intervention activities. Table 1 also indicates that the parent/caregivers reached were from practices that were predominantly smaller, 20–40% Medicaid-enrolled, suburban, and in Cuyahoga County.

Table 1

Caregivers reached and recruitment time length according to practice characteristics.

Practice characteristics	OverallN (%)	Caregivers Reached N (%)	Mean Months of Recruitment
Size of Practice
Small (1–3)	12 (67.0)	643 (62.8)	7.0
Medium (4+)	6 (33.0)	381 (37.2)	5.5
Medicaid Population
20–40%	10 (56.0)	525 (51.3)	7.5
>40%	8 (44.0)	499 (48.7)	5.3
Geographic Region
Urban	4 (22.2)	249 (24.3)	5.3
Suburban	9 (50.0)	490 (47.9)	7.1
Rural	5 (27.8)	285 (27.8)	6.4
County
Cuyahoga	12 (67.0)	677 (66.1)	6.6
Non-Cuyahoga	6 (33.0)	347 (33.9)	6.3

Table 2 indicates that providers were 76% female, 78% not Hispanic/Latino, 78% Caucasian, 86% pediatricians, with a mean age of 47 ± 11.3 years, and average work experience of 16.5 ± 11.1 years. About 30% reported that they had no formal oral health education as part of their health school curriculum. Caregivers were 90% female, 95% not Hispanic/Latino, 56% non-Black, 64% single, 98% spoke English, 93% with a high school education or more, 62% employed and a mean age of 31.2 ± 7.8 years (Table 2). For external validity, comparison with Cleveland-Elyria, OH Metropolitan Statistical Area data (52% females, 94% not Hispanic/Latinos, 81% non-Blacks, 54% singles, 95% employed, 90% English-speakers) indicated that our caregiver sample was representative in terms of non-Hispanic, being single, English speaking population. Comparison with Ohio Medicaid Assessment Survey data of enrolled and potentially Medicaid-eligible Ohio adults (56% females, 95% not Hispanic/Latinos, 80% non-Blacks, 53% employed, showed similar results for non-Hispanic, and English speaking, with the addition of employment status as well [39]. Thus, the caregivers were representative of the Medicaid population.

Table 2

Socio-demographics characteristics of providers and caregivers.

Variables	Mean (SD)	N	%
Provider Level Characteristics
Age (n = 63)	47.0 ± 11.3
Sex (n = 63)
Female		48	76.2
Male		15	23.8
Ethnicity (n = 63)
Not Hispanic/Latino		49	77.9
Unknown		14	22.2
Race (n = 63)
Black/African American		4	6.4
Caucasian		49	77.8
Other		9	14.3
Medical Degree (n = 63)
MD/DO		54	85.7
MNP/DNP		9	14.3
Work Experience (n = 63)	16.5 ± 11.1
Formal Oral Health Education (n = 63)
No Training		19	30.2
Some Training		44	69.8
Caregiver Level Characteristics
Age (n = 1024)	31.2 ± 7.8
Sex (n = 1022)
Female		921	90.1
Male		101	9.9
Ethnicity (n = 965)
Not Hispanic/Latino		914	94.7
Hispanic/Latino		51	5.3
Race (n = 987)
Black/African American		438	44.4
Non-Black		549	55.6
Marital Status (n = 1007)
Married		359	35.7
Single		648	64.4
Level of Education (n = 1008)
< High School		60	5.9
≥ High School		948	92.6
Employment (n = 996)
Employed		619	62.15
Unemployed		377	37.85
Language at Home (n = 1013)
English		995	98.1
Spanish		19	1.9

Practices that were smaller, those with 20–40% Medicaid population, suburban, and in Cuyahoga County took on average a longer time (months) to recruit (Table 1). However, this was because a majority of the caregivers were recruited from these practices. Table 3 indicates the average months at each practice for recruitment with a total time length of 21 months to recruit the required 1024 caregiver-child dyads. Length of recruitment time varied from 3 to 10 months at the practices.

Table 3

Recruitment time length and parent/caregivers reached and not reached by individual practice.

Practices	Scheduled	Approached	Caregivers Reached		Recruitment Length of Timeb	Caregivers Not Reached
			Consented	Rate of Recruitment (%)a		Refused, Ineligible	No-Show	Cancelled, Rescheduled, Missed
P01	110	76	62	81.6	6	14	7	24
P02	166	79	67	84.8	6	12	35	20
P03	113	78	62	79.5	8	16	18	12
P04	187	72	61	84.7	4	11	78	27
P05	58	38	35	92.1	6	3	7	11
P06	147	81	68	84.0	7	13	35	17
P07	75	44	34	77.3	7	10	5	18
P08	67	46	35	76.1	9	11	12	7
P09	197	78	68	87.2	6	10	67	5
P10	116	81	58	71.6	5	23	15	14
P11	96	48	40	83.3	10	8	22	13
P12	114	74	62	83.8	8	12	25	12
P13	140	71	63	88.7	9	8	18	16
P14	132	67	57	85.1	9	10	24	21
P15	142	87	69	79.3	3	18	10	34
P16	112	77	68	88.3	3	9	29	4
P17	114	70	57	81.4	8	13	6	24
P18	142	66	58	87.9	3	8	43	31
Total	2228	1233	1024	83.7	21	209 (9.4)	456 (20.5)	310 (13.9)

a Calculated by dividing the caregivers that were consented with those that were approached.

b The number of months to recruitment.

cN (%).

Table 3 indicates that overall, 2228 caregivers had a WCV appointment scheduled for their child at the 18 practices. A total of 55% of caregivers (1233 out of 2228) came to their appointment and were approached by research staff and of those approached 1024 participants were consented to the study. The participation rate among caregivers were 84% (1024/1233) among those approached. Caregivers that were considered not reached were those that refused or were ineligible (9%), no-show for the appointment (21%), and cancelled, rescheduled, or missed by research staff (24%).

Table 4 indicates the association between practice characteristics and caregiver socio-demographic characteristics. Caregiver age and sex were not found to be significant with any of the practice characteristics. In smaller practices, a significantly (p < 0.05) greater proportion of non-Blacks and single caregivers were recruited compared to medium size practices. In practices which had >40% Medicaid population, the recruitment of caregivers was significantly larger among Blacks, single, employed caregivers, and caregivers that had at least completed a high school education when compared to practices with a 20–40% Medicaid population size. Practices in an urban geographic area had significantly greater Black, single, employed, and at least completed high school educated caregivers compared to practices in suburban and rural areas. Practices located in Cuyahoga County had significantly higher recruitment among Blacks, single, and employed caregivers when compared to non-Cuyahoga practices.

Table 4

Characteristics of caregivers in relation to practice characteristics.

Variables	Size of Practice			Medicaid Population			Geographic Region				County
	Small (1–3) N (%)	Medium (4+) N (%)	p-value	20–40%N (%)	>40%N (%)	p-value	UrbanN (%)	SuburbanN (%)	RuralN (%)	p-value	CuyahogaN (%)	Non-CuyahogaN (%)	p-value
Mean Age ± SD	31.2 ± 7.8	31.3 ± 7.8	0.1937	31.5 ± 7.5	30.9 ± 8.0	0.2096	30.4 ± 7.2	31.4 ± 7.9	31.0 ± 7.9	0.8073	31.3 ± 7.5	31.2 ± 8.2	0.8644
Sex			0.5653			0.3880				0.7302			0.5490
Female	575 (89.7)	346 (90.8)		469 (89.3)	452 (91.0)		226 (90.8)	436 (89.3)	259 (90.9)		611 (90.5)	310 (89.3)
Male	66 (10.3)	35 (9.2)		56 (10.7)	45 (9.1)		23 (9.2)	52 (10.7)	26 (9.1)		64 (9.5)	37 (10.7)
Race			0.0191a			<0.0001a				<0.0001a			<0.0001a
Black	257 (41.5)	181 (49.2)		151 (30.0)	287 (59.4)		213 (89.5)	210 (45.1)	15 (5.3)		417 (64.9)	21 (6.1)
Non-Black	362 (58.5)	187 (50.8)		353 (70.0)	196 (40.6)		25 (10.5)	256 (54.9)	268 (94.7)		226 (35.2)	323 (93.9)
Marital Status			0.0108a			<0.0001a				<0.0001a			<0.0001a
Married	243 (38.6)	116 (30.7)		223 (43.2)	136 (27.7)		48 (19.6)	188 (39.1)	123 (43.8)		208 (31.3)	151 (44.2)
Single	386 (61.4)	262 (69.3)		293 (56.8)	355 (72.3)		197 (80.4)	293 (60.9)	158 (56.2)		457 (69.7)	191 (55.9)
Education			0.2223			0.0399a				0.0084a			0.4266
< High School	33 (5.3)	27 (7.1)		23 (4.5)	37 (7.5)		23 (9.3)	18 (3.7)	19 (6.9)		37 (5.5)	23 (6.8)
≥ High School	596 (94.8)	352 (92.9)		493 (95.5)	455 (92.5)		225 (90.7)	465 (96.3)	258 (93.1)		632 (94.5)	316 (93.2)
Employment			0.6800			0.0158a				0.0071a			0.0051a
Employed	386 (61.7)	233 (63.0)		296 (58.5)	323 (65.9)		165 (67.6)	301 (63.6)	153 (54.8)		428 (65.2)	191 (56.2)
Not Employed	240 (38.3)	137 (37.0)		210 (41.5)	167 (34.1)		79 (32.4)	172 (36.4)	126 (45.2)		228 (34.8)	149 (43.8)

Significant at α < 0.05.

Discussion

To our knowledge, this is the first study to report on recruitment strategies for a pragmatic trial in primary care settings where pediatricians and nurse practitioners are delivering the oral health interventions. This is also the first study to use the RE-AIM evaluation framework for a pragmatic oral health trial. We successfully recruited the necessary practices, providers and parent/caregiver participants required for testing the primary objectives. The trial employed a multi-level recruitment approach that has previously been reported to result in more cohesive and successful strategies, widespread study support and increased rates of enrollment [14,18]. At all levels, active rather than passive strategies were employed. The successful recruitment strategies in our trial are as follows: (1) Practices and providers were approached proactively by an opinion leader or “champion” who solidified interest in the oral health trial, a successful strategy as previously reported [18]. This was followed by in-person meetings and webinar to detail the activities of the oral health trial; (2) Providers were given American Board of Pediatrics Maintenance of Certification (MOC) part IV credits, a valuable motivator for participation. Twenty five part credits were given for improving quality of care for their participating children; (3) parent/caregiver recruitment strategies included research staff developing strong relationships with practice staff and providers to gain support and assistance with introducing the study to those eligible, as well as a cash incentive [18,19]. A research assistant was assigned as the main contact for each practice, which helped with relationship building and identification of key practice staff (e.g., office managers) to consult for tailoring logistics for each practice. An additional strategy was to offer participants the choice to complete study questionnaires on paper or tablet, based on personal preference. Most of our caregiver participants preferred the use of tablets (88%) and this may have been due to the younger age group of our caregivers consistent with prior literature [40,41]. In these prior studies, no significant difference was found in completion rates when comparing paper versus electronic mode of data collection.

The use of the RE-AIM framework was helpful in evaluating the trial in terms of adoption (providers agreeing to deliver the interventions) and reach (providers and caregiver participation rate). Our findings indicate that adoption of interventions by providers at the practices and the rate of recruitment among providers and caregivers was high. Further, our sample was representative of our target population in some characteristics, but also had a significantly higher percentage of females in comparison to the Medicaid-enrolled and overall population of the geographic area. A study reporting influence of caregivers on children's entry into the dental care system found that a vast majority of caregivers of Medicaid-enrolled children are female, which supports generalizability of our findings to our target population [42]. Overall, evaluation of adoption and reach indicates that the intervention has a high likelihood of being adopted in similar pediatric primary care practices and that study results may be generalizable to similar populations as northeast Ohio, particularly those with children enrolled in Medicaid. We also examined the reasons for non-participation of providers and caregivers in our trial to help investigators design future interventions and recruiting from a similar population/setting as suggested previously [25]. The few providers choosing not to participate indicated that they had a very high patient load and thought the intervention could be presented more effectively as written material for the caregiver to read. The caregivers who refused to participate gave reasons such as a lack of interest in research, lack of interest in oral health, lack of time to complete the paperwork, and child behavioral/medical issues.

The recruitment of parent/caregiver participants from the practices was longer than originally anticipated. While the target sample size was reached, the caregiver recruitment took 21 months, more than twice the originally projected time length of 9 months. The lack of realistic estimates of caregiver no-shows, cancellations, and missed visits from diverse practices may have contributed to our over ambitious projections. Further, the information used from the pilot study may have led to an overestimate of the pool of eligible caregivers who could be recruited during this period of time, as reported in similar studies experiencing Lasagna's Law [43]. However, our results are similar to another oral health observational study in primary care settings that reported face-to-face recruitment being most successful, but also the most time consuming and resource intensive [24]. In the future, investigators aiming to recruit from primary practices should consider these challenges in their recruitment plan, and our study affords some valuable estimates of no-shows, cancellations, and missed visits in community practices.

Our findings indicate that enablers to achieving the recruitment goal included Identification of a practice champion, research staff having access to the scheduling and EMR system, and practices with a higher percentage of Medicaid enrolled children. The research staff having access to the EMR facilitated checking schedules of patients in real time. In addition to barriers previously reported in pragmatic trials [44], such as missed visits, same day appointments, no-shows and cancellations, the introduction of a new scheduling/billing system at a majority of the practices caused delays in practice roll-out and logistical limitations for research staff. The frequency of scheduled visits was sporadic at some of the farthest away practices, which made it impractical to be present for all of the scheduled visits. To address slower rates of recruitment at some of the practices, the target goal for some of the slower practices was adjusted, while still balancing recruitment numbers between intervention arms. Additional barriers included practice specific requests to limit recruitment to only certain days/week due to patient flow, provider schedules and physical space constraints.

Interestingly, caregivers who had a high school education or more predominantly participated in our intervention trial across all practices regardless of practice characteristics. A prior study indicates that a higher education level is associated with greater clinical trial awareness and participation [45]. However, for generalizability of results, it is necessary that investigators have recruitment materials and consent documents that are easy to understand and simple to motivate caregivers with less education to participate. Our recruitment materials were at 6th grade level and research staff had a simple recruitment script that was used to explain the consent form. But, the higher grade level of the consent forms due to IRB necessitated language for obtaining consent may have refrained those caregivers with less than a high school education to sign up for the trial.

A limitation of our study in terms of recruitment is that we did not employ different means of recruitment to draw comparisons as to which was most effective. We relied on a previously proven active face-to-face strategy, which while successful, required a great deal of time and resources. We were, however, able to identify specific strategies within our approach which were beneficial to meeting our goals. Other limitations related to evaluating reach, were a lack of demographic information from individuals who were approached for participation, but refused, so that characteristics of the reached and non-reached groups could be compared. Also, our participation rate was limited to calculation of the proportion of those who participated to those approached, rather than to all eligible individuals at the practice. In the future, utilization of electronic medical records/scheduling software at the practices to identify eligible individuals may be helpful to further evaluate reach in similar studies.

In conclusion, the RE-AIM framework was used to evaluate the reach and adoption components. Our recruitment strategies were helpful to recruit and reach the necessary practices, providers, and caregivers for a large-scale oral health intervention trial in primary care settings.

Funding

This work was supported by the (NIH), specifically: the [grant number UH3 DE025487-01; Principal Investigator: Suchitra Nelson, and Coordinating Center Award number U01DE025507, Principal Investigator: Stuart A. Gansky, University of California, San Francisco]. The funders were not involved in any of the following activities: study design; data collection, analysis, and interpretation; writing of the report; and the decision to submit the article for publication.

CRediT authorship contribution statement

Shelley Curtan: Investigation, Writing - original draft, Writing - review & editing, Supervision, Project administration. Tashyana Copeland: Investigation, Writing - original draft, Writing - review & editing. Erin McNamee: Investigation, Writing - original draft, Writing - review & editing. Jelena Debelnogich: Investigation, Writing - original draft, Writing - review & editing, Visualization. Taylor Kula: Investigation, Writing - original draft, Writing - review & editing, Visualization. David Selvaraj: Software, Validation, Data curation, Writing - review & editing. Jeffrey Albert: Resources, Writing - review & editing. Andrew Hertz: Methodology, Formal analysis, Writing - review & editing. Suchitra Nelson: Conceptualization, Methodology, Writing - review & editing, Supervision, Project administration, Funding acquisition.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.