Approaching Retention within the ABCD Study.

Retention efforts are critical to maintain relationships with research participants over time. This is especially important for the Adolescent Brain Cognitive Development (ABCD) study, where families are asked to stay engaged with the study throughout the course of 10 years. This high-degree of involvement is essential to longitudinally track child and adolescent development. At a minimum, we will connect with families every 6 months by telephone, and every year in person, with closer contact with the youth directly as they transition into adolescence. Differential retention, when related to non-random issues pertaining to demographic or risk features, can negatively impact the generalizability of study outcomes. Thus, to ensure high rates of retention for all participants, the ABCD study employs a number of efforts to support youth and families. This overview details the framework and concrete steps for retention.

Introduction

Retention efforts are critical to developing and maintaining relationships with research participants. It is of utmost importance for the Adolescent Brain Cognitive Development (ABCD) Study where families are asked to stay engaged throughout the course of 10 years, with all of the life challenges that will invariably occur in this span of time. Starting with families’ first interactions with the project, positive relationships with study families are integral to successfully keeping them involved and invested in ABCD. ABCD will, at a minimum, reach out to families every 6 months by telephone, every year in person, with closer contact planned for children directly as they transition into adolescence (e.g., more frequent telephone/text-based check-ins starting at age 12).

The pivotal issue with retention is the potential threat to validity. One of the greatest risks to longitudinal research is non-random attrition (Poulton et al., 2015). This can happen, often unwittingly to study investigators and/or participants themselves, when one segment of the study population increasingly begins to “drift” away and stop responding to study investigators. Resultant non-random missingness of data at follow-up threatens study generalizability, particularly if it is differential by demographic group. For example, if one subgroup has very high retention, such as by youth from certain geographic locations (e.g., high retention across the Pacific Coast, but poor retention across the Atlantic seaboard), socio-economic status (SES) (e.g., high retention among middle income youth, but poor retention among high SES youth), gender, and/or cultural groups, then the results from the study can only be reasonably extended to youth within the represented demographic subgroup. Concretely, data gained from the study could not be used to make inferences about the nature of brain or behavioral development for adolescents from the subgroups with poor retention.

Some questions asked within the study revolve around potential risk and resilience markers; if ABCD does not retain youth within a certain demographic group, we undermine our ability to generate representative data about critical risk and resilience signatures that may inform next-step prevention and intervention programming. Additionally, another hazard to retention can include known risk factors. Families that have greater stressors (e.g., families characterized by high conflict; families with youth who are transitioning into psychopathology, beginning to use substances, and/or making risky decisions) may have a harder time organizing their time and resources to make it to research and other study visits. While families of youth with the highest levels of risk features may be the most difficult to retain, they also are arguably one of thrtain demographic group, we undermine our ability to generate representative data about critical risk and resilience signatures that may inform next-step prevention and intervention programming. Additionally, another hazard to retention can include known risk factors. Families that have greater stressors (e.g., families characterized by high conflict; families with youth who are transitioning into psychopathology, beginning to use substances, and/or making risky decisions) may have a harder time organizing their time and resources to make it to research and other study visits. While families of youth with the highest levels of risk features may be the most difficult to retain, they also are arguably one of the most important subgroups to capture within this longitudinal study. Thus, to effectively address the driving a priori aims of this large-scale multi-site study, we cannot have differential retention by risk or demographic subgroup.

Importance of retaining special populations

Compared to what is often described as the Caucasian majority (77.1%), minority races [including African Americans (13.3%), Asian Americans (4.8%), American Indians/Alaskan Natives (1.2%), Native Hawaiians or other Pacific Islanders (0.2%), and individuals of two or more races (2.6%)] collectively comprise almost a quarter of the United States (U.S.) population overall, and in some regions of the nation, represent the majority race/ethnicity in certain states. According to current U.S. racial/ethnic categorization of cultural affiliation, each racial group may be further categorized into Non-Hispanic/Latino (82.4%) or Hispanic/Latino (17.6%) (Census, 2015). While many teams, including our own, have begun to demonstrate otherwise (Cottler et al., 2016, Cottler et al., 2017, Montanaro et al., 2014), recruiting racial and ethnic minority populations for, and retaining them in, research studies can be difficult for many scientific teams. As a consequence, racial/ethnic minorities remain comparatively underrepresented in many U.S. research studies.

Inclusion of racial/ethnic minority groups is not only important to ensure valid findings and avoid bias, but also to assure a representative sample. According to the 2010 American Community Survey (ACS), although Caucasian and Asian children had poverty rates below the U.S. average (∼20%), African American children (38.2%), Hispanic children (32.3%), and children who identified with two or more races (22.7%) reported relatively higher rates of poverty (Macartney, 2011). The relevance of poverty is that families struggling to stay afloat financially can have a tougher time making it to study visits; this can be due, for instance, to limitations around obtaining “time off” during primary, or even, supplementary employment. In turn, studies that centrally operate within traditional business hours may overlap with parents’ work schedules, and as a result, make it virtually impossible for low income and/or working families to successfully show for appointments. To protect against omission or differential retention of special or “hard to reach” populations (e.g., those living in poverty) which can bias research findings (Western et al., 2016), additional efforts are necessary to ensure robust retention of more difficult and harder-to-reach populations in research studies. Resultant, more inclusive and representative samples consequently facilitate optimal prevention and intervention programs to combat target issues for all youth throughout the nation.

Challenges for recruitment and retention of racial/ethnic minorities

Different under-represented communities, including racial/ethnic minority communities and/or impoverished communities, may be reticent to participate in long-term studies that involve continuous connection with research. Some reasons for this include that many U.S. communities have had very difficult intersections with research, with devastating cultural, physical, and health implications. As examples, interested readers are encouraged to review materials about historical and recent research tragedies within the African American (www.tuskeegee.edu) and Native American communities (Harmon, 2010).

Additional publications in this area (e.g., Kelley et al., 2013, Thomas et al., 2011, Venner et al., 2007) report that some racial/minority communities describe their experience with outside scientists as “helicopter research” wherein outsiders come into their community with predetermined questions, collect data and leave, without taking the time to connect with community members at any point during the scientific process. There is particular sensitivity around occasions where researchers have not discussed study findings with racial/ethnic minority community stakeholders prior to sharing them with the outside world. This has resulted in the reported sense, by some racial/ethnic communities, that outside research groups do not recognize or respect racial/ethnic individuals, communities, traditions and cultures; as a consequence, many researchers report experiencing a sense of “impasse” when trying to develop relationships with racial/ethnic communities. Ultimately, due to the bumpy, and in some regions, quite fractured history between the scientific and racial/ethnic minority communities, there is an understandable degree of wariness of research efforts by some racial/ethnic minority communities, and their leaders. Thus, it is critical for scientific teams, including ABCD, to respect the reluctance of communities who believe that many types of scientific research are high on risk, and low on potential benefit.

It is the position of ABCD to invite everyone to have a voice in research, and especially, within this project. Within enrollment, we have made many targeted efforts to develop strong positive relationships across communities, with an open awareness of prior research experiences. In terms of retention, the ABCD approach is to be alert that these current and historical research experiences and perceptions might interfere with a family’s interest in continuing to stay engaged, even in situations where the family has begun participation with the study. The ABCD study staff are thus encouraged to utilize a number of tools to engage and retain reticent families (detailed below), with an open line of communication with families around their sense of inclusion, comfort, safety, and protection.

Prior retention successes for traditionally hard-to-reach populations

A recent review on recruitment and retention of racial/ethnic minority populations for developmental science studies found three important themes: the role of trust, researcher identity and insider/outsider status, and responsibility (Rivas-Drake et al., 2016). Since one major difference of special populations is culture, a culturally appropriate and sensitive retention plan is key to successful retention in longitudinal studies, such as ABCD. Here are some examples that have approached retention in precisely this way.

A study of weight management in two school districts of multi-ethnic elementary school children in North Carolina found that successful recruitment and retention strategies was needed at all levels. Specifically, at the district level, researchers met with superintendents and were introduced to school principals, with lines established for consistent and clear communication throughout the study. At the school level, they developed relationships with other key personnel, for whom ongoing open communication was also essential. With children and parents, researchers allowed adequate time for questions during consent and assent, provided a free nutrition and exercise program, a light meal, homework assistance, child care for other family members who came with the respondent, and transportation vouchers as needed. Overall, clear and good communication at all levels, including attending to the participants’ needs and challenges, led to the successful recruitment and retention of these multi-ethnic participants (Berry et al., 2013).

Cottler (1987) discussed difficulties recruiting participants, and their effect on estimates of psychiatric disorders in the St. Louis Epidemiologic Catchment Area (ECA) project, the first study of psychiatric disorders in the U.S. of the general population (conducted between 1981 and 1982). Being young, male, educated and living in an urban area required more contact attempts. Those with alcohol use disorder (AUD) required almost 20% more contact attempts than those without AUD. The authors found that 32% of the respondents were recruited after two attempts, 66% after by the 5th attempt, and 95% after the 14th attempt. Overall, the average number of attempts was 5.3, with 7.8 attempts the threshold number required to prevent prevalence estimates from being affected. In another study, the Cottler team discussed methods to achieve a 96.6% completion rate among people recruited for a substance abuse prevention study. A comprehensive locator form, good phone and systems tracking, creative and persistent team work, detailed protocols and doggedness, were all integral for successful retention. They include vignettes to highlight examples (Cottler et al., 1996).

One of the key examinations of retention with adolescents, specifically, includes Feldstein Ewing and colleagues’ work with high-risk and underserved adolescents (Montanaro et al., 2014). In an explicit comparison of outreach and retention efforts between youth who were recruited from a high-risk, low-SES, predominantly racial/ethnic minority recruitment site (juvenile justice; n = 240), as compared with a same-age sample of low-risk, high-SES predominantly Caucasian youth from the identical geographic area (private schools; n = 40), the team observed no statistically significant differences in retention at either 3 or 6 month follow-ups, with over 90% retained for all youth across follow-up waves. When examined, retention efforts that generated the most success, particularly with high-risk racial/ethnic minority youth, included additional diligent phone calls, with some degree of allowance and flexibility around rescheduling. Specifically, high-risk youth required a mean of 10.04 (SD = 19.64) calls to achieve a successful show for the 3-month follow-up, and an average of 8.63 (11.19) calls for the 6-month follow-up.

In summary, on-the-ground, community-based research is essential to engage and retain individuals, particularly within and across minority and underserved populations. Understanding and being cognizant of the many challenges involved in retention of underrepresented populations, such as mistrust of researchers, losing contact, logistical barriers (e.g., transportation, childcare, scheduling), and adoption of empirically supported successful strategies, will take an important step toward ensuring better retention of historically under-included and underrepresented study participants in ABCD.

How to keep a cohort involved for decades

Efforts at the start of a large-scale, multi-site study like ABCD can ultimately predict long-term retention. In 1979 and 1980 when the National Institute of Mental Health (NIMH) launched the ECA study, investigators were unaware that Coordinators and Assistant Professors from the original study would be following participants and publishing together 40 years later (Cottler et al., 2016, Eaton et al., 2013, Lopez-Quintero et al., 2015). Prior to the advent of electronic communication, investigators relied on developing strong relationships with research participants, locator forms with telephone “land lines”, contacts through family and community members (e.g., relatives’ phone numbers), and additional creatively-brainstormed, often basic aids. Rarely did these efforts include social security numbers and/or other personal identifiers.

The field has classic articles that discuss the “nuts and bolts” of implementing follow-up studies, addressing training and management of staff, data entry (Stouthamer-Loeber et al., 1992) and working with the participant directly (Robins, 1963). As early as 1963, the term “reluctant respondent” was used to describe participants who are more difficult to handle, and whose “personality” requires additional effort (1963). Yet, in her 30 year follow-up of children from a child guidance clinic, Robins (1963) set no limits on the number of contact attempts and often went to people’s homes directly to garner support for the study; an approach that has often even worked for contemporary research teams (Montanaro et al., 2014). Ultimately, like with our own research outcomes (Cottler et al., 2016, Cottler et al., 2017, Montanaro et al., 2014), Robins (1963) found that people with stereotypical “derogatory information”, including psychiatric histories, had no bearing on willingness to be followed; moreover, “difficult-to-interview” participants reported their histories as honestly as those who were easier to interview.

Three decades later, investigators now have access to an enormous array of new aids to improve prospective study success. In fact, the American Association for Public Opinion Research Task Force on Survey Refusals published a compendium on current knowledge and considerations regarding survey results. Farabee et al. (2016) discuss the substantive, mostly technological, changes that have occurred in the substance use field that are helping to navigate and achieve low attrition rates. This is critical, as there is a high cost to maintain a research cohort over time, and history indicates that retention is difficult for most research teams. To this end, investigators found that 27% of National Institute of Drug Abuse (NIDA)-funded studies could not recruit over 80% follow-up, with outcomes that were subsequently correlated with biased estimates of risks and outcomes (Farabee et al., 2016). Moreover, as reported by Galea and Tracy (2007), rates of non-response have been increasing in research studies each year.

Specific approaches for retention within ABCD

In line with the Dunedin study (Poulton et al., 2015), a critical premise of the ABCD approach is to “Treat people as you’d like to be treated”. That involves anticipating what families might need at different time points in the study (e.g., snacks, childcare, letters for school to support study-related absences), and to ensure that those materials are available for families when they show for study visits. Ultimately, the fundamental framework that guides retention throughout ABCD is that fostering and maintaining strong relationships with youth and families is the key to retention. Throughout the study, the goal is to ensure that families’ experiences with us are as positive as possible. The ABCD study utilizes thoughtful, pro-active approaches that focus on working to solidify connections with families to keep them engaged, with an eye to preventing loss. In terms of staff, project, and family time, the ABCD approach is that it is exponentially easier to keep a family than to re-engage a lost one.

In addition to a number of standard operating procedures (SOP) for ABCD (see Table 1), the guiding principles utilized during visits are captured with the study acronym (ABCD). The effort is to convey to families that ABCD staff are aware that they are doing us a favor by participating.

Table 1

Methods to establish positive interactions, and build a strong rapport with the participants and their caregivers.

a Memorize name of youth and parent before they walk in the door b Be friendly, engage in conversation during down time c Positive lab atmosphere d Be sure to have an activity for family members who may be sitting around (e.g., videos on laptops, magazines). e Have empathy when sensitive information is shared f Show appreciation for their time and effort g Have same research staff conduct initial and follow-ups if rapport has been established h Make the participant as comfortable as possible i Frequent breaks j Snacks and caffeine-free drinks k Ensure that parents have a way to cover their other children who may need childcare l Ensure that parents/families have letters “excusing” their absences from work/school m Pay youth and families at each assessment point n Stress confidentiality, especially between children and parents o Provide research team’s project business cards and study website p Be proactive about keeping families updated about delays or inconveniences that affect them as a participant (slow turnaround in Accounts Payable to print checks, scanner servicing that bumps a previously scheduled visit, etc.)

(A) Anticipate their needs. To this end, many families have traveled, as well as missed other organizational or family time to make it to our appointments. Thus, ABCD sites ideally accommodate those families (and their potential additional children), by providing snacks and drinks for participants and their family members. Sites also ensure that families, including parents, participants, and potential siblings, have something fun to do to occupy themselves during downtime (e.g., videos; magazines). ABCD sites are encouraged to offer “doctor’s notes” for families missing school or work for study participation. Finally, ABCD sites are asked to make every effort to ensure coverage or assistance with transportation to/from study appointments.

(B) Be positive and respectful. Staff working within each project site are careful to convey their sincere appreciation of families’ contribution. To this end, sites also ensure that families receive prompt and timely compensation/payment – ideally at each interaction.

(C) Care for family members. Sites work hard to ensure that MRI scans and assessments are scheduled for days/times mutually convenient for families and ensure that families have a comfortable place and/or plan for care for their non-participating children.

(D) Develop rapport. The goal here is for the experience within the project to be sufficiently positive so that participants and their families want to come back.

During intervals between visits, ABCD sites take several steps to ensure that families stay connected to the project. First, site staff reach out and connect with families to offer helpful reminders of upcoming appointments. Here, site staff use telephone and text to check in with families around planning for upcoming visits. In addition, during these calls site staff can ensure that families and participants have what they need in order to successfully make it to the appointment. Second, site staff stay closely connected with participants and families. Based on prior successful retention studies, an average of 10 contact points per family may be utilized to keep even very-high risk families engaged and retained in longer-term follow-ups (Montanaro et al., 2014). Thus, through a variety of contact methods, including dissemination of ABCD newsletters by the ABCD Coordinating Center (CC), birthday cards and other holiday outreach efforts, ABCD study staff have a variety of outreach and interaction opportunities at the ready, to frequently connect with families between data collection points. In addition, given the quickly changing landscape of electronic communication, the overall ABCD consortium will continue to monitor and flexibly adapt to whichever methods of contact are most effective to successfully reach families as well as the adolescents themselves, as they age through the project (e.g., phasing out landline calls in lieu of more effective text connection).

Navigating hard-to-reach families

In an active effort to avoid differential attrition by demographic or risk group, study staff are alert to knowing that hard-to-reach families are precisely the families who require the most attention. The following efforts are utilized to prevent loss with hard-to-reach families.

(1) Keeping detailed locator information up to date. Loss of contact with participants is a major cause for attrition in longitudinal studies, especially for those with unstable living situations. Thus, obtaining contact information from at least three family members, relatives or friends, is necessary to maintain contact and communication (Cottler et al., 1996, Montanaro et al., 2014). Concretely, within ABCD, comprehensive tracking and location information, initially completed primarily by the parents/guardians of the participating youth, is obtained during the initial individual session (see example locator form; Fig. 1), and updated at every subsequent visit to ensure that it is current. As youth age within the program, ABCD will transition from having families complete most of the contact information, to having pre-teens/teens actively complete and update contact information for themselves.

Fig. 1

ABCD Study Locator Form.

(2) Navigating collateral reports for retention information. Given the age of youth at entry into the study, those initially best qualified to provide detailed contact information are parents or legal guardians with whom the child lives. Parent collateral reports afford opportunities to maintain location information of youth participants as youth and families transition residences over time. Optimal collateral reporters will likely change over the duration of ABCD. As youth age and become more independent from the family, others may better help locate youth. While parents/guardians should continue to be contacted for locating information, collateral reports from other sources should be considered if the subject moves away from home with personal belongings for longer than 3 months, is in a stable romantic relationship for 6 months or longer, or has a legal change in residence with new roommates. Consideration of change in collateral reporters, including obtaining information on ways to locate youth, from parent to other important individuals’ in the youth’s life, should be conducted with full consultation and consent of the subject. He/she may wish to discuss the change directly with the new source or prefer to ask research staff to make the contact on his/her behalf. Depending on the degree of participation in the study, the collateral reporter might be required to complete an independent IRB approved consent form for the study specifying content and duration of their involvement.

(3) Weekly meetings. Project staff are encouraged at each site to meet weekly with ABCD site PIs to discuss the hardest to reach families and carefully problem solve strategies that might help to resolve barriers that are obstructing potential participation. This is also a time where ABCD staff can discuss any feedback that they are receiving from families so that they can adaptively integrate any potential adjustments in approach that might improve connection with families.

(4) Diversifying contact times and methods. Site staff are encouraged to try to reach families at different times of the day (e.g., early AM before work, around dinner time, in evening around 7) and on weekends in order to improve chances of catching them when they are free to talk. When possible, contacts are by the staff that the family worked with during their initial visit, to maximize the rapport and connection that was developed during the initial visit. Staff are also encouraged to use different numbers available to the research site; this may circumvent families’ reluctance to hear from the university (who they may be avoiding for other appointments/reasons), and increase the likelihood of responding to the call. Staff are encouraged make notes of the family’s preference for, and most successful times to call, methods (e.g., landline, cell phone, text), and numbers to use when they have success reaching a participant/family.

(5) Other routes for maintaining contact and facilitating retention. Study staff also reach out to families using other methods (e.g., text, email) if they are not responsive to phone calls. For sites that don’t have cell phones, research staff can “text” phones from their email accounts. If families are no longer at their original residence, reaching out to the school to see if they have updated contact information for that family, also can help keep families retained.

(6) Swapping out staff. Sometimes families and participants connect with a specific staff member at a site. For this reason, if a family is not responsive or connected with a study staff member, ABCD sites are encouraged to change out the staff that is trying to connect with the family. At a minimum, the change of energy/personality can help prevent against ABCD study staff burnout, and at best, this can be a way to re-connect with a family who may have an easier time communicating with a different staff member.

(7) Giving families a little break. Given the length of participation within this study, families are invariably going to come up against distracting, and sometimes, very serious life issues. If a family seems resistant and/or is not responding, it may very well be because they are contending with other life issues. Thus, ABCD staff have been encouraged to give non-responding families a week or two before reaching out to them again, with the caveat that it is better to be two weeks late on an assessment than to annoy a family and subsequently lose a family to withdrawal.

(8) Maximize flexibility to accommodate families. The more creative and adaptive sites can be in working with families, the better the retention. To that end, sites have been encouraged to schedule, to the degree possible, in accordance with the families’ preferences and needs. For example, if families are coming a long way, stacking appointments might be preferable, in order to avoid additional travel. For other families, it might be preferable to break study participation into several separate appointments, rather than one very long one.

(9) Offer food at all data collection points. Snacks/meals are provided to youth and families at every in-person assessment or contact period. Many ABCD sites have a cabinet of caffeine-free snacks (e.g., juice, water, cookies, pretzels) for families, including parents/guardians, and participating and non-participating youth (e.g., youth and their non-participating siblings). In addition, every effort is made to provide lunch or dinner to families when participation times run through standard meal times (e.g., breakfast, lunch, dinner).

(10) Assisting with travel. Travel is one of the largest logistical barrier for most families. Research staff work closely with families who have to travel to ensure that they have transportation to the site for the follow-up. For example, in regions where public transportation is effective, families are provided bus/metro passes to offset the cost of their travel. Families living in rural areas or regions without reliable public transportation should have an effective plan for coming to the appointment (not counting on older siblings who might be unreliable). Some sites may choose to reimburse taxi/ride-sharing costs to ensure that families can make it to appointments. Families who cannot complete appointments in the office (where possible) may have the opportunity to complete measures over the phone.

(11) Navigating family moves. If a youth has moved out of the area, every attempt is made to schedule the follow-up for when they return for a visit. If it is in the area of a partnering site, ABCD sites will coordinate to turn over to the family to the physically closest partner site. If the family cannot participate in a certain session(s), ABCD study staff will forgo that session in order to achieve the larger goal of staying in touch with them. As many families move back to their original location, staff is encouraged to make every effort to stay in contact with the family (via telephone, text, email, for example), even if they are not residing close to a participating site, so that ABCD can re-engage them upon their return.

(12) Monitoring of rates of success in retention during the bi-monthly Council of Investigators (COI) call. On a higher administrative level, all PIs within the ABCD Consortium are closely watching their overall success, not only in terms of recruiting participants across critical demographic variables (e.g., race/ethnicity, SES), but also their site’s degree of success in retention, so that they can be commended for strong efforts, and adjust their site’s approaches to improve their success across these domains. Ultimately, the more flexible, adaptive, and positive sites are able to be, the more successful the retention over the course of 10 years is likely to be.

Motivational interviewing

One way to navigate communication with all, and particularly reticent families, is through the use of motivational interviewing (MI) (Miller and Rollnick, 2013). MI is a client-centered, respectful way to communicate with families. It is particularly good at fostering the development of strong working relationships with families and participants. And, it has been shown to be highly effective in communicating across cultural lines (Miller et al., 2007). ABCD staff have been trained in MI via annual trainings with expert MI trainers who specialize in the use of MI with children/adolescents (including the first author). These have been memorialized via videorecording, to facilitate onboarding of new staff in this approach. In addition, MI approaches are reviewed with ABCD site staff during the monthly RA teleconference calls.

Aspects of MI that are particularly useful in terms of bringing in and retaining families, include the fundamental notion of ambivalence (Feldstein Ewing et al., 2016). The idea of ambivalence is that people can strongly feel two different ways about things; in the case of retention, a family can, to equally strong degrees, want to come in for their appointment, while also feeling like they are over-burdened and unable to take on the efforts involved in returning for a visit on top of normal, routine life demands.

A second component of MI that is highly relevant in bringing families in for retention includes rolling with resistance. The idea herein is that study staff working with families will encounter families’ resistance, or disclosures of the negative side of ambivalence (e.g., “It is pretty tough for us to return”; “I’m not sure we are going to be able to keep doing this”). The goal for ABCD study staff is to be patient with families’ expression of the potential annoyances around participation, but not to be pulled into an unproductive conversation that exacerbates families’ frustrations with the project; rather, the goal is for staff to “roll with” negative comments.

ABCD study staff are to respond with accurate empathy, a third element of MI, that reflects staff’s true, attentive, listening to families’ experiences (positive and negative) with the study, and warm support of families throughout the scheduling, participating, and follow-up processes. A fourth relevant component of MI includes support of self-efficacy. Self-efficacy represents the dimension of “feeling able to do something”; in this respect, a family might not feel like they are able to assemble all of the organizational pieces requisite to return for a follow-up visit. Support of self-efficacy in this context would therefore include warmly reminding families of their ability to show for prior appointments, staff’s supportive belief that they can use those same skills to help facilitate the return for the next appointment, and collaboratively problem solving around how to overcome existing barriers to successfully return (e.g., “You guys did such a great job coming last year! What was something that helped you successfully make it here last September?”)

Emphasis on autonomy is a fifth element that will become increasingly important for adolescents as they transition into more active and independent participation roles in the project. In this vein, once adolescents are closer to 12 years of age, and can potentially participate on their own, emphasis on autonomy would include staff support for adolescents’ movement into independence, and their ability to successfully take responsibility for completing tasks on their own (e.g., “Now that you are growing up, we want to check in directly with you!”) Finally, a sixth relevant element of MI includes affirmation of successes, even when those successes are small. Families have overcome a lot of barriers and inconveniences to show for their follow-up appointments. Even if they have not brought everything that they needed, or even if they are not able to complete the full assessment, ABCD study staff applauds what successes they have achieved. This reflects our positive, supportive position that families truly are doing the project a favor by participating, and site staff are thankful for families’ continued support and involvement (see Table 2 for specific MI skills used in ABCD retention).

Table 2

Basic motivational interviewing (MI) skills tailored for use in ABCD retention efforts.

K: Know your families	O: Open-ended questions
I:Interest in their experiences	A: Affirmations
N: Work with them to meet their Needs	R: Reflections
D:Dedication to and respect for their service	S: Summary statements

Enhancing efforts to retain families across cultural backgrounds

The goal is to ensure that ABCD has a sample that accurately reflects the demographics of the U.S.. To achieve this, ABCD must be particularly good at retaining under-represented youth. Due to even greater life burdens for low SES and under-represented families, families experiencing high life burden (e.g., multiple jobs; low income; insecure housing; undocumented status; immigration issues), may have a more difficult time returning for subsequent appointments. In order to insulate against this risk, the ABCD study has taken a number of steps, including identifying barriers that might prevent families from participating during annual ABCD staff training meetings, and COI calls. Additional facilitative efforts include meeting with relevant representatives and groups (e.g., potential elders in American Indian/Alaska Native communities) to identify and collaboratively resolve potential obstructions to continued long-term participation.

Iterations of resistance

Resistance is a critical obstructing factor to continued participation. It can take many forms, including from overt statements about appointment-scheduling frustrations (e.g., “We simply don’t have time to do this right now.”), to more passive expressions, including not responding to ABCD study staff calls and contact efforts and/or not showing for planned appointments. Ultimately, ABCD study staff must attend to not only fatigue expressed by families, but also, staff expressions of fatigue. In a long-term, highly involved study such as this one, staff have to exert a ton of effort to keep families involved. This means that they often have to work in the mornings, evenings, and weekends, in order to accommodate families’ schedules. The result is that sometimes study staff can be tempted to express their difficulties with families directly (e.g., “But, I just scheduled you!! You can’t no-show! Do you know how much work went into organizing this?”) Expressing negativity to families directly can have devastating deleterious impact on retention.

Thus, ABCD study staff are trained to employ their MI skills when encountering family resistance, with particular encouragement to use affirmations to help move families from feeling frustrated back to feeling positive and engaged in the project (e.g., “I know how much you have on your plate. That’s exactly why we are so thankful that you are making time to come back in. What can we do to help you out in scheduling your next appointment?”) As an additional level to protect against this, ABCD study PIs actively monitor, through weekly meetings and informal staff conversations how study staff are faring. Together, ABCD study site teams collaboratively work to identify specific resistance issues and problem solve how to address them. As an example, if a certain study staff member is having difficulty with a family, the PI or coordinator has the flexibility to rotate out that study staff so that a different staff member works with that family. To protect against staff burn-out ABCD site PIs are also very supportive and affirming of their site staff, actively attending to areas where staff might be feeling stretched too thin, and working to insulate staff against those experiences.

Efforts for locating lost participants

Despite all of these efforts to retain participants, there will be participants that do not respond to follow-up inquiries or refuse to continue participation. There are several different strategies to address this risk.

(1) Internet Tracking. Tracking is much more common now through systems that are both superficial and deep. Places like Google, Yahoo, and others are considered superficial, while the National Death Index, Department of Corrections and the Federal prison system are good, deep systems, to track people, especially with drug use histories. Additionally, old fashioned credit bureaus (most require fees) remain solid sources of information on addresses (Corsi et al., 2006).

(2) Social Media. Social media offers researchers a way to continue to engage participants. With the use of Facebook, for example, longitudinal studies are finding people they have not otherwise been able to locate. Community engagement programs around the country have been helpful regarding giving underrepresented populations access to computers to allow people the opportunity to check their email and their Facebook accounts. These resources have especially helped younger populations connect with study personnel. In ABCD, social media is not used as an avenue to track participants, but is engaged as an avenue to disseminate project information (e.g., ABCD newsletters) to interested participants and families.

(3) Cell Phones. A large proportion of residents in the US have cell phones, with increasing number of individuals and families obtaining smart phones. In a recent community engagement program, HealthStreet, participants were queried whether they had a cell phone and if they used text messaging. Among the cohort of 8800 community residents in Florida, 74% endorsed use of this type of communication/social media (Varma et al., 2016). Use increased with age; even older adults reported texting. The caveat of cell phones as a route of contact for low SES and high-risk populations, including those who use substances, is that prepaid plans are often the norm. Thus, phone plans are often turned off and on at various times throughout the month. ABCD study staff will try to re-contact people, even in these cases, as phones and plans are often reinstated within a few days of being shut off or expiring.

(4) Refusal Conversion. Refusal conversions are the procedures that survey researchers use to gain cooperation from a sampled respondent who has refused an initial survey request. The goal of converting initial refusals is to raise the survey response rate, under the assumption that this may lower the potential for refusal-related unit nonresponse error. The research literature contains reports of successfully converting refusals in telephone surveys between 5% and 40% of the time.

Survey research firms coined the term refusal converter—often known as the closer, or the person who comes in at the end and talks to people who have already refused (more than once) to engage and convince them to join/stay in the study. When a potential participant refuses to join or continue in the study, there are two options: let the person go, or continue to recruit. The conversion might be related to a temporary event or feeling, or might be more serious (the person really does not want to be bothered).

The art of refusal conversion has not been well documented. Papers that discuss protocols for conversion state that: 1) refusal conversion should be over the phone or in person, but not through mail; 2) waiting at least 7 days before re-contacting again after there is a refusal and even longer interval for highly recalcitrant individuals; 3) there are two types of refusal conversion: use of the exact same survey request approach as the initial contact, or use of a different approach to convert the initial refusal (modify survey procedures); 4) use of a tracking form to document the contact and particulars of the contact such as different times of the day and days of the week; and 5) the cost-benefit of converting a refusal versus not attempting further contact. The field also suggests that investigators: 6) examine the characteristics between those who do and do not respond, and the benefit of varying incentives that might change the person’s mind and talk them into coming back to the study (Burton et al., 2006). However, this has historically been tricky for IRBs to navigate with substance users (Cottler et al., 1995, Festinger and Dugosh, 2012, Festinger et al., 2008).

Other advice for converting refusals in longitudinal studies includes knowing when to end the contact. This is a highly variable signal that only “converters” may understand. One approach for this is to end the relationship with a letter that states this will be the final attempt unless the person calls back. Rewards can be offered to the refuser for recontacting the study one last time and giving the team advice about why they are not interested any longer.

Converted refusers in face to face interviews contributed to 1.2-8% of completed interviews (Lynn and Clarke, 2002) and 7.5% of completed telephone survey responders (Lin and Schaffer, 1995). Research shows that converted refusers are more likely female, persons with lower income, residing in urban areas. With participation rates steadily decreasing in recent decades, it is important to focus on innovative ways to recruit and retain these hard to reach participants (Galea and Tracy, 2007).

Efforts by the broader ABCD consortium

Given the large time commitment to the study, it is critical that families feel like a valued member of the team. We want to ensure that they do not have the experience of feeling “forgotten about” during the year, only to be called upon when we need to collect data. To that end, the ABCD Coordinating Committee (CC) is developing thank you cards and certificates of participation to acknowledge their commitment and show our appreciation. The ABCD CC has also developed materials to help sites remain in touch with participants and families throughout the year with birthday and seasonal cards, wishing families, for example, a fun summer break, or celebrating the New Year.

With the goal of keeping participants engaged on what is occurring within the broader ABCD study, the ABCD CC has developed a monthly newsletter (disseminated via email) and are continuing outreach and connection to families information to let participants know about what is going on in ABCD nationally. In these efforts, different study teams and activities are highlighted, news about the study, and/or about brain development more generally is presented. Artwork and quotes from participants to build a sense of community among our many stakeholders is included. And as results begin to emerge, these vehicles will keep participants informed about how their participation in the study is making a real contribution to science.

Conclusions

Good systems, strong planning, and positive interactions with families on the front end will ensure good retention down the road. It is therefore our goal to build strong relationships with families, so that they want to stay in touch with the ABCD project as they transition and move throughout their lives. With highly trained ABCD study staff conducting positive and professional interactions with families, especially during the initial years of the project when the children are very young, the ABCD project aims to continue to capture and effectively retain under-represented families who may be hesitant to interact with research staff, and/or who may have had poor interactions with medical and other social service professionals. This important positive, pro-active approach that focuses on working to solidify connections to keep families engaged allows retention efforts to be geared toward preventing loss of youth, rather than trying to re-engage families that have been lost.

Conflict of Interest

None.