Innovative methods for remote assessment of neurobehavioral development.

In response to the COVID-19 pandemic, research institutions across the globe have modified their operations in ways that have limited or eliminated the amount of permissible in-person research interaction. In order to prevent the loss of important developmentally-timed data during the pandemic, researchers have quickly pivoted and developed innovative methods for remote assessment of research participants. In this manuscript, we describe methods developed for remote assessment of a parent child cohort with a focus on examining the perinatal environment, behavioral and biological indicators of child neurobehavioral development, parent-child interaction, as well as parent and child mental and physical health. We include recommendations relevant to adapting in-laboratory assessments for remote data collection and conclude with a description of the successful dissemination of the methods to eight research sites across the United States, each of whom are involved in Phase 1 of the HEALthy Brain and Child Development (HBCD) Study. These remote methods were born out of pandemic-related necessity; however, they have much wider applicability and may offer advantages over in-laboratory neurodevelopmental assessments.

Introduction

In response to the COVID-19 (SARS-CoV-2) pandemic, research institutions across the globe modified their operations in ways that have limited or eliminated the amount of permissible in-person research interaction (Korbel and Stegle, 2020, Omary et al., 2020). Even when restrictions lifted at points during the pandemic, many participants and researchers remained reticent to resume in-person research interaction (Omary et al., 2020). This is particularly true for research being conducted with pregnant people and parent-infant dyads, given that data suggest their increased vulnerability to severe disease when they contract the coronavirus (Zambrano et al., 2020) and limited knowledge of the long-term consequences of COVID-19 infection.

A challenge arises in projects for which data collection is time-sensitive (Korbel and Stegle, 2020). A notable example of this is human longitudinal neurodevelopmental studies for which key assessment time points are being delayed or missed. In these studies, the timing of neurodevelopmental assessment is critical due to the rapid progression of brain development over the first several years of life (Payne and Bachevalier, 2009, Tau and Peterson, 2010). In order to prevent the loss of important developmentally-timed data during the pandemic, researchers had to pivot and rapidly develop innovative methods for remote assessment (Barroga and Matanguihan, 2020).

In this manuscript, methods developed to allow for remote assessment of the perinatal environment, child neurobehavioral development, parent-child interaction, and parent and child biological functioning are described. Methods described focus on measures that a) capture distinct dimensions of infant neurobehavioral maturation; b) emerge and can be reliably assessed early in life; c) are sensitive to elements of the prenatal and early postnatal environment; and d) may represent risk factors for psychopathology. We also address methods that capture the quality of caregiver-child interactions and remote biological measures that can be obtained in both parent and child, with potential to influence brain development. In developing and implementing these methods, great consideration was given to standardization of task administration, measurement reliability, and validity of the measures following task modification. Preliminary data provide encouraging evidence that we achieved high rates of standardization and that the observational measures obtained can be coded reliably, though validity studies are still needed. We are currently undertaking one such validation study, where participants are asked to complete two study visits, one in their home using the remote methods and one in the laboratory. The visit type (remote vs. in-person) will be counterbalanced and the data obtained at the two visits will be compared to assess agreement.

These modifications to existing research protocols were born out of pandemic-related necessity, however the resulting set of remote methods have the potential to accelerate the evolution of the field of developmental psychology. That is, these methods have much wider applicability and have already proved to offer benefits over in-laboratory neurodevelopmental assessments for both study operations and ease of participation. For example, remote data collection methods do not require travel and thus tend to be less burdensome for participants. Remote methods may also lead to less data loss and higher retention rates among participants who move out of the area. Another potential advantage offered by these methods is that expanded study designs are now feasible in which researchers have access to a wider participant pool, including participants from rural areas. This could potentially increase the diversity and representativeness of study samples. Finally, these methods have significant potential to facilitate the harmonization of study design and data collection methods across institutions, for multi-site and consortium studies. Data harmonization and large-scale multi-site studies are critical in research areas such as brain imaging where recent studies highlight the necessity of large sample sizes in order to generate reliable and reproducible data (Button et al., 2013, Kennedy et al., 2019).

Details are provided about the adaptations to in-laboratory assessments of child neurobehavior, parent-child interaction, and parent and child biology to allow for remote collection. We also include a description of how we successfully disseminated these methods to eight research sites across the United States, each of whom were involved in Phase 1 of the HEALthy Brain and Child Development (HBCD) Study (Volkow et al., 2020). The process of implementing these methods at multiple sites across the country provides a basis for harmonization and collaboration and has potential to inform study design of Phase 2 of the HBCD Study. The goal of this manuscript is to describe the principles and procedures that we used to adapt traditional in-laboratory data collection procedures for use in a remote data collection context, we believe that these have relevance for future study design, both for the HBCD and beyond. However, given the substantial rationale and empirical support needed it is outside of the scope of this manuscript to nominate particular measures for inclusion in Phase 2 of the HBCD Study.

Remote data collection methods

Infant neurobehavioral development

As is true for most dimensions of child development, laboratory-based assessments of infant neurobehavior are considered a valuable measurement modality because they provide a standardized assessment context and yield objective ratings of infant behavior. Therefore, they are commonly employed in longitudinal, developmental studies. One of the first priorities in creating these procedures was to design methods for collecting task-based observational data remotely. Multi-modal assessment is critical, and these assessments are complemented by parent-report questionnaire data, that can be collected remotely via secure web-based data capture tools. The primary method selected for data collection was a secure videoconferencing platform to conduct study visits. Conducting visits via videoconference allows for flexible interactions between research assistants (RAs) and study participants. The medium has proved to be reasonably intuitive and familiar for most participants (based on visit ease and participant feedback), and most videoconferencing platforms are available for use with any electronic device with an internet connection (i.e., smart phones of all brands, tablets, laptops, and desktops), offering maximal flexibility for participants.

The process of translating in-laboratory protocols began with consideration of which in-laboratory tasks could reasonably be conducted via a videoconference session (Table 1). Tasks that require the child to interact with highly controlled stimuli and tasks predicated on the child being in a non-familiar setting were eliminated. We reasoned that these tasks could not be conducted via videoconference without fundamentally changing the information gathered. At the first stage of this process, these determinations were made by three investigators who are authors on this report. However, the remote data collection protocol was subsequently reviewed by more than twenty experts in infant neurobehavior as part of the methods dissemination described below. The next consideration was whether the task could be administered by the parent with minimal RA coaching. This determination was made after piloting the remote data collection protocol, with feedback from RAs and pilot participants. Last, we, in collaboration with expert coders, considered whether non-critical components of the in-laboratory assessment (e.g., certain camera angles) could be modified or eliminated without losing information critical for assessing infant neurobehavior. The following tasks were selected based on these criteria.

Table 1

Feasibility of Modifying Infant Behavioral Measures.

Task	Feasibility of Remote Adaption	Construct	Preliminary Quality Rating (% Useable)
Arm Restraint Procedure	3	Negative Valence	94%
Still Face Paradigm	3	Negative Valence	96%
Barrier Task	1	Negative Valence	–
Unpredictable Toy/Spider Task	1	Negative Valence	–
Stranger Approach	0	Negative Valence	–
Mother-Child Free Play	3	Social Processing,	97%
Don't task	3	Social Processing, Cognitive Processes	95%
Do task	3	Social Processing, Cognitive Processes	95%
Strange Situation	0	Social Processing, Negative Valence	–
Object Permanence	3	Cognitive Processes	98%
3-D Object	1	Cognitive Processes	–
Visual A-not-B (Delayed Response)	1	Cognitive Processes	–
Detour Reaching	1	Cognitive Processes	–
Snack delay	2	Cognitive Processes	99%
Visual Attention Task	3	Cognitive Processes	94%

*Scale for feasibly of remote adaption of the behavioral task: 0 = remote administration not possible, 1 = administration possible but not feasible, 2 = administration feasible with modification, 3 = minimal/no modification needed. Quality of ratings (the percentage of videos that were administered that were deemed usable) are reported on the subset of videos that have been assessed at OHSU and NYU at the time of this report (n = 348 for most tasks).

As is true for any type of data collection, a critical feature in adapting these remote methods involves training research staff on essential elements of each task for capturing the constructs of interest during coding. This includes providing training on which environmental features need to be captured and in what degree of detail, which features or camera angles should be prioritized if the ideal cannot be achieved, and techniques for ensuring the highest quality video recording possible.

General procedures

The typical workflow for remote neurobehavioral assessments is to identify interested participants, provide study information, obtain remote informed consent, and schedule a virtual study visit. Scheduling the visit primarily involves sending the participant a unique link to a secure videoconferencing meeting session that is password protected and meets all other security requirements put in place by the sponsoring institution.

Prior to the visit (as part of standard visit confirmation communication) participants are sent a series of photos that illustrate the types of tasks that they will be asked to complete, including examples of how a video-enabled device might be positioned relative to the parent-child dyad (Fig. 1). In this communication, parents are also asked to collect a few items to aid task administration. Though it would be ideal for each family to be provided with a standard set of stimuli for all aspects of the visit, it was not logistically or financially possible to provide all of the needed items to each participant. Instead, the parent is emailed a list of common household items (e.g., a high chair or child seat; an attractive toy; two identical washcloths or other small pieces of fabric and a small toy that will fit under one of these; a small snack and plastic cup; a few age-appropriate toys) as well as photos of the stimuli that are typically used in a laboratory setting. The parent was asked to assemble items similar to those depicted in the photos. It is recommended to start the virtual visit by reviewing the items that the parent has selected to ensure that their use will not compromise the task administration or video quality (e.g., a toy that is large and obscures the child’s face).

Fig. 1

Example Images of Remote Behavioral Assessment. Prior to the remote visit participants are provided with example images. (A) The recommended positioning of the video enabled device for still face and arm restraint task which captures both a head-on view of the child and at least a profile view of the parent throughout the tasks. (B-C) Images of the still face paradigm. (D-E) Images of the arm restraint task. (F) The recommended positioning of the video enabled device relative to the parent-child dyad for the free-play task and (G) an example image of the free play task.

Negative affect tasks

Heightened infant negative affect is an early emerging transdiagnostic risk factor for psychopathology (Frick and Morris, 2004, Nigg, 2006, Nigg et al., 2004, Rettew and McKee, 2005) that is associated with qualities of the prenatal (In, Monk et al., 2019) and early postnatal (Lengua and Kovacs, 2005, Putnam et al., 2002, Sroufe, 1997) environments. Neuroimaging studies suggest that differences in negative affect are associated with subcortical structures including the amygdala and hippocampus as well as prefrontal structure including the anterior cingulate cortex and right dorsolateral prefrontal cortex (Fox et al., 2008, Nelson, 1994, Whittle et al., 2006). Additionally, observed infant negative affect has been associated with greater relative right-frontal electroencephalographic (EEG) alpha asymmetry (Fox et al., 2008, McManis et al., 2002, Perone et al., 2020), providing support that negative affect is a relevant marker of infant brain development.

Still face paradigm

In the Still Face Paradigm (Mesman et al., 2009, Tronick et al., 1979), the parent and child are seated across from one another, face-to-face. The child is secured into a highchair throughout the task, while the parent is seated in a chair directly opposite the highchair during three two-minute episodes. The first episode involves face-to-face interaction between the parent and the child (they are asked to play “peek-a-boo”). In the second episode, the parent is asked to assume a neutral, expressionless face and to not interact with or respond to the infant. In the final episode, the “reunion,” the parent and child are asked to interact as they typically would. When conducted in the laboratory, the RA, who is typically seated in a control room adjacent to the testing room, narrates the episode via a microphone/speaker system, using a standardized script. In the laboratory, this task is typically recorded using two cameras, one that captures a head-on view of the mother and a second that captures a head-on view of the child.

Adaptation

Because the in-laboratory protocol already has the RA providing instructions from a separate room, relatively few changes were required to adapt this task to a remote data collection format. The main modification was that, rather than asking the parent to sit immediately across from the child, they were asked to place their chair at a 45-degree angle, so that a single camera could capture both a head-on view of the child and at least a profile view of the parent throughout the task (Fig. 1). This modification was made to accommodate the fact that many participants only have one internet enabled video-capable device.

Challenges and solutions

Though some nuances of the parent’s facial expressions can be more challenging to view when only their profile is visible, we found that the 45-degree angle modification has resulted in minimal loss of information. Of note, we found that a profile view is sufficient to monitor whether the parent interacts with the child during the still face episode (i.e., if they “break” the still face), which is the most important factor in ensuring data quality for the primary outcome, child negative affect during the still face episode.

Gentle arm restraint

This task is an adapted version of the classic Laboratory Temperament Assessment Battery (Lab-TAB) task (Planalp et al., 2017). In this task, the child is secured into a highchair with a tray attached. The parent stands behind the high chair and is asked to remain silent throughout the four-epoch procedure. In the first epoch, an attractive toy is placed on the child’s tray and the child is allowed to interact with the toy for 30 s. When signaled, the mother gently holds the infant’s arms down by the child’s sides for 30 s (epoch two). These procedures are repeated (epochs three and four). In our adaptation, the task ends with a two minute “reunion” episode where the parent and child are instructed to interact as they typically would.

Two modifications to this task were made in order to administer it remotely. First, the parent was asked to place the attractive toy on the child’s tray (in the laboratory, the RA would typically place it there). Second, the parent was allowed to select the attractive toy used in this task (vs. in the laboratory, where the same stimulus would be used).

Stimulus uniformity would be ideal in the remote data collection context as well, however it is logistically difficult and cost prohibitive, as described above.

Cognitive tasks

The cognitive tasks focus on measures of emerging working memory, inhibitory control, and attention, which emerge in primitive form in early life and become the developmental precursors to later mature executive functioning (Diamond, 2013). As was the case for the negative affect tasks, these constructs were selected because they are widely used in developmental research and are linked with numerous important correlates including prenatal exposures (Engel et al., 2010, Graham et al., 2018, Rudolph et al., 2018), brain maturation and functioning (Diamond, 2013, Káldy and Sigala, 2004, Zayas et al., 2014), and future adjustment (Huang-Pollock et al., 2017, Rothbart et al., 1995).

We describe two kinds of cognitive tasks—those used for infants and toddlers and those used for school aged-children, as comparability and correlation across ages is a critical goal of longitudinal and developmental studies.

Object permanence task

This classic Piagetian task (Piaget, 1954) is used as a test of emerging working memory. Though more typically thought of as reflecting the function of medial temporal structures (Káldy and Sigala, 2004), evidence from functional near-infrared spectroscopy (Baird et al., 2002) and EEG (Bell and Fox, 1997) conducted while administering the object permanence task suggest that the frontal cortex is also involved in infant performance.

When this task is administered in the laboratory, the parent is seated at a table with the child sitting on their lap. The experimenter sits on the opposite side of the table, facing the parent-child dyad. On the table are two identical cloths placed next to, but not touching, one another. The experimenter begins by bringing the child’s attention to a small toy in their hand. Once the child has oriented to the toy, the experimenter places it under one of the identical cloths and asks the child where the toy is located. This procedure is often repeated, for example, three times. The task is video recorded such that both child gaze and reaching behavior are captured and can be used to determine if the child correctly identifies the toy’s location.

For the remote administration of this task, the primary modification was that the parent administered the task, with close guidance from the RA. The nature of the toy presentation and the prompts to the child remained the same, however the parent, as opposed to the RA, placed the toy under the cloth and asked the child to locate it. A second modification is that the parent and child were asked to sit next to one another as opposed to the child sitting on the mother’s lap. This was done so that the parent could more easily bring the child’s attention to the toy without blocking the camera’s view of the child. Our study staff made an effort to mail each participant a small rubber duck to use in this task, however there were instances when the duck did not arrive prior to the virtual visit. Thus, stimulus uniformity was an issue here, though arguably the nature of the hidden toy is not critical to this task’s scoring or interpretation.

Snack delay task

In this inhibitory control task (Kochanska et al., 1996) the child and parent are seated next to one another at a table. A small, attractive snack (e.g., M&M candy) is placed on the table in front of the child and a clear plastic cup is placed over the snack. The child is asked to wait for the experimenter to ring a bell before they retrieve the snack. The length of time that the experimenter waits to ring the bell varies by trial (10, 20, and 30 s). The parent is present during the task but they are asked to not interfere with the child’s reaction.

The major modification made for remote assessment is that the parent helps to administer the task. At the beginning of this task, the RA explains the task to both the parent and child using the same script used in the laboratory. The RA then asks the parent to repeat the instructions to the child (this was done to ensure that the videoconferencing medium did not affect infant comprehension). The RA then instructs the parent to place a snack on a surface in front of the child, and to place a plastic cup over the snack. Once the parent places the cup over the snack, the RA begins the trial. At the end of each trial, the RA presses a buzzer, indicating the end of the trial.

Visual attention task

Attention develops rapidly over the first year of life and individual differences in infant attention are linked to later higher-order cognitive skills like language and executive functions (Cuevas and Bell, 2010, Yu et al., 2019). One method to assess attention during infancy is by measuring look duration. Past studies reported that infants classified as “short lookers” were more likely to process information efficiently and also scored higher on IQ measure than infants classified as “long lookers” (Colombo, 1993, Colombo et al., 1991). In this visual attention task, infants are shown a 45-second Sesame Street video clip (Cecile–Up, Down, In, Out, Over, and Under), as described by Blankenship et al. (2019). In the laboratory, video cameras are positioned to capture information about child eye movements as well as look duration and shift rates, metrics of early attention (Rose et al., 2001).

Modifications for the virtual remote environment centered around positioning of the child and ensuring that the video recording of the child’s face could be coded for both look duration and gaze shift. PowerPoint slides were created so that the experimenter could walk the caregiver through the necessary steps to ensure that the video is in full-screen, the experimenter’s video is hidden, and that the audio is loud enough for the infant. Within the videoconferencing software recording, when the experimenter shares their screen, the video of the caregiver-infant dyad is too small to accurately code from. To increase resolution for coding purposes, experimenters expand the caregiver-infant video box and use screen capture recording software like QuickTime or Movavi to separately record the participant’s video box. Next, the caregiver is instructed to have their child seated on their lap, eye-level with the laptop or cellphone. The infant is positioned far enough away so that they can see the entire screen, but close enough for the experimenter to clearly see the infant’s gaze movements. Parents are given a premeasured 17-inch tape to estimate their distance from the screen, and parents also report the size/specifications of the device used. This information is used offline to calculate the visual angle of the screen diagonal to ensure adequate viewing size to detect gaze shifts, which can be used as an offline filter, grouping variable, or covariate in analyses.

Right before the task video is shown, a quick calibration check is conducted where the infant watches a slide containing four brightly colored objects. The experimenter “activates” each object, causing the object to spin and play a sound. The purpose of this calibration is to confirm that shifts in gaze can be seen by the experimenter (up, down, left, and right). If the gaze shifts cannot be clearly seen, the experimenter guides the parent closer to the camera and calibration is repeated. After calibration, the experimenter plays the 45-second video without interruption. Like the in-lab version of this task, caregivers are asked not to interfere or direct their child’s attention during the short video clip.

Screen-based tasks administered to school-aged and adolescent youth

In studies with school-aged children, computer-based measures of cognition are the standard. The computerized tasks are commonly programmed using commercial software such as E-Prime (Psychology Software Tools, Pittsburgh, PA). Tasks are then ordinarily administered using a computer in the laboratory. In our usage, the child is seated at a computer screen and is presented with a keypad that they can use to indicate their responses. The tasks evaluated to date for remote use include the following. We currently are administering these tasks remotely to children ages 16–21 years old, but we have successfully used these tasks in the laboratory for children ages 7 years and older.

The Go-Stop Task

The Go-Stop Task (Logan, 1994, Logan and Cowan, 1984, Verbruggen et al., 2019) is a 15–20 min response-inhibition task. In this task, two targets are presented one at a time in a random order. The participant’s task is to press the correct key using two fingers on their dominant hand as quickly as possible without making mistakes. However, periodically a “stop tone” sounds during which the child is not to respond. The timing of this tone is varied stochastically trial-by-trial to maintain an average 50% success rate at stopping.

The Identical Pairs Continuous Performance Task (IP-CPT)

The CPT has been a workhorse in developmental psychopathology research for decades (Kera et al., 2004, Nigg et al., 2018). We prefer the IP version for older children because it is more difficult than commercial tasks, yielding a higher error rate and thus stronger differentiation of functioning. In this vigilance task, children are presented with a series of four-digit numbers that are displayed one at a time in pseudo-random order. When two identical numbers appear back-to-back, the child is asked to push the response button.

Spatial Span Task

Short term store (sometimes called working memory) is assessed using a modified version of the CANTAB Spatial Span Task (De Luca et al., 2003). In this task, children are presented with a screen containing 10 squares arranged in a fixed position. Individual squares change color in a fixed sequence. Children are instructed to click on the squares in the order in which they change color (forward condition); in the backward condition, they were asked to do so in reverse order. The number of squares in the sequence begin at three and increases to nine. The task discontinues when the child fails all trials at a specific sequence length.

The tapping task

Time cognition is extensively studied in cognitive neuroscience and is also related to developmental psychopathology. The child is asked to tap their finger along with an auditory tone that occurs every 500 ms. The tone then stops and the child is instructed to continue tapping at the same pace. The reaction time for each tap is recorded and reaction time variability is computed using the Kristofferson model (Wing, 2002, Wing and Kristofferson, 1973a, Wing and Kristofferson, 1973b) to computationally isolate time perception from motor speed.

To administer these tasks remotely, E-Prime Go (Psychology Software Tools, Pittsburgh, PA) was used to covert standard E-Prime files to executable files that can be e-mailed directly to the participant to run on their personal computer. Once the program is launched, the tasks and images appear unaltered to the child compared to in the laboratory presentation. One key difference from the in-person experience is that rather than having an external keypad to enter their responses, the child is instructed to use specific keys on their at-home keyboard.

Depending on the size of the files they can either be emailed to the participants as an executable file, or for larger files uploaded into a private and secure shared cloud-based folder. The participant is instructed to download the executable file and are given instructions on how to run the program and complete the tasks using their personal computer. When complete, the participant is instructed to upload the data files (that are automatically generated and saved to their computer’s desktop by E-Prime) to the same cloud-based folder.

The primary obstacles to this system were participant hardware compatibility issues, slow computer response if background processes running on the computer were not stopped, participant comfort operating a computer mouse (which may be confounded with child age), as well as participant ability and patience to access and open the files if they are not familiar with doing so. We recommend that research staff provide participants with detailed instructions relevant to successful task administration, including directing participants to close all other applications before beginning the task. We also recommend that study staff make themselves available via telephone or video conference so that they can guide participants through the process of downloading and running the program. Using screen sharing features of videoconferencing software is useful for this process. Notably, E-Prime Go is not currently compatible with Mac computers or Chromebooks, both of which are popular computer devices. Additionally, E-Prime Go is not compatible with cell phones or tablets, which may further limit participant access. Our preliminary data suggest that 24% of participants who were sent the E-Prime Go tasks were not able to complete the tasks due to hardware incompatibility and other technical problems. Alternative software or data collection platforms that are compatible with a larger number of device types are clearly needed. Until such alternatives are identified, researchers might consider loaning participants a compatible device that is delivered to the participant’s home via mail or via a no-contact drop off.

Caregiver behavior

Understanding the role of caregiver and other family dynamics in moderating in utero effects and in supporting or undermining emerging brain development is critical. The quality of early caregiving is associated with many metrics of infant brain development (Belsky and De Haan, 2011, Posner and Rothbart, 2018) as well as with various dimensions of infant neurobehavior described above (Fay‐Stammbach et al., 2014, Putnam et al., 2002). In designing in-person and remote visits, an effort is made to capture caregiving behaviors in a number of contexts. This includes caregiver behaviors during a free-play paradigm as well as parental responses to child negative and positive emotion, and during a compliance task. Together, these assessments provide a more comprehensive assessment of caregiver behavior than is often captured.

Free-play and compliance tasks

In this task, the parent and child are typically seated on a blanket on the floor and presented with a standard set of age-appropriate toys. They are asked to interact as they typically would if they had some free time. These interactions last a standard amount of time (typically 4–10 min) and are video recorded for later coding for caregiver behaviors and qualities of the dyadic relationship (e.g., Cox and Crnic, 2002; Network, 1999). In our in-laboratory protocol, we opted to begin this task with a five-minute toy-free epoch, where the dyad is seated on the blanket on the floor and given the standard instructions but not provided with toys. After five minutes, the standard set of toys are introduced and the dyad are asked to continue interacting as they typically would. This epoch was added based on observations that some caregivers who appear sensitive in the traditional task struggle when they do not have toys to use when interacting with their infant.

When infants were over 12 months of age, two additions were made. The first is that the dyad are given the set of toys at the beginning of the task (i.e., prior to the toy-free epoch), but the parent is instructed to not allow the child to play with the toys until the toy-free epoch is completed. The second change is that a brief “clean up” period is added at the end of the free-play, where the parent is asked to instruct the child to stop playing and to place all of the toys into a container. These epochs were modeled after the Kochanska “do” and “don’t” paradigms (Kochanska et al., 1998) and are designed to yield information about child compliance and parental strategies for encouraging compliance.

Adaptation

The procedures associated with this task did not require substantial modification; all of the instructions to the parent remain the same between remote and in-laboratory formats. Because the parent-child dyad generally remains seated on the blanket during this task, it is fairly easy to acquire a high-quality recording of the interaction using the videoconferencing format. Much like what is done in the laboratory or at an in-home visit, the parent is provided with context for what we are trying to record prior to beginning the task. For example, parents are told that “we would like to capture at least a profile view of you and your child at all times.” The RA then works with the parent to place the camera in a location and at an angle that allows the camera to capture an adequate view. These methods and coaching techniques have been successful in recording parent-child interactions in several large-scale studies conducted both in the laboratory and in the home (Mills-Koonce et al., 2015, Propper et al., 2008).

Challenges and solutions

The most challenging part of translating these tasks was attempting to standardize the toys used in the free-play task. As described above, the protocol begins with a free-play epoch without toys, which can be administered remotely in a standardized way across all participants. For the second (traditional free-play) epoch in the laboratory or at-home visits, the parent-child dyad are presented with a standard set of toys, which helps to ensure that all parents have equal opportunities to actively try to engage their child with the task. Purchasing and providing all participants with multiple toys is unfortunately cost-prohibitive. As described above, the participants were instead sent a picture of the toys typically used in the laboratory version of the task and asked to assemble 3–4 age-appropriate toys similar to those depicted. Most of the toys used in the laboratory are common (e.g., ring stacker) so it is possible to achieve some stimulus uniformity across participants. During coding, an effort should be made to capture and control for this heterogeneity in stimuli.

Caregiving during affective tasks

As described above, this protocol also allows for the characterization of caregiver behaviors during several tasks that are designed to evoke infant emotions, including the reunion epochs to the still face (described in 2.1.1.1.) and arm restraint (2.1.1.2.), tasks meant to elicit fear/sadness and frustration/anger, respectively. Additionally, caregiver behavior can be characterized during the peek-a-boo epoch that proceeds the still face (2.1.1.1.), which is designed to evoke infant positive emotion.

Biospecimens

The primary consideration for prioritization of the biological samples to collect remotely were their utility for measuring potential prenatal and postnatal moderators or mediators of brain development including exposure to substance use, stress, and inflammation, as well as epigenetic and genetic risk factors that alter neurodevelopment. The ability to collect the sample during critical periods of development, the invasiveness of collection, and participant burden were carefully considered. The feasibility and logistics of remote collection, including cost and ease of specimen collection, processing, storage and transportation across a multi-site study were also examined. This process led to the recommendation of the following biological specimens as feasible to collect remotely from participants (Table 2). The remote sample collection methods including shipping, storage, and possible analytes from each sample type are described.

Table 2

Remote Biospecimen Collection Methodologies.

Biospecimen	Targets of Interest	Participant	Remote Collection Method	Transportation	Processing	Storage	% of Sent Samples Returned Usable
Blood (Plasma/Serum/Erythrocytes, White blood cells)	Nutriture, neurotoxicants, inflammation	Parent, Child (over 12 months of age)	Tasso	4 °C	Centrifuge	-20 − -80 °C	71%
			Mitra	RT/4 °C	RT/4 °C	RT/4 °C
			DBS	RT	None	RT
Urine	Drugs, EtOH, nicotine metabolites	Parent, Child		4 °C	Aliquot	-20 − -80 °C	78%
	Other Environmental Exposures	Parent, Child		4 °C	Aliquot	-20 − -80 °C
Saliva	Genetics & epigenetics	Parent, Child	Oragene	RT	None	RT	66%
	Hormones, Inflammation	Parent, Child	Salimetrics	4 °C	Aliquot	-20 − -80 °C
Finger/toe nails	Drugs, EtOH, nicotine metabolites	Parent, Child		RT	Weigh	RT	65%
Hair	Hormones, Drugs, EtOH, nicotine metabolites	Parent, Child (over 6 months of age)		RT	Weigh	RT, light-protected	55%
Stool	MicrobiomeMetabolome	Parent, Child	Zymo Research DNA RNA collection kitGenotech	4 °C or RTRT4 °C	None	-80 °C-80 °C-80 °C	57%
Breastmilk	Inflammation, Nutriture, Drugs, Microbiome	Parent		4 °C	Aliquot	-20 − -80 °C	69%
Deciduous Teeth	Nutriture & Neurotoxicants	Child (over 6 years of age)		RT	None	RT	–

*DBS, dried blood spot; RT, room temperature; EtOH, ethanol; if not explicitly noted the biospecimen can be collected from the child at any age. The percentage of useable samples returned is reported for samples that were sent to participants from OHSU and NYU at the time of this report (n = 436), with the exception of urine, which was only reported for OHSU (n = 118 prenatal biospecimen kits). Notably, 76% of the participants (n=436) returned at least one sample. The samples described in the table met our initial quality criteria, which included arriving intact, with a sufficient volume, and in an insulated shipper that was still cold. Of note, few samples that were returned to our laboratories violated these quality criteria. Rather, most of the missing samples were ones where the participant did not return the biospecimen collection kit, or where a particular sample type was not relevant to them (e.g., not all women were still lactating at the time of the postnatal sample collection, and thus did not provide a breastmilk sample).

The typical workflow for remote biological sample collection is to identify interested participants, provide study information, obtain remote informed consent, verify their current address and ship the sample collection materials to the participants. In addition to the sample collection kit, the participants will be provided with orientational videos, detailed instructions, and the opportunity to meet with a study RA over video conference to go over the sample collection plan and address any remaining questions. The sample collection kit includes a return-shipper, ice packs to allow certain samples to be maintained at 4 °C during shipping (Table 2) and a return shipping label. As the shipping of biological samples must follow institutional and federal guidelines such as ensuring the containers containing liquid samples are taped closed and enough absorbent material is enclosed to absorb the liquid contents should a leak occur, tape strips and absorbent material are included in the collection kit. Once the package is shipped by the participant, the study staff receives an alert and a tracking number and processes and stores the samples upon arrival. Overnight shipping is recommended for sample return if samples/analytes are not room temperature stable (see Table 2).

Blood

Blood and its components are one of the most common biological sample types collected in research studies. Plasma and serum have the potential to inform about the circulating levels of a diverse range of signaling molecules including hormones (i.e. glucocorticoids), levels of nutrients (i.e. glucose and fatty acids), and immune status (i.e. cytokines). The isolation of white blood cells allows the extraction of DNA and isolation of red blood cells allows examination of the fatty acid composition of their plasma membranes to inform about participants nutrition over the past three months. Whole blood is important in some applications, for example to measure environmental toxicant exposures. Traditionally blood is collected in the clinic and laboratory via venipuncture. However, during the pandemic, modified operations and safety concerns reduced the feasibility of in-person venipuncture.

In order to allow the remote collection of blood, innovative methods such as the Mitra Blood Collection Kit by Neoteryx (Torrance, CA, https://www.neoteryx.com/) and Tasso SST OnDemand devices (Tasso, Inc., Seattle, WA, https://www.tassoinc.com/tasso-sst) facilitate convenient and accurate remote sampling of capillary blood from participants (Fig. 2). Collection of peripheral blood via finger prick is not new. Dried blood spot microsampling is a well-established, convenient and inexpensive method in which capillary blood is collected on filter paper. However, variation in hematocrit and blood volumes between individuals and sample collection can influence the concentration of the analyte (Alsous et al., 2020) making this method not ideal for the quantification of the concentration of a molecule in circulation.

Fig. 2

Blood Microsampling Techniques. Innovative methods for remote blood collection such as the Mitra Blood Collection Kit by Neoteryx (Torrance, CA, https://www.neoteryx.com/) (panels A-D) and Tasso SST OnDemand devices (Tasso, Inc., Seattle, WA, https://www.tassoinc.com/tasso-sst) (panels E-H) facilitate sampling of capillary blood from participants. (A) The Mitra™ volumetric absorbent microsampler is a polymer-based cartridge which absorbs a controlled volume of blood (up to 30 uls) of capillary blood from a finger-prick via a lancet. (B-C) The tip of the cartridge is dipped into the finger‐prick blood spot, and blood is adsorbed into the tip in a few seconds. (D) The tips are then air‐dried at room temperature. (E) The participant applies the Tasso SST OnDemand devices to their upper arm using a light adhesive. (F) The participant then presses a button that causes a vacuum to form and a lancet to prick the surface of the skin. (G) The vacuum draws blood out of the capillaries and into the sample collection tube attached to the bottom of the device. (H) The sample can then be centrifuged to obtain serum.

The Mitra™ volumetric absorbent microsampler (VAMS; Fig. 2A-D) is a polymer-based cartridge which absorbs a controlled volume (up to 30 uls) of capillary blood from a finger-prick via a lancet. As the volume is consistent across samples, this method allows the accurate quantification of analyte concentration. The cartridge tip is dipped into the blood drop, and blood is adsorbed. The tips are then air‐dried for a minimum of two hours. The cartridges are stable at room temperature and should be stored in the provided foil packet with desiccant. For longer-term storage a − 80 °C freezer is recommended and the desiccant will need to be monitored and replaced every few months. The analyte of interest can then be extracted from the inert polymer cartridge and subsequently assayed for a wide range of analytes.

The Tasso SST OnDemand devices (Fig. 2E-H; Tasso Inc., Seattle, WA) are able to collect 300 µLs of capillary blood. The Tasso-M20 allows the option of collecting 4 dried blood samples of the same volume. To collect blood, the participant applies the device to their upper arm using a light adhesive then presses a button causing a vacuum to form while a lancet pricks the surface of the skin. The vacuum draws blood out of the capillaries and into the sample collection tube attached to the bottom of the device. The sample can then be centrifuged to obtain serum. The centrifugation (2000 g at 4 ⁰C for 10 min) can happen in the laboratory after the blood is shipped overnight at 4 ⁰C. For sensitive analytes such as certain cytokines that are released from the red blood cells into the serum it is recommended that investigators ship the participants an inexpensive centrifuge and request that they spin the sample prior to shipping. After processing and receipt, the serum is stored at − 80 °C and the analyte of interest can be assayed. The advantage of this method is that there is no need for extraction of the analyte from filter paper or the polymer tip as many assays and multiplex systems are optimized for serum.

In addition to allowing remote blood collection, microsampling techniques overcome some of the challenges of obtaining blood in infants and young children, as the blood can be obtained without venipuncture and collected in the child’s home. For the remote collection of the child’s blood the parent is asked to collect capillary blood either by applying a Tasso SST OnDemand device on the child’s lower back or from a finger prick via a lancet using a Mitra device. We do not recommend asking a parent to collect blood from a child that under 1 year of age.

Hair

Hair provides both retrospective information regarding physiologic status including chronic stress response (Iob and Steptoe, 2019) pubertal status (Uban et al., 2018), nutrition status (Sulek et al., 2014), and exposure to environmental factors such as drugs (Al-Delaimy, 2002) and heavy metals (Santos Serrao de Castro and de Oliveira Lima, 2018). The popularity of hair as a biospecimen is in large part because it is easy and noninvasive to collect and stable at room temperature as long as it is stored protected from light. Limitations include the infants not having grown enough hair for collection or adults or children with hair that is too short. Also, dyeing or bleaching of hair can alter the concentration of analytes such as cortisol (Russell et al., 2012).

Sample collection involves the collection of up to 60 strands of hair, ~3 mm, from the base of the participant’s neck, close to the scalp. For this to be collected remotely we provide a reference video and detailed instructions with pictures and suggest that the participant seeks the help of another adult household member. The participant is then requested to tie the hair strands with a provided string and place the hair in the provided foil packet with the scalp side oriented towards the side of the foil packet labelled “scalp”. The participant then ships the sample at room temperature with the other biospecimens. When the sample arrives the weight of the sample is recorded. This information is necessary when the analyte of interest is extracted from the hair sample. The sample is then returned to the foil packet and stored at room temperature until time of analysis. The most common method of extracting analytes from hair samples includes finely cutting the hair with scissors or milling with beads followed by alternating incubations with methanol and acetone (Slominski et al., 2015).

Saliva

Saliva is a valuable biospecimen that can provide information about hormone levels (i.e. cortisol (Hibel and Mercado, 2019)), RNA and DNA (Rushing et al., 2020), and substance use (Kwok et al., 2014). In addition to supplying useful biomarkers for research, it is non-invasive and relatively stable making it ideal for remote collection.

For remote sample collection, participants are asked to collect saliva by three methods: passive drool into an aliquot tube, passive drool into a Oragene-DNA (OG-500) (DNA Genotek, Ottawa, Ontario, Canada, https://dnagenotek.com) and swabbing buccal areas of their child’s mouth. For non-DNA passive drool collection, participants fill the provided aliquot tube with saliva. For passive drool collection into the Oragene DNA tube, participants spit into the funnel cap that feeds into the attached plastic tube until the marked fill line. The Oragene DNA tube contains stabilizers allowing the sample to be kept at room temperature for later DNA extraction. To collect a saliva sample from children, parents are instructed to use either an infant (SalivaBio Infant Swab Device (5001.08)) for infants less than 6 months of age or child swab (SalivaBio Children's Swab Device (5001.06) for children 6 months to 6 years of age. The parent is instructed to insert the swab into their child’s mouth and collect pooling saliva from the corners of their mouth or beneath their tongue. After approximately 90 s of swabbing, the parent places the swab saliva-side down into the aliquot tube.

Once the participant completes saliva sample collection, they are asked to ship the samples overnight with ice packs in their insulated shipper to ensure the temperature of their saliva samples remain cold. When received by the lab, the shelf-stable Oragene DNA saliva sample tube is kept at room temperature in a locked light-protected cabinet for long-term storage. The parental non-DNA passive drool sample is aliquoted and frozen at − 80 ⁰C and the child saliva swab is frozen in its tube at − 80 ⁰C for long-term storage.

Nails

Finger and toe nails are shelf-stable and can provide information about environmental toxins and trace elements in a non-invasive manner (Slotnick and Nriagu, 2006). Being easy to collect and process lends nails well to remote collection.

For remote collection the participant is instructed to collect finger or toe nails from themselves as well as their child. Nails are to be cleaned then clipped and placed into a small aliquot tube. This tube can be stored at room temperature prior to returning to the lab. The specimens are returned to the laboratory with the other biospecimens or shipped back in a padded envelope. When nail samples are received by the lab, the aliquot tube of clippings is placed in a light-proof locked cabinet at room temperature for long-term storage.

Urine

Urine can yield a great deal of physiologic information including levels of hormones, metabolites, exposures to environmental variables such as trace metals and substance use.

For remote collection, adult and toilet trained child participants are asked to urinate directly into the collection cup first thing in the morning, to account for diurnal variations in analytes. For non-toilet trained children, absorbent pads are placed in the child’s diaper. Once removed from the diaper, the pads are placed into a large bore syringe and urine is extracted by depressing the plunger. After the sample is collected, it can be stored in the participant’s refrigerator until it is returned. Participants are asked to include ice packs when returning their urine sample to ensure it stays at 4⁰C during shipping. When the sample is received, processing involves measuring the specific gravity of the urine as this influences the level of many of the analytes. The urine is then aliquoted into microtubes and stored at − 20 ⁰C. To avoid the contamination due to leaching of chemicals from the plastic microtubes, it is recommended that at least one aliquot is stored in a glass microtube, particularly if measuring the concentration of phthalates and plastics.

Breastmilk

Breastmilk provides information about both the lactating parent and their infant. Nutritional status and metabolic state can be assessed via analysis of the lipid profile, signaling molecules and growth factors present in the breastmilk. Information about the maternal microbiome (National Academies of Sciences, 2017) and immune system (Toscano et al., 2017) and environmental exposure to factors such as lead (Ettinger et al., 2014) can also be gleaned.

Remote collection of breast milk is straight-forward. Participants are requested to collect approximately 40 mL of breastmilk using the sterile hand-held silicone breast pumps that is provided, or their own lactation devices. To control for the feeding state of the parent and infant, it is recommended that the sample is collected in the morning prior to the parent eating and the infant being breastfed. Participants are provided a sterile collection cup and requested to store their sample in their refrigerator until it is returned to the lab. As most analysis of the breast milk samples require that the sample is kept at 4 ⁰C, the breast milk samples are shipped overnight at 4 ⁰C, aliquoted into microtubes and stored at − 80 ⁰C.

Stool

The collection of maternal and infant stool allows experimenters to examine the metabolome and microbiome of the mother and infant. Recent evidence indicates that environmental factors such as diet and obesity (David et al., 2014, Davis, 2016, Ley, 2010, McDonald et al., 2018) and early life adversity and stress (Dong and Gupta, 2019) modify the gut microbiome community membership and function. Also, the maternal microbiome “programs” the infant microbiome (Jasarevic et al., 2015, Milani et al., 2017, Rautava et al., 2012, Walker et al., 2017). Alterations in the microbiome are associated with maternal and child depression and anxiety (Sanders et al., 2019). Thus, recent studies are examining the microbiome and metabolome as a potential mechanism for associations between the early environmental factors and alterations in offspring brain development trajectory and risk for psychopathology (Forssberg, 2019, Lu and Claud, 2019).

Fecal samples can be readily collected from the parent and their infant remotely. Sample collection kits for adults include fecal collection tubes, self-collection feces catcher, specimen information form including the Bristol Stool card (allows participants to rate their stool consistency), detailed collection instructions gloves, biohazard bags, absorbent pads, and gel packs. The collection of stool/meconium from infants is similar, but specimens are collected from diapers until the child is toilet trained. It is recommended that investigators provide diapers to decrease variation in material that could affect sequencing results. Several types of fecal collection tubes have stabilizers optimized for the collection of microbial DNA and RNA that allow the samples to be shipped a room temperature. For example, for the collection of microbial DNA, the OMNIgene®•GUT (DNA Genotek, Ottawa, Ontario, Canada, https://dnagenotek.com) is stable at room temperature for 8 weeks. The DNA/RNA Shield Fecal Collection Tube -DX (Zymo Research Irvine, CA, https://www.zymoresearch.com) allows collection of both microbial RNA and DNA and is stable at room temperature for shipping. Samples for metabolomics analysis should be collected in a tube without stabilizer, but require overnight shipping at 4 °C. It is recommended that all collection tubes be stored at − 80 °C upon receipt. Fecal samples can be analyzed using an array of techniques. Nucleotide sequencing of RNA and DNA including amplicon sequencing, metagenome sequencing, and metatranscriptome sequencing assess host and microbial gene expression. Metabolomic and metaproteomic techniques utilize mass spectrometry and nuclear magnetic resonance spectrometry to profile secreted and intracellular microbial products and metabolites, including fatty acids, vitamins, bile salts, and polyphenols (National Academies of Sciences, 2017).

Deciduous tooth

Deciduous or “baby” teeth are a unique biospecimen as they provide retrospective information about a child’s exposure starting during fetal development. Natural tooth shedding typically starts at age 6 and continues until the child is 12 years of age. Exposure to factors such as neurotoxicants and substances of abuse known to alter brain development and influence risk for psychopathology can be determined from deciduous teeth (Needleman et al., 1979). Toxicants level in the whole tooth measures cumulative lifetime exposure to the chemical (Needleman et al., 1979, Sitarik et al., 2020). New methodology combining laser ablation and micro-dissection allows the objective determination of the dose and timing of environmental exposure during gestation and early childhood (Arora and Austin, 2013).

The parents of participants are provided with protective vials for each tooth and a padded envelope to prevent damage and are requested to mail each shed tooth to the laboratory. The parents are asked to describe how each tooth was lost and how it was stored. Like hair and nails, advantages of baby teeth as a biomarker include their non-invasive collection and that they can be shipped and stored at room temperature.

Water samples

Household water samples can be used to measure a number of environmental exposures, including biological contaminants, metals, organic and non-organic compounds, and chemicals. Of particular relevance to studies of infant brain development may be heavy metals (e.g., lead), fluoride, and phthalates, given their known associations with infant brain development (Green et al., 2019, Hu et al., 2006; Zhang et al., 2019).

While some analytes require that water samples be preserved (see the United States Environmental Protection Agency’s Guide to Drinking Water Sample Collection (2016) for one set of guidelines), many are stable and can be shipped and stored at room temperature in a light-protected container. It is recommended that a sample collection container that does not leach plastic be used, particularly if measuring the concentration of phthalates and plastics in water. Samples should be collected from the home’s primary cold-water faucet.

For studies interested in assessing heavy metals, water samples should be collected first thing in the morning prior to the water being used. The participant should be instructed to not use this faucet for at least 6–8 h prior to the sample collection to ensure that the water has been sitting undisturbed in the pipes for a sufficient duration of time. If the faucet has a screen or aeration device attached to it, these should be left in place. If feasible, a mid-day water sample can provide complementary information about metal concentrations in water. Other sample types may be best measured using samples collected after flushing the tap for a specified period of time and/or with screen or aeration devices removed.

Dissemination of methods across HBCD sites

A novel aspect of the COVID-19 pandemic was that this health crisis and responses to it were somewhat uniform across the United States. There was notable local variation (White and Hebert-Dufresne, 2020), but overall, the timing and circumstances of this crisis had similar effects on neurodevelopmental researchers across the country. This is in contrast to past events, such as Superstorm Sandy (Nomura et al., 2019; Zhang et al., 2018), the Dutch Hunger Winter (Painter et al., 2005, Roseboom et al., 2006), and natural disasters such as the Quebec ice storms (King and Laplante, 2005, Laplante et al., 2016) that impacted only select populations at the time of their occurrence. As a result, it was logical to consider our adaptations in the context of ongoing national collaborations underway at the time of this major event. As members of the HBCD linked Phase 1 research sites, we sought to share methods and prospectively harmonize the move to remote collection, which required specific additional cross-site communication and training strategies be developed.

The HBCD Study will be a longitudinal, nation-wide study, sponsored by the National Institutes of Health. The major aims of this initiative is to characterize normative infant brain and behavioral development, and early life factors that influence growth and development, spanning birth to middle childhood. The study will include a representative cohort of pregnant women recruited from the general population to examine normative brain development as well as a subset that includes pregnant women whose infants were exposed pre- or perinatally to prescription and illicit opioids, marijuana, stimulants, alcohol and tobacco/nicotine; as well as women from comparable high risk environments, who did not use substances during pregnancy (Volkow et al., 2020).

A strength of the coordinated shift to remote neurodevelopmental and biological assessment was opportunity to (a) increase sample diversity and representation by gathering data at a national level, (b) increase the overall sample size and foster opportunity for cross-site data sharing, and (c) set precedent for coordination efforts critical to Phase 2 of this major scientific initiative. Eight phase-one sites were awarded supplemental funding under the PA-18–935 “Urgent Competitive Revision to Existing NIH Grants and Cooperative Agreements” to carry out this work. Collectively referred to as the COVID-19 and Perinatal Experiences (COPE) study, and further harmonized with activities happening in parallel as part of the COVID-19 GENeration Alliance, www.covgen.org, these sites developed a core research protocol, encompassing many of the remote assessments described above. The overall cross-site sample size was 800 participants, with 320 of these participating in the full behavioral and biological assessment protocol, including multiple points of longitudinal assessment.

In support of coordination, specific efforts were made with regard to communication and documentation. In addition, formal evaluation of potential relevance of site-specific constraints on overall study rigor and reproducibility was required. Altogether the primary adaptations made centered on maintaining scientific rigor, minimizing investigator burden, and setting standards.

Scientific rigor

Sharing materials and implementing cross-site training were critical in assuring uniform data collection across sites. Specifically, questionnaires, electronic data capture structures, participant communication materials, and stimuli were shared. Weekly RA training meetings were held, each focusing on different aspects of study implementation. These were also opportunities to identify potential barriers to implementation and troubleshoot as a group. Lead investigators also held bi-weekly cross-site meetings where more administrative topics and larger scope objectives were discussed and research findings presented.

Minimizing burden

Coordination has the advantage of reducing redundancy of efforts, for example development of stimuli or instruction. However, there is a cost in time spent on dissemination and harmonization. This included needing to remain up to date on changes occurring locally at sites, such as staffing changes that impact the chain of communication. This also included development and maintenance of centralized materials, and effort to take additional steps to assure future cross-site data sharing would be possible, such as common naming and coding of variables. The approach taken to ease this burden was to rely on the Open Science Framework (OSF) and Google docs and train sites in how to maintain coordination. That is, rather than individually storing or circulating site specific information or materials, a repository for that information was established and the responsibility to update and understand use fell to the sites. The OSF page included folders with study materials, information about timing and location of weekly/bi-weekly virtual meetings, and links to editable Google docs. Google docs served to summarize study characteristics at each site, for example primary source(s) of recruitment and additional measurements planned, and also maintained email dissemination lists for key personnel at each site. In this way sites were able to update these common sources as changes were made over time and these were easily referenced by all without need for added communication. This also assured that the most current information was available at all times. In addition, separate data capture projects and IRB protocols were maintained. This was necessary as this common COPE research protocol was added to ongoing studies at many sites, and thus there was deviation in wrap around assessments and study populations that precluded centralization administrative functions from the outset.

Setting standards

Investigators across sites sought to balance individual objectives with common objectives. Processes were established for both proposing common research products and for data sharing. In brief, sites were encouraged to generate written proposals when proposing a new shared research product. These were circulated to the group with two parts, an abstract, and which data variables would be requested. An authorship standard was established wherein the primary site would decide on authors named at the start and end of the author order, and that all authors in-between would be ordered alphabetically by last name. Further guidelines were that contributing sites would suggest approximately two authors for each work, and these individuals would take a lead position on review and coordination of that specific work. The choice of authors could rotate across products in recognition that large teams operated to support the work at each site. Deviation from these standards was permitted and discussed, but the establishment of standards set a framework in support of these conversations. Further, it was also overtly stated that (a) agreed upon standards for authorship would be followed, and that (b) there was not a requirement that all sites either be invited to all proposed studies or be obligated to participate. The understanding reached was that scientific freedoms were regarded as valuable to preserve.

Future considerations

Though these remote data collection methods have already proved to offer advantages over in-laboratory assessments and have been essential for building nation-wide collaborations that will be critical to the success of Phase 2 of the HBCD Study, there are a number of relevant issues that still need to be addressed.

Despite standardization of protocols and staff trainings within and across sites, there remain participant-level issues with standardization for some tasks. For example, it may not be possible to achieve complete standardization of stimuli (e.g., participants may use different toys during the free-play) and other task-supportive materials (e.g., a high chair is not always available). Because many of the tasks are intended to mimic real-life situations, in practice we find that participants typically have items that aid in task administration in their home. However, not all participants have the same items or resources, and this is likely to be confounded with factors such as socioeconomic status or the size of their dwelling (which may be linked to geography or rurality). Our team has developed several task modifications to standardize these assessments (e.g., a toy-free free-play) and specific instructions for staff regarding modifying each task. Staff also utilize standardized scripts that include the same parental instructions that we use in the laboratory where it is emphasized that we would like them to not interfere with their child’s task performance. Staff are also instructed to take detailed notes when collecting and coding data, and these factors will be considered during analyses.

There are also issues relating to video and audio quality and the uniformity of the camera angles captured. Depending on the recording device used and the space in which the visit is occurring, it can be challenging to capture certain camera angles. As described above, our staff are trained on the features of the video to prioritize if the ideal camera angle cannot be achieved, improving standardization. They are also trained to accommodate different recording devices, for example by suggesting that a cell phone be propped up against a book or other stable item that can be moved in order to capture the best angle possible. Similarly, there are individual differences in participants’ ability to complete the task in a private, quiet space (particularly in urban areas, where dwellings tend to be smaller), which may impact audio quality or bias participant behavior. Another notable concern is that not all participants have the same access to internet or internet-enabled devices, many low- and moderate-income families are “under-connected” (meaning that they have mobile-only access), and there is large variability in internet bandwidth and speed (Lourenco and Tasimi, 2020, Rideout and Katz, 2016), which can affect video recording. Though any internet-enabled device should be compatible with the videoconferencing software, it is unrealistic to think that all participants will have such resources. Contrary to expectation, we have not yet encountered a participant who has not had some internet access and at least one device that can connect to the internet, however this will obviously not be the case for all populations. There are a number of ways to address this including mailing participants an internet-enabled device or conducting a no-contact drop off of a cell phone, laptop, or tablet that has a cellular plan or that is connected to a hotspot. For some tasks (e.g., the free play) it may be possible for the participant to videorecord the interaction themselves and then to e-mail or upload the video to research staff when they have internet access.

To increase and maintain high quality videos of our neurobehavioral assessments, we developed a rubric for rating video quality. Within this rubric, each video recorded assessment is coded on a scale from “1” (unusable) to “4” (excellent) based on key criteria for behavioral coding. For example, in the visual attention task a score of “4” would indicate that (1) the video shows a close view of the infant’s face; (2) the infant’s eyes are clearly visible during the calibration check; (3) no disruptions occurred within the home context or video quality; and (4) the mother did not interfere in the task. RAs use this rubric to rate the quality of the video data after each visit. Doing so allows experimenters to quickly course correct if needed in any of their instructions to the participant or camera angle positioning prior to the next remote visit. Together, these metrics are intended to provide regular, real-time feedback to our data collectors. Preliminary analysis of these data quality metrics from our two sites (OHSU and NYU) (n = 348) suggest that 94–99% of videos passed quality control (rates vary by individual task), and that we can achieve high inter-rater reliability using these videos (e.g., for the infant negative affect that have been coded at the time of this report, all kappas >0.80).

There are also several noteworthy issues regarding remote biospecimen collection. First, although the participants are provided detailed written and video instructions, they may not follow the procedures as described. This could lead to increased variability in sample quality, insufficient sample volume and variability in the timing of sample collection. The shipping of the collected samples by the participant may also be challenging. Some participants may not feel comfortable dropping the shipper off at a shipping facility. In this case, it may be advantageous for the investigators to offer no contact pick up of the samples. Although participants are instructed to freeze gel packs ahead of packaging the samples, it is possible that the samples will be exposed to varying temperatures during transport, or that the shipping carrier will be delayed in delivering the samples. It is also possible that the participants will not return the samples. To reduce noncompliance, it is critical to establish a rapport with participants and closely track and follow up with participants who have not yet returned the samples. These methods have allowed us to achieve a biospecimen sample kit return rate of 76% at OHSU and NYU. Although many of the remote samples are collected in a similar fashion to in-lab collection, blood is collected using microsampling of capillary blood instead of venipuncture, thus validation studies need to be conducted for each analyte of interest to determine if differences in concentration exist between capillary and venous blood and pave the way for widespread use of microsampling techniques.

One of the purported benefits of using remote data collection methods is that they may increase the recruitment and retention of underrepresented populations in longitudinal developmental research. The method holds promise for expanding the available participant pool to a greater geographic area and increasing access to underrepresented communities in contexts where travel to the laboratory is a significant barrier. Future studies must devote significant effort to prioritizing the recruitment and retention of such populations for this to be successful.

Another important future direction will be formal validation studies that confirm the reliability and validity of these remote behavioral and biological data collection methods. Though preliminary data surrounding quality control measures (e.g., the video quality rubric described above) and data from our coding teams (who established reliability on videos collected in the laboratory, and were able to directly translate their coding schemes to the remote visits while maintaining high levels of reliability) are encouraging, formal validation studies are needed.

Despite these issues, the remote data collection methods described herein have great potential to push the field of developmental psychology forward, and in particular to increase the feasibility and success of large-scale multi-site studies of infant brain and neurobehavioral development. In this report, we highlight the ways that our work provides a strong foundation for Phase 2 of the HBCD Study. However, it is important to note that these methods have much broader applicability and greater potential than can be realized in a single study.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.