Principles for Guiding the Selection of Early Childhood Neurodevelopmental Risk and Resilience Measures: HEALthy Brain and Child Development Study as an Exemplar.

The vast individual differences in the developmental origins of risk and resilience pathways combined with sophisticated capabilities of big data science increasingly point to the imperative of large, neurodevelopmental consortia to capture population heterogeneity and key variations in developmental trajectories. At the same time, such large-scale population-based designs involving multiple independent sites also must weigh competing demands. For example, the need for efficient, scalable assessment strategies must be balanced with the need for nuanced, developmentally sensitive phenotyping optimized for linkage to neural mechanisms and specification of common and distinct exposure pathways. Standardized epidemiologic batteries designed for this purpose such as PhenX (consensus measures for Phenotypes and eXposures) and the National Institutes of Health (NIH) Toolbox provide excellent "off the shelf" assessment tools that are well-validated and enable cross-study comparability. However, these standardized toolkits can also constrain ability to leverage advances in neurodevelopmental measurement over time, at times disproportionately advantaging established measures. In addition, individual consortia often expend exhaustive effort "reinventing the wheel," which is inefficient and fails to fully maximize potential synergies with other like initiatives. To address these issues, this paper lays forth an early childhood neurodevelopmental assessment strategy, guided by a set of principles synthesizing developmental and pragmatic considerations generated by the Neurodevelopmental Workgroup of the HEALthy Brain and Child Development (HBCD) Planning Consortium. These principles emphasize characterization of both risk- and resilience-promoting processes. Specific measurement recommendations to HBCD are provided to illustrate application. However, principles are intended as a guiding framework to transcend any particular initiative as a broad neurodevelopmentally informed, early childhood assessment strategy for large-scale consortia science. © Springer Nature Switzerland AG 2020.

The rapid growth of neurodevelopmental science over the past decade provides unprecedented opportunity for nuanced characterization of the developmental origins of risk and resilience pathways beginning even before birth. At the same, major advances in big data science provide the tools and power to detect brain:behavior patterns with precision and to detect population heterogeneity and sub-group patterns. Together these lay the groundwork for high impact neurodevelopmental consortia science. However, this opportunity also brings with it the challenge of generating an assessment strategy that leverages the state-of-the-science from multiple convergent fields of salience to neurodevelopment, while at the same time optimizing pragmatic features to maximize precision while minimizing burden. The objective of this paper is to provide a set of principles to provide strategic guidance within this neurodevelopmental assessment selection process. Their application within the HEALthy Brain and Child Development (HBCD) planning phase is presented as exemplar.

Developmental Origins of Risk and Resilience

Pre-perinatal adverse exposures shape neurodevelopmental risk and resilience trajectories (McEwen, 2003; Shonkoff & Garner, 2012). In particular, there is robust evidence that pre-perinatal exposure to substances (e.g., opioids, smoking, marijuana, alcohol) and psychosocial stress are linked to alterations in the developing brain and associated regulatory systems engendering neurodevelopmental vulnerability to mental health and other corollary adaptational problems across the lifespan (Clark, Espy, & Wakschlag, 2016; Conradt, Crowell, & Lester, 2018; Ernst, Moolchan, & Robinson, 2001; May et al., 2018; Mayes, 1999; McLaughlin, Sheridan, & Lambert, 2014; Wakschlag et al., 2020). Thus, early life exposures and their neurodevelopmental sequelae lay the foundation for adaptive or maladaptive pathways across the lifecourse. A large population-based national study sufficiently powered to account for population heterogeneity (in exposure and outcomes) across regions, and sociodemographic strata will provide novel insights on individual differences in risk and resilience trajectories, including protective processes that can be strategically leveraged for prevention.

HEALthy Brain and Child Development (HBCD) Study

The HEALthy Brain and Child Development (HBCD) Study will be a longitudinal, nation-wide study, funded by the National Institutes of Health (NIH). It will characterize normative infant brain and behavioral development, and early life factors that influence growth and development and deviations from birth to middle childhood. The study will include a representative cohort to examine normative brain development, oversampling for exposure to opioids and other substances, and other chronic stressors with intent to elucidate developmental origins of health disparities. This multi-site study will recruit and assess mothers prenatally, and regular assessments are planned throughout childhood. Importantly, the HBCD study will be the first and largest longitudinal consortium explicitly designed to prospectively examine the effects of early life stress and Adverse Childhood Experiences (ACEs), as well as protective factors and resilience processes beginning during pregnancy, that impact structural and functional brain development and outcomes in a diverse sample of children. Moreover, data from the HBCD study will be de-identified and available to the broader research community.

Scores of studies delineate the effects of early adversity on brain development (Nelson & Gabard-Durnam, 2020; Teicher, Samson, Anderson, & Ohashi, 2016), mental and physical health, and adult functioning (see Hays-Grudo & Morris, 2020 for a review). Thus, studies like HBCD will build on this robust science base to provide an integrated platform that will inform both science, intervention, and prevention efforts to help ameliorate and understand the effects of early adverse exposures and pathways to resilience. HBCD is poised to launch at a unique time in history, with economic and health disparities becoming clearly evident, and it will benefit from lessons learned from other large national neurodevelopmental studies. It is imperative that the study is designed with careful planning and strategic decision-making.

With this in mind, NIH funded a Phase I planning for HBCD that is currently underway at 28 sites across the USA (see https://heal.nih.gov/research/infants-and-children/healthy-brain). The authors of this paper are part of the HBCD Neurodevelopmental Workgroup that included experts from planning grant study sites. Our mandate was to recommend a neurodevelopmental assessment protocol starting at birth through age 5 years, generating a framework to guide the assessment strategy for the larger Phase II study which is planned to begin in the fall of 2021 (https://grants.nih.gov/grants/guide/notice-files/NOT-DA-20-069.html). Critically, HBCD is designed for alignment with the ongoing Adolescent Brain Cognitive Development (ABCD) study, a landmark investigation funded by NIH that includes almost 12,000 youth at 21 sites across the USA. ABCD recruited 9- and 10-year-olds between 2016 and 2018 with plans to follow participants into early adulthood (Volkow et al., 2018; https://abcdstudy.org/). This alignment will allow for an in-depth examination of risk and resilience trajectories.

The planned HBCD-ABCD bridging design provides a unique opportunity by building in “from the get-go” planned linkage from exposure to vulnerability to (mal)adaptation from pregnancy through the transition to adulthood. This is poised to elucidate explanatory pathways that drive the vast diversity of outcomes in developmental trajectories to health and disease. In addition to examining normal brain development, the HBCD design is structured to provide exquisite characterization of the developmental precursor pathways that determine the probability that exposed children will develop the maladaptive substance abuse and mental disorders, and their neural correlates, and co-occurring functional impairment or adaptive outcomes, which are deeply phenotyped in ABCD.

A Moment in Time of Great Neurodevelopmental Importance

The powerful influence of these early exposures has never been more salient than at the present time of unparalleled crisis. The convergence of the escalating opioid epidemic, the COVID-19 pandemic, and systemic and institutional discrimination and violence together represent an acute case of cumulative risk. Their ensuing stress and devastation may permeate virtually all aspects of health, family life, and functioning and further increase disparities, with long-lasting reverberations for decades to come (Dongarwar et al., 2020; Nelson et al., 2020; Wade, Prime, & Browne, 2020). While this will likely have a detrimental neurodevelopmental impact on many children, others will demonstrate resilience and adaptive outcomes (Cicchetti & Curtis, 2007; Masten, 2011; Masten & Motti-Stefanidi, 2020).

The guiding principles and recommendations presented in this paper are the result of extensive discussions and literature review by a transdisciplinary group of scientists reflected in the authorship of this paper. Principles reflect guideposts for the intricate balancing act required in selecting a feasible, scalable, maximally sensitive and minimally burdensome assessment strategy requisite for a complex longitudinal cohort study. It is important to note that the principles in the paper are not independent of one another, and overlap somewhat conceptually and practically. When refining the principles, we weighted considerations of pragmatics, developmental measurement, teratology, continuity with ABCD and leveraging strategies used by other neurodevelopmental consortia such as Environmental influences on Child Health Outcomes (ECHO), and taking advantage of extant national measurement systems (not reinventing the wheel) throughout the process (e.g., PhenX, consensus measures for Phenotypes and eXposures). For heuristic and practical decision-making purposes, the focus of the workgroup was “narrowly” honed on neurodevelopmental assessment recommendations. However, we underscore that nuanced characterization of risk and resilience patterns cannot possibly be achieved without in-depth assessment of broader family and ecological and historical context. Of special salience to resilience processes are moderating influences of promotive processes extrinsic to the child, such as responsive parenting, and interconnected family and community networks (Bush et al., 2020; Masten & Motti-Stefanidi, 2020; McLaughlin, 2016; Wakschlag et al., 2011; Waller et al., 2015). The importance of capturing these domains has been central to the HBCD planning process, has been covered by other HBCD Planning Phase Workgroups, and is beyond the scope of this paper. A cross-cutting consideration was also ensuring socio-cultural validity of measures selected. Although cultural considerations require much more in-depth discussion than this paper allows (NIH- Office of Behavioral and Social Sciences Research, 2001), whenever possible, we recommended measures/tasks validated across socioeconomically and socioculturally diverse young child populations and that have been validated in languages other than English (particularly Spanish).

HBCD Guiding Principles for Neurodevelopmental Assessment Strategies

Principle 1: Balance Considerations of Developmental Sensitivity with Lifespan Coherence

The construct of developmental sensitivity assessment reflects the extent to which methods are calibrated to the capabilities of an age period and have broad enough range to capture the full spectrum of normative variation and the occurrence of maturational change, typically emphasizing this within a developmental period. While salient at any age, developmental sensitivity is of particular importance in early childhood when developmental change and the extent of normative variation are so extensive (Wakschlag, Tolan, & Leventhal, 2010). Thus, developmental sensitivity pushes towards specificity. The contrasting construct of lifespan coherence emphasizes characterization of phenomena across developmental periods (Carter, Gray, Baillargeon, & Wakschlag, 2013). It conceptualizes latent constructs that are equivalent across the lifespan although their particular expression may vary at different developmental periods. This element pushes towards being more general. Thus, while not antithetical, these two constructs do carry an inherent tension between them in selecting an assessment strategy. Principle 1 suggests balanced and deliberate consideration of both in measurement selection.

These joint considerations are of particular salience for HBCD due to the dual objectives of characterizing normative early brain and behavioral development along with specifying nascent patterns that presage early expressions of key ABCD substance abuse and mental health outcomes and their exposure-related predictors. The challenges and trade-offs to this are most evident in relation to assessment of clinically-salient constructs. Guided by Principle 1, we have suggested a balanced approach requires moving beyond traditional syndrome-specific, symptom-based clinically focused assessment strategies to incorporate a transdiagnostic dimensional indicator of neurodevelopmental vulnerability to mental health problems risk, i.e., irritability, measurable from birth, characterizes the full normal:abnormal spectrum of expression, robustly linked to exposure and is an efficient marker of self-regulation which subserves long-term (mal) adaptation (Beauchaine & Cicchetti, 2019; Wakschlag et al., 2018; Wakschlag et al., 2019).

Disciplinary divides between developmental and clinical science have traditionally impeded achieving this balance of developmental sensitivity and lifespan coherence in the assessment of mental health constructs (Blackwell et al., 2020; Wakschlag et al., 2010). Barriers to achieving this synthesis for capturing heterotypic continuity in psychopathologic expression reflect (a) the fact that traditional nosologies (e.g., DSM) have typically “downward extended” symptom sets from their youth expression such that they fail to capture young child expression in a developmentally meaningful manner (Buka & Gilman, 2002; Tremblay, 2000; Wiggins, Briggs-Gowan, Brotman, Leibenluft, & Wakschlag, in press); and (b) prevailing clinical views that subscribe to a “they’ll grow out of it myth” (Luby, 2012). As a result, past assessment strategies for characterizing young children’s typical and atypical neurodevelopment in developmental science and clinical research have largely utilized disparate conceptualizations and paradigms to measure highly overlapping constructs capturing capacities in self-regulation that subserve temperament (in young children) and psychopathology (in youth) (Cole, Luby, & Sullivan, 2008; Wakschlag et al., 2010). Fortunately, a number of advances in neurodevelopmental science over the past decades provide a science-base and paradigms for the requisite synthesis of these domains essential to optimizing HBCD-ABCD linkage, making it possible to adopt a lifespan coherent approach beginning at much younger ages than previously targeted.

There is now definitive evidence that pediatric psychopathologies (e.g., internalizing and externalizing syndromes that are the most robust intrinsic child risk indicators for substance use/abuse and other adaptational problems) have origins in early childhood and can be reliably and validly identified at preschool age with developmentally based methods (Bufferd, Dyson, Hernandez, & Wakschlag, 2016; Chapman, Tarter, Kirisci, & Cornelius, 2007; Dougherty et al., 2015; Egger & Angold, 2006; Tarter, 2002; Zucker, 2008). Moreover, modern psychopathological frameworks (such as the NIMH Research Domain Criteria -RDoC) no longer conceptualize clinical syndromes as discrete, categorical entities based on constellations of extreme, heterogeneous entities (Cuthbert, 2014). Rather, these are now understood (and measured) as dimensional spectra from typical to atypical with corollary developmentally sensitive measurement tools, with emphasis on narrow band phenotypes for mechanistic linkage and characterization as developmentally unfolding and beginning in the earliest phase of the clinical sequence (Casey, Oliveri, & Insel, 2014; McGorry, Ratheesh, & O'Donoghue, 2018; Mittal & Wakschlag, 2017). Importantly, advances in developmental neuroscience and infant cognitive science now enable specification of brain:behavior linkages that presage maladaptation beginning in the first year of life, further paving the way for mapping previously unimagined continuities across the period from infancy to adolescence (Barch, Belden, Tillman, Whalen, & Luby, 2018; Bosl, Tager-Flusberg, & Nelson, 2018; Graham et al., 2016; Rogers et al., 2017). Leveraging these advances informs the HBCD neurodevelopmental assessment strategies described below.

Within Principle 1, we tended to advantage lifespan coherence when possible with survey measures, adhering to the established notion of “good enough” (Blackwell et al., 2020; Scheeringa, 2003) towards the planned HBCD:ABCD mapping. That is, even if more developmentally sensitive survey measures existed, we proposed using surveys originally developed for older youth employed by ABCD or other trans-NIH initiatives (e.g., ECHO), if developmentally validated versions for early childhood were available (Blackwell, Wakschlag, Gershon, Cella, & Core, 2018). In contrast, developmental-sensitivity took precedence for direct assessments (Principle 2) where we typically recommended tasks specifically developed for early childhood, reflecting rapidly changing capacities of this age period and developmental constraints that precluded tasks administered in youth. We also incorporated state-of-the science methods not available in some of the traditional checklist measures that are frequently used because they have been used in prior consortia. For example, we recommend capitalizing on advances in health information technology, such as computer adaptive testing (CAT) methods, which reduce burden while improving precision to address the challenges of extensive survey batteries that plague consortia (Veldkamp & van der Linden, 2002), and

Principle 2: Beyond Parent Report- Child Direct (Performance-Based) Assessments Are Critical for Capturing Developmental Heterogeneity and Change in a Manner Sufficiently Sensitive to Teratologic Effects and Neural Mechanisms

Parent surveys have tremendous practical and ecological advantages for large consortia, including feasibility/ease of administration, low burden and parents’ in-depth knowledge of their children’s development and functioning. However, while parents are good historians about their children’s behavior, they are less reliable at judging whether behaviors are developmentally normative or atypical (Lord, 1997; Wakschlag et al., 2005). This requires systematic, nuanced information based on a standardized set of observations and task demands that has been rigorously validated within appropriate age bands, particularly in early childhood when capacity changes occur in weeks or months rather than years (Wakschlag et al., 2005). Further, parental accuracy in reporting is influenced by a variety of factors including developmental knowledge and experience, subjective thresholds for misbehavior and parental stress and mental health (Briggs-Gowan, Carter, & Schwab-Stone, 1996; Hay, Castle, Davies, Demetriou, & Stimson, 1999). There is also evidence that task-based assessments of discrete processes (e.g., executive function, emotion processing) are more sensitive and specific to teratologic effects than indicators of global developmental functioning (Jacobson & Jacobson, 1996). These discrete information processing tasks are also important for capturing the full range of normative brain:behavior variation enabled by tremendous advances in neuroimaging that enable brain-based assessments beginning in neonates (Grabell et al., 2017; Howell et al., 2019; Marshall & Fox, 2007; Perlman, Luna, Hein, & Huppert, 2014). As detailed in the sections below, application of this principle was interwoven with other principles (e.g., pragmatics and correspondence to other large-scale consortia) to guide recommendations. For example, several executive function tasks were recommended from the NIH Toolbox (including from its nascent platform for infants/toddlers) reflecting its widespread use and its design for pragmatic administration in epidemiologic studies where expertise of staff in neurodevelopment may vary widely. A key underlying feature is close coordination of task selection with neuroimaging protocols. Recommendations for HBCD neuroimaging protocols were generated by a linked workgroup and are beyond the scope of the present paper.

Principle 3: Deploy Pragmatic Strategies Whenever Possible

The ambitiousness and scope of large national neurodevelopmental consortia necessitates scientific engagement across a broad range of disciplines. While this brings depth and breadth, it can also lead to investigators from varied fields championing their respective favorite measures with a resultant protocol that is lengthy, burdensome, and often difficult to deploy at sites that vary substantially in the nature of their expertise and staff training. Furthermore, as highlighted in Principle 2, neurodevelopmental fields have traditionally heavily weighted intensive, highly specialized laboratory-based assessments, which are essential but are often not sufficiently evaluated in terms of predictive and clinically meaningful added value relative to adjacent tasks recommended. Moreover, traditional survey protocols have often drawn on “legacy” measures which are lengthy and proprietary, constraining ability to apply modern psychometric methods for data reduction to improve efficiency. This approach lags behind the state-of-the-science, engenders great cost and burden, and may hamper participant engagement and result in missing data, particularly for disparities populations. For all these reasons, the use of pragmatic measurement strategies is a guiding HBCD workgroup principle. In pragmatic approaches, research is conducted in diverse contexts; measures must be meaningful to stakeholders, feasible and actionable in the real world, sensitive to change, broadly applicable, brief, and publicly available if possible (Glasgow, 2013). Pragmatic strategies do not mitigate the need for psychometric rigor, developmental sensitivity, or characterization optimized for mechanistic linkages (Glasgow & Riley, 2013). Indeed, we underscore that this pragmatic principle is not in conflict with Principle 2 emphasizing the importance of direct assessment of neurodevelopmental capacities. Developmental sensitivity and optimizing assessments to enable mechanistic linkages remains of utmost importance in neurodevelopmental research (Blackwell et al., 2020). Rather, joint consideration of these principles emphasizes that this is not an “either/or” choice. Instead, this principle raises pragmatic concerns to a co-equal place at the measurement selection table. For survey measures, this includes advantaging non-proprietary measures that can be adapted to short-forms and/or computer adaptive test (CAT) administration that reduces length while maximizing precision (Segawa, Schalet, & Cella, 2020). In the case of key outcomes where clinical interviews are needed, this points to epidemiologically validated measures that can be modified (e.g., shortened, administered remotely by a single coordinating site) for feasibility (Egger et al., 2006). In the neurodevelopmental arena, these may include selection of tasks that can serve more than one measurement objective, can be administered remotely if possible, and whose training requirements are geared to lay administrators. While weighting pragmatic vs. nuance and depth considerations in measurement selection is not an easy task for consortia in which much is invested and much is at stake, acknowledging that each is of great importance will help ensure that selection of less pragmatic measures will occur only for essential constructs of high importance to charting brain:behavior pathways most robustly predictive of long-term (mal)adaptation.

Principle 4: Characterization of Developmental and Ecological Assets Is Imperative

While a central focus of many neurodevelopmental consortia is risk detection, HBCD also has characterization of normative brain:behavior pathways beginning even before birth as a key objective. To capture individual differences in pathways (e.g., why do some neurodevelopmentally vulnerable young children develop adaptively, what factors determine which children will have normative pathways), we must go beyond risk to effectively capture compensatory factors. While ecological protective factors are central to this (National Association for the Education of Young Children, 2004), our focus here is on ensuring that the HBCD neurodevelopmental assessment strategy moves beyond a deficit orientation to include a strengths-based focus which captures developmental strengths, wellbeing, and positive health (Fenton, Walsh, Wong, & Cumming, 2015; Forrest, Blackwell, & Camargo Jr., 2018; Restoule, Hopkins, Robinson, & Wiebe, 2015). This includes measuring the full spectrum of child capacities (not merely above or below a threshold), developmental domains that may serve a compensatory function when neurodevelopmental vulnerability is present (e.g., language, sociability), and indicators of well-being (e.g., engagement, curiosity, persistence) (Blackwell et al., 2020; Moreno & Robinson, 2005; Roben, Cole, & Armstrong, 2013).

Exemplar Constructs and Proposed Assessments

Neurodevelopment connotes developmental changes in behavior associated with changes in neural structure and function. Neurodevelopmental health represents the balance and interconnectivity of neuromaturation (brain and behavior) which supports the child’s capacity to function adaptively. One of the most salient features of neurodevelopmental health within this context is self-regulation, i.e., the capacity of the child to inhibit, shift, control, and flexibly re-direct and re-orient behavior, emotions, and attention in response to changing environmental demands and contexts. Other neurodevelopmental features to consider include motor skills, cognitive abilities, executive function, and emotion processing. Good neurodevelopmental health in young children portends adaptive pathways and may buffer the adverse effects of exposures, supporting more positive health trajectories even under high-risk conditions. In contrast, poor neurodevelopmental health is conceptualized as dysregulated affect and behavior that reflects impairments in disinhibitory processes resulting in decreased ability to inhibit and modulate emotions and behavior in the face of anticipated negative consequences, and/or shifting contextual demands. This includes impairments in top-down cognitive control processes in lateral prefrontal and anterior cingulate cortex, impaired development of bottom-up arousal and reward centers in the nucleus accumbens and orbitofrontal cortex, as well as memory-related hippocampal structures. The role of developmental processes such as cognition, language, motor, and socio-emotional development, which are impacted by teratogenic exposures, should also be examined as part of a neurodevelopmental health framework. While traditional clinical outcomes (i.e., DSM-oriented) should be incorporated into any assessment framework, the neurodevelopmental health and vulnerability framework provides a broader, dimensional and more developmentally oriented framework for this age period.

In this section, we discuss developmental domains and constructs that are often studied as outcomes of adversity, exposure, and stress, and are indicators of neurodevelopmental health. These constructs and domains were decided upon by workgroup members, and potential assessments were proposed by smaller groups within the workgroup who had expertise on a particular construct/domain. Much debate and discussion around choice of measure, pros and cons for different assessments, and possible options resulted in a proposed set of constructs and assessments (see Table 1). Decisions were informed by the guiding principles discussed above, and the principles were primary drivers in final recommendations. We acknowledge that there are many options for assessments for the constructs proposed. Our recommended measures are exemplars and possible options. Final decisions are yet to be made regarding the specific assessments that will be utilized in HBCD, although we believe that the principles proposed in this paper will be foundational in these decisions.

Table 1

Exemplar constructs and assessments

Construct and proposed assessment	Modality and time	Data collection points	Strengths/limitations	Use in other consortia/frameworks	Possible pragmatic adaptations
Socio-emotional development
Self-regulation
Behavioral Inhibition/fear				ABCD uses BIS/BAS
Laboratory temperament assessment battery (Lab-TAB)a stranger approach	Direct assessment (4–10 min)	6 months–2 years (once in first year, annual)	Widely used laboratory task, adaptable for many ages. Can be performed in Spanish. Observational, needs coding.	Lab-TAB listed as “alternative option” in ECHO	Possible live coding; examiner coded as part of any lab visit. Proposed in Baby Toolbox.
Negative affect and irritabilityb
Infant Behavior Questionnaire (IBQ-R)c Fear, anger, and sadness scales	Mother/care-giver survey (10 min)	3–12 months (once in first year)	Short form available, no cost. Widely used instrument and easily accessible. Spanish version available	NIH Toolbox uses similar items; ECHO uses Very Short Form	Use of short-form; remote data collection possible
Early Child Behavior Questionnaire (ECBQ)c Fear, anger, and sadness scales	Mother/care-giver survey (10 min)	18–36 months (annual, age 2 and 3)	Short form available, no cost. Widely used instrument and easily accessible. Spanish version available	NIH Toolbox uses similar items; ECHO uses Very Short Form	Use of short-form; remote data collection possible
Child Behavior Questionnaire (CBQ)c Fear, anger, and sadness scales	Mother/care-giver survey (10 min)	3–7 years (annual, age 4 and 5)	Short form available, no cost. Widely used instrument and easily accessible. Spanish version available	NIH Toolbox uses similar items; ECHO uses Very Short Form	Use of short-form; remote data collection possible
LabTAB Attractive Toy Barrier	Direct assessment (3–5 min)	6 months–2 years (once in first year, annual)	Widely used laboratory task, adaptable for many ages. Can be performed in Spanish. Observational, needs coding.	Lab-TAB listed as “alternative option” in ECHO	Possible live coding Examiners can provide additional ratings. Proposed measure in Baby Tool-box
Inhibitory controld/regulation
Child Behavior Questionnaire (CBQ) Effortful control	Mother/care-giver survey (3 min)	3–5 years (annual)	Short form available, no cost. Widely used instrument and easily accessible. Spanish version available	EC PROMIS	Use of short-form; remote data collection possible
LabTAB Snack delay	Direct assessment (5 min)	3–5 years (annual)	Widely used task, can be adapted for different ages. Can be performed in Spanish. Observational, needs coding	Lab-TAB listed as “alternative option” in ECHO	Possible live coding
Social engagement and communication
Bayley Scales of Infant and Toddler Development IVe (social-emotional scale – social interaction, self-regulation, emotion communication, and sensory processing)	Direct Assessment 30–90 min (time for total assessment)	1–3 years (annual)	Popular in studies of pre-natal exposure. Provides standardized scores. Available in Spanish.	Used in PhenX and National Children’s Study (NCS)	Use only at ages 1 and 2 and then switch to Toolbox only. Includes cognitive, motor, and language assessments.
Disruptive Behavior Diagnostic Observation Schedule (DB-DOS)f Social engagement and socially directed positive affect scales	20–30 min (parent and examiner context)	24 months–5 years (annual)	Uses “presses” to efficiently elicit high levels of variation in capacity to regulate frustration, respond to environmental demands. Yields global codes ranging from normative variation to of clinical concern. Provides clinically informative data on variation across parent-examiner contexts.	ECHO (t)	Can be coded for parental responsiveness and discipline. Can be adapted for live coding.
Brief Infant-Toddler Social Emotional Assessment (BITSEA)g Competence scale	Survey (10 min) for full survey	1–5 years (annual)	Designed for early childhood Can be obtained free for research.	NCSAW ECHO (alt.)	Is already a short form
Positive affect/approach (surgency, reward responsiveness)
Infant Behavior Questionnaire (IBQ) Smiling and laughter scales	Mother/care-giver survey (5 min)	3–12 months (once in first year)	Short form available, no cost. Widely used instrument and easily accessible. Spanish version available	NIH Toolbox uses similar items; ECHO uses Very Short Form	Use of short-form; remote data collection possible
Early Child Behavior Questionnaire (ECBQ) High intensity pleasure, smiling scales	Mother/care-giver survey (5 min)	18–36 months (annual, age 2 and 3)	Short form available, no cost. Widely used instrument and easily accessible. Spanish version available	NIH Toolbox uses similar items; ECHO uses VeryShortForm	Use of short-form; remote data collection possible
Child Behavior Questionnaire (CBQ) High intensity pleasure, smiling	Mother/care-giver survey (5 min)	3–7 years (annual, age 4 and 5)	Short form available, no cost. Widely used instrument and easily accessible. Spanish version available	NIH Toolbox uses similar items; ECHO uses VeryShortForm	Use of short-form; remote data collection possible
LabTAB Puppets or peekaboo	Direct Assessment (4–10 min)	6–30+ months (prelocomotor) (annual)	Widely used laboratory task, adaptable for many ages. Can be performed in Spanish. Observational, needs coding	Lab-TAB listed as “alternative option” in ECHO	Possible live coding.
LabTAB Blowing bubbles or balloon pop	Direct Assessment (4–10 min)	36–60 months (annual)	Widely used laboratory task, adaptable for many ages. Can be performed in Spanish. Observational, needs coding	Lab-TAB listed as “alternative option” in ECHO	Possible live coding.
Developmental psychopathology
Internalizing and externalizing problems (aligned with DSM syndrome framework)
Brief Infant-Toddler Social Emotional Assessment (BITSEA) Scales: Externalizing Internalizing (competence)	Survey (10 min)	1–5 years, (6 months intervals or annual)	Designed for early childhood Can be obtained free for research	NSAW ECHO (alt.)	Redcap version is already a short form
Preschool Age Psychiatric Assessment (PAPA)h	Interview (1 h, see pragmatics)	2–5 years (annual)	Gold standard measure, widely used and validated, developmentally sensitive- Includes assessment of behaviors across contexts and impairment Does generate DSM syndromes consistent with INT/EXT for ABCD but provides more developmentally-based information for symptom determination and dysregulation	CDC Field Trial	Designed for lay interviewers Originally lengthy training and administration. Has been significantly adapted for pragmatic use in redcap and for remote administration Requires training but could be administered by single coordinating center team remotely Developer is committed to working with consortium to ensure 1-h length)
Autism spectrum (aligned with DSM syndrome framework)
Modified Checklist for Autism in Toddlers (MCHAT-R)i (16–30 months)	Survey 10 min	Close to 16 and 30 months (align with other timepoints; annual)	Widely used screener. Data to be combined with developmental data from standardized assessment (E.g. Bayley) in re: developmental disorders	ECHO
Transdiagnostic dimensional phenotype of emotion dysregulation: irritability (neurodevelopmental rather than syndromal). Predictive of most developmental psychopathologies
Multidimensional Assessment Profile of Disruptive Behavior (MAP-DB)j Temper Loss Dimensional Scale	5–7 min	12 months–5 years (annual)	Specifically designed as the full normal:abnormal spectrum for early childhood to differentiate typical and dysregulated irritability includes assessment of context, developmental expectability and dysregulation Large scale validation Has been shown to detect irritability-related neural disruptions and exposure-related effects relative to traditional symptom or temperament scales	ECHO (alt.)	In redcap Could be administered within CAT
DB-DOS Measure (Anger Modulation Scale) (and DB-DOS BIoSync)	20–30 min (parent and examiner context)	24 months–5 years (annual)	Uses “presses” to efficiently elicit high levels of variation in capacity to regulate frustration, respond to environmental demands Yields global codes ranging from normative variation to of clinical concern Provides clinically informative data on variation across parent-examiner contexts in social and developmental context and also includes coding system for parental and dyadic behavior	ECHO (alt)	Has been adapted for use with biologic measures (DB-DOS BioSync) such as EEG/NIRS and biosensors Adaptation for live coding possible Can be done in the home –potentially remote
Impairment
Family Life Impairment Scale (FLIS)k	Survey (10 min) 12 months–5 years	Concurrent with BITSEA (annual)	Assesses domains of impairment (family, child functional, childcare)-cross-domain impairment has been shown to predict poor outcomes	No (impairment is an important typical:atypical differentiator but has not received sufficient attention in prior consortia)	Could administer select domains
Cognitive development and executive function
Early cognitive function
Bayley Scales of Infant and Toddler Development IV (visual preference, attention, memory, sensorimotor, exploration and manipulation, concept formation)	Direct assessment 30–90 min (time for total assessment)	1–3 years (annual)	Popular in studies of pre-natal exposure. Provides standardized scores. Available in Spanish.	Used in PhenX and NCS	Use only at ages 1 and 2 and then switch to Toolbox only
Short-term memory/attention
Change detection taskl	Direct assessment (5–10 min)	(4–13 months) 12 months and once prior	Non-verbal task
Inhibitory control and attention
NIH-Toolboxm - Flanker Inhibitory Control and Attention Task on iPad	Direct assessment (3 min)	3–7 years (annual)	Widely used. Available in Spanish.	Part of NIH Toolbox, used in ABCD; NCS
NIH-Toolbox - Dimensional Change Card Sort Task (DCCS) on iPad	Direct assessment (4 min)	3–7 years (annual)	Widely used. Available in Spanish. Linked with studies of exposure	Part of NIH Toolbox, used in ABCD; PhenX, NCS
Language development
Bayley Scales of Infant and Toddler Development IV (receptive and expressive language scores)	Direct assessment 30–90 min	1–3 years (annual)	Popular in studies of pre-natal exposure. Provides standardized scores. Available in Spanish.	Used in PhenX and NCS
MacArthur-Bates Communication Development Inventory (MB-DCI)n	Survey (20 min)	8 months–3 years (annual)	Screening measure of early communication and language milestones. Widely used measure of language acquisition. Standardized and normed, available in many languages	Part of NIH-Toolbox	Parents complete a form
NIH-Toolbox Picture Vocabulary Test (PVT) and Oral Reading Recognition Test (ORRT)	Direct assessment PVT = 4 min ORRT = 4 min	3–6 years (annual)	Low burden and standardized	Part of NIH-Toolbox “Early childhood cognition battery”	Done on an iPad
Motor development
Bayley Scales of Infant and Toddler Development IV	Direct assessment 30–90 min (could use subscales)	1–42 months (annual)	Popular in studies of pre-natal exposure. Provides standardized scores. Available in Spanish.	Used in PhenX and NCS,
Vineland Motor Domain (included in Bayley IV)	Questionnaire/interview 10–20 min for survey	Birth—9 years (annual)	Assesses adaptive function broadly, commonly used in disability research. Available in Spanish.	Used in PhenX and has a large age span, parent report
Health and growth
Clinical metricso	Physical exam, parent report, or HER 5 min	Neonatal on (annual)	Provides critical information such as low-birth weight. May be difficult to obtain as children age	PhenX
Physical activity Wearable sensorp	Actigraph or biosensor;	Neonatal/toddler for physical activity (annual)	Provides real-world data and could be collected at home. Expense could be high.	Phen X and Assessed in ABCD
Physical activity Screen timeq	Parent report 5 min	Adapt for use at all ages (annual)	Screen time survey–proxy measure, easily administered	Assessed in ABCD
Nutrition Dietary Screener Questionnairer	Parent report Less than 15 min	For infants and toddlers		PhenX
Nutrition Child Eating Behavior Questionnaires	Parent report Less than 15 min	for 4 + (4–12 years)		PhenX
Sleep Brief Infant Sleep Questionnairet	Parent report 10 min	Birth–2 years (annual)		ECHO
Sleep PROMIS Pediatric sleep-related impairment and disturbanceu for ages 2–7 – parent report	Parent report 10 min	Ages 2–7 years (annual)		ECHO
Ages and stages (ASQ)v	Parent report 10–15 (could do select scales)	1 month–5.5 years (once prior to 1 year, annual)	Popular in pediatric settings	PhenX, NCS, ECHO	Can be done remotely or via interview. Collect when not doing Bayley.

ECHO, Environmental influences on Child Health Outcomes; NCSAW, National survey of Child and Adolescent Well-being; NCS, National Children’s Study; EC PROMIS, Early Childhood Patient-Reported Outcomes Measurement Patient Information System

aPlanalp, Van Hulle, Gagne, & Goldsmith, 2017; https://epsy.tamu.edu/gagne-lab/lab-tab/

bConstruct assessed in social and emotional development and developmental psychopathology domain

chttps://research.bowdoin.edu/rothbart-temperament-questionnaires/

dConstruct assessed in social and emotional and cognitive development domain

eBayley, 2006

fWakschlag et al., 2008; Wakschlag et al., 2008

gBriggs-Gowan, Carter, Irwin, Wachtel, & Cicchetti, 2004; Gardner et al., 2013

hEgger & Angold, 2004

iRobins et al., 2014; https://m-chat.org/en-us/page/take-m-chat-test/print-version

jKaat et al., 2019

kMian, Soto, Briggs-Gowan, & Carter, 2018

lRoss-Sheehy et al., 2003

mWeintraub et al., 2013

nHeilmann et al., 2005

ohttps://www.phenxtoolkit.org/domains/view/20000

pSmith et al., 2015; Trujillo-Priego et al., 2017; Pate et al., 2006

qBarch et al., 2018

rhttps://www.phenxtoolkit.org/protocols/view/231201

shttps://www.phenxtoolkit.org/protocols/view/650301

tDel-Ponte et al., 2020

uForrest et al., 2018

vChen, Squires, & Scalise, 2020; https://agesandstages.com/

Socio-Emotional Development

Individual differences in socio-emotional development are a key foundation for children’s adaptation and health and are a central determinant of whether adverse exposures result in maladaptive or resilient outcomes (Phillips & Shonkoff, 2000). These have been extensively studied as normative variation within the temperamental realm, often as dimensions of regulation and reactivity, and in atypical form as developmental psychopathology (Behrendt, Wade, Bayet, Nelson, & Bosquet Enlow, 2020; Cole et al., 2008; Fox & Pine, 2012; Rapee & Coplan, 2010; Sroufe, 1990). When extreme, they also may represent nascent forms of the outcomes of central interest in ABCD and other neurodevelopmental consortia, i.e., mental health and wellbeing, mental illness, and related problem behaviors. Nevertheless, practical tradeoffs must be made in determining what to measure when. Thus, we chose to recommend more intensive, multi-method measurement of temperament across the infant-toddler period and a parallel approach to developmental psychopathology across the preschool period (with temperament still measured via surveys; see Table 1).

Temperament can be defined as predispositions towards patterns of regulation, reactivity, and sociability which underlie normative socio-emotional functioning (Chen & Schmidt, 2015; Rothbart, 2007). These facets have been linked to neurocircuitry and neurological development in a number of studies (Adam, Klimes-Dougan, & Gunnar, 2007; Clauss et al., 2014; Fox, Hane, & Pine, 2007; Henderson & Wachs, 2007). There is particularly strong evidence that reactivity domains of behavioral inhibition, fearfulness, and sadness when extreme often are associated with internalizing problems (Buss, 2011; Cole et al., 2008; Fox & Pine, 2012; Rapee & Coplan, 2010); and negative affectivity and irritability, when extreme often are associated with externalizing problems (Frick & Morris, 2004; Wakschlag et al., 2018). Regulation domains (i.e., effortful control, inhibitory control, and emotion regulation) are key facets in the ability to control behavioral responses that underlie normative development (Eisenberg & Morris, 2002; Rothbart, Sheese, & Posner, 2007) and many different forms of psychopathology such as ADHD, internalizing, and externalizing (Compas et al., 2017; Martel & Nigg, 2006; Sheppes, Suri, & Gross, 2015). We propose to assess sociability and social competence as domains of social communication and engagement, positive affect, and approach. Social communication and engagement include communication and affiliation/relational assessments. Deficits in these domains may be associated with autism and externalizing problems (e.g., Clifford, Hudry, Elsabbagh, Charman, & Johnson, 2013; Rubin, Burgess, Dwyer, & Hastings, 2003). Positive affect and approach, which underlie surgency and response to reward, are often associated with ADHD and internalizing and externalizing problems (i.e., lack of positive affect) (Forbes & Dahl, 2005; Olino et al., 2011). It should be noted that variations in temperament are not deterministic, rather variability in temperament at young ages has been associated with symptomology at later ages when also considering the caregiving context (see Fox & Calkins, 1993; Miller, Degnan, Hane, Fox, & Chronis-Tuscano, 2019).

We suggest using a combination of parent report and direct observation to assess the above constructs at the following time points: once between ages 3 and 9 months, and at approximately 12 months, 24 months, 36 months, 48, and 60 months. At 48 and 60 months, we are proposing to only obtain parent/caregiver reports of temperament to ease participant burden temperament-related measures we are proposing are listed in Table 1.

Developmental Psychopathology

When considering socio-emotional development along a continuum, atypical patterns and extremes can be considered forms of psychopathology. Thus, we advise assessing internalizing and externalizing behaviors and symptoms of autism along a continuum, aligned with a DSM syndrome framework, annually in children ages 3, 4, and 5. We propose assessing emotion dysregulation and irritability along a spectrum from normal to atypical, which is in line with a transdiagnostic phenotype that predicts many forms of psychopathology and is aligned with a neurodevelopmental perspective (see Wakschlag et al., 2018). Specifically, we propose measuring clinical signs and symptoms/patterns of emotional and behavioral functioning that interfere with normative developmental activities/expectations and are outside the broad range of expectable developmental variation in self-regulatory capacities and social-emotional wellbeing within an age period. Our emphasis is on patterns and dimensions rather than categorical diagnosis. In addition, it is important to examine how symptoms interfere with everyday functioning and impairment. Impairment represents interference with age-graded achievement of developmental expectations, roles, and activities. It includes interference with functioning across settings and impact on the family (i.e., not only interference with child functioning but also family functioning, given the impact of clinically salient patterns on this most salient proximal environment).

We argue that clinically salient patterns should always be understood within neurodevelopmental and environmental context, and should not be considered as reified or necessarily as an immutable pattern intrinsic to the child. While there is clear evidence of some developmental discontinuities, there is also robust evidence that demonstrates that clinical syndromes are present in young children and associated with increased risk of chronic psychopathology (Wakschlag et al., 2019). The most robust science base links exposure to common and modifiable early onsetting patterns of psychopathology within the internalizing/externalizing domains which are understood as problems in self-regulation. Conceptually, we recommend consideration of transdiagnostic approaches that transcend individual syndromes, perhaps emphasizing “impairing mental health problems” using approaches such as P-factor modeling rather than emphasis on specific narrow band syndromes, probabilistic risk of problems rather than dichotomization. We have also included autism spectrum measurement in our recommendations partly because of emerging evidence of linkages between prenatal opioid and other substance exposures and in light of its early life onset (e.g., Sandtorv et al., 2018). We are proposing a mix of caregiver surveys, a diagnostic interview with the parent, and observational assessments to examine the constructs described above annually (see Table 1).

Cognitive Development and Executive Function

There are a host of possible assessment options for cognitive development in infancy and early childhood. Our group chose to focus on components of executive function and cognitive constructs that align with early brain development, as brain development is the core focus of HBCD. Indeed, neurodevelopmental assessments will be collected along with functional and structural images of brain development over time, using state-of-the-art magnetic resonance imaging and EEG.

Executive function, also known as executive or cognitive control, refers to top-down mental processes invoked for concentration and paying attention, and it requires mental effort. There are three core components of executive function: inhibition (inhibitory behavioral control and selective attention), working memory (active manipulation of information in memory), and cognitive flexibility (set shifting, task switching, and mental flexibility). From these, higher-order executive functions (e.g., reasoning, problem solving, and planning) are built (Collins & Koechlin, 2012). Executive function skills are essential for mental and physical health, academic success, and cognitive, social, and psychological development (Carlson, 2009; Diamond, 2013). We propose to assess constructs of short-term (working) memory, inhibitory control, and attention, and general cognitive functioning. Studies have linked early exposure to adversity and prenatal substance use to general cognitive functioning and processing (e.g., Salzwedel, Chen, Chen, Grewen, & Gao, in press; Tomalski & Johnson, 2010), and executive function has been associated with academic achievement and other measures of adjustment in numerous studies (e.g., Best, Miller, & Naglieri, 2011; McDermott, Westerlund, Zeanah, Nelson, & Fox, 2012). Importantly, recent advances in technology have aided researchers in studying executive function in very young children, such as portable eye-tracking and recording systems. In particular, our workgroup chose to utilize the NIH Baby Toolbox (in development) which has direct assessments of executive function; the Bayley IV scales of visual preference, attention, memory, sensorimotor, exploration and manipulation, concept formation as a general assessment of cognitive functioning, and the change detection task which can be used in very young children to assess visual attention and working memory (Ross-Sheehy, Oakes, & Luck, 2003. Most assessments will take place annually (or in concert with imaging/EEG data collection), with the change detection task proposed once prior to 12 months and then annually (see Table 1).

Language Development

Language refers to a complex system of symbolic representations that consists of several component layers that include but are not limited to phonology, morphology, syntax, and semantics (Novack & Waxman, 2020). Major developments in language understanding occur in the first years of life that are continuously shaped by the environment (Kohl, 1993; Werker & Tees, 1984). Two primary domains within language assessment include receptive and expressive language, with receptive language reflecting attention, recognition, and comprehension of the auditory and visual cues to speech, and expressive language reflecting the ability to produce and utilize speech (e.g., lexical and syntactic development). Focusing on these two primary areas provides the most comprehensive view of language development, although we acknowledge the importance of the multiple domains, such as pragmatics and phonological development, that are essential to successful language utilization. Our working group prioritized input from multiple sources and therefore chose both direct assessments of the child and a caregiver survey as the primary measures. Two direct assessment measures have been recommended to cover the age range of HBCD, with the Bayley Scales of Infant and Toddler Development IV applied to ages 1–3 years (receptive and expressive language) (Bayley, 2006) and the NIH Toolbox Picture Vocabulary Test (PVT) and the Oral Reading Recognition Test (ORRT) (Weintraub et al., 2013) applied to aged 3–6 years. Both assessments are standardized and address receptive and expressive language skills in children. The MacArthur-Bates Communication Inventory (MB-DCI) (Heilmann, Ellis Weismer, Evans, & Hollar, 2005) is a standardized, caregiver report of the child’s receptive and expressive language skills. We are proposing annual direct assessment of the child’s language abilities and potentially more frequent caregiver report given the ease of administration associated with survey measures (see Table 1).

Motor and Physical Development, Health and Growth

It is critical to consider the impact of motor and physical development, including health and growth factors, when characterizing developmental change across infancy and childhood. Here, the motor domain includes the skillful maintenance, modification, and control of voluntary postures and movement patterns. Similar to other domains, the workgroup chose assessments that included caregiver report and direct behavioral assessment given the importance of each type of feedback. The Bayley Scales of Infant and Toddler Development IV will be utilized to directly assess infant and child motor skills, and caregivers will complete the Vineland Adaptive Behavior Scales: Motor Domain, which can be completed as a questionnaire or in interview format. A shortened adaptive version addressing behavioral content from the Vineland-3 is now available within the Bayley-IV assessment package. Health and growth factors to be taken into consideration include clinical metrics, such as information from medical records of pediatrician visits and physical exams, and direct measures of the infants and children’s physical activity and movement patterns using wearable sensor technology (see Abrishami et al., 2019; Pate, Almeida, McIver, Pfeiffer, & Dowda, 2006; Smith, Trujillo-Priego, Lane, Finley, & Horak, 2015; Trujillo-Priego et al., 2017). Caregiver report measures include a screen time survey adapted from the ABCD study (Barch et al., 2018), the Dietary Screener Questionnaire for infants (National Center for Health Statistics, 2009) and Child Eating Behavior Questionnaire for older children to assess nutrition (Wardle, Guthrie, Sanderson, & Rapoport, 2001), the Brief Infant Sleep Questionnaire for infants (Del-Ponte et al., 2020), and the PROMIS Pediatric Sleep Related Impairment and Disturbance for ages 2–7 years: Parent Questionnaire (Forrest et al., 2018) for older children to assess sleep, and the Ages and Stages Questionnaire (Squires & Bricker, 2009) to assess sensory function. Many of these measures have been administered in prenatal exposure studies and have been validated across diverse populations. The workgroup is recommending annual assessments of motor and physical development, although caregiver surveys could be conducted more frequently, for example, the Ages and Stages Questionnaire could be administered at 6-month intervals during the first years of life. Sensor-based assessment could also be assessed more frequently during infancy and linked with early MRI and EEG assessment.

Pragmatic Adaptations and Considerations

Much like for the rest of the world, the spread of COVID-19 (SARS-CoV-2) and subsequent closure of businesses, schools, and institutions created a new world for neurodevelopmental researchers. Longitudinal studies that were in on-going data collection were now losing critical data with each passing month as infants/young children aged out of developmentally specific protocols. Rather than mourn the loss of these timepoints, many researchers rushed to adapt protocols for more pragmatic assessment, with particular innovation in the use of remote assessment and the rapid publication of adapted protocols (Manning, Harpole, Harriott, Postolowicz, & Norton, in press). Listservs, such as the one hosted by the Cognitive Development Society, and conferences that were held virtually, including the International Congress of Infant Studies 2020, quickly became the center of information-sharing on protocol development and behavioral coding of sessions conducted and recorded in the participants’ homes often with a remote administrator. In addition, numerous funding opportunities to facilitate rapid adaption of protocols were made available via NIH supplemental funding (including HBCD supplements) and NSF RAPID proposals, among many others. Importantly, these adaptations facilitated immediate data collection in a world shut-down by a global pandemic. Innovations in assessment will have far-reaching influence on the field of neurodevelopmental science, which struggles with issues related to accessibility and engagement when designing longitudinal studies with multiple in-person highly specialized visits placing burden on participants and study teams (Hu, Wake, & Saffery, 2020).

Remote assessment has been growing in its use by developmental scientists, for example LookIt by MIT (a public platform for hosting studies) hosts infant “looking times” studies that can be administered to caregivers with their infants at home (MIT Early Childhood Cognition Lab., 2020). The recently launched Children Helping Science website allows caregivers and their children to connect with researchers online either synchronously via video chat or asynchronously to participate in developmental studies conducted on a phone, tablet, or computer (Parent Researcher Collaborative, 2020). Pragmatic assessments often follow the guideline of “Less is More”, for example, the NIH Toolbox (now being extended for infant/toddler assessment) aims to provide direct assessment measures that are less than 5 min and can be primarily conducted on an iPad; self or proxy report measures are often approximately one minute to administer (Gershon et al., 2013). During COVID-19, the NIH Toolbox adapted the standard administration guidelines for a number of their direct assessments so that they could be administered remotely via screen sharing (NIH Toolbox, 2020). This engendered considerable deliberations about the inherent trade-offs (e.g., some loss of nuance in exchange for meaningful data on key constructs). This type of adaptation was prevalent within study cohorts as well. For example, adaptation efforts to adapt the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) (Wakschlag et al., 2008) for assessment of child regulation of irritability across interactional contexts, and parent-child co-regulation of behavior as well as child competencies, one of our recommended assessments for the Developmental Psychopathology domain, became a priority. This direct assessment was designed to efficiently elicit variations in child (dys)regulation via “presses” as varied demands challenge and/or induce frustration across both a parent and an examiner context. The examiner context was deemed impossible for remote administration because it involves semi-structured examiner responses to child (mis)behavior and also requires the ability to challenge the child without actually engaging in person. However, the parent context was considered feasible and of high importance as it is where most variability is observed and is sensitive to exposure effects (Gray et al., 2012; Massey et al., 2018), and was thus adapted for remote data collection.

Pragmatic design considerations included the following: (1) what can be assumed available in the home environment and what has to be provided by the research team (e.g., toys for stimuli; technology for Zoom access by phone, tablet, or computer; WIFI access); (2) what privacy concerns are present (e.g., multiple family members present; research assistants are recording from home); (3) what instructions should be provided in advance to caregivers to address set-up and concerns; (4) how to deliver instructions to the caregiver without providing too much information to the child; (5) what camera angles are best depending on what task is being administered; (6) what tasks can be presented on screen and what requires manipulatives; and (7) critically, what is feasible to do in the home and what requires an in-lab visit to be conducted when it is safe to return to face-to-face research practices. Preliminary results reveal the feasibility of conducting remote assessment of child behavior when administrators are able to think flexibly, for example, talking with the caregiver about where the child is most comfortable (e.g., on the floor, in a high chair, at a table) to create a safe, conducive environment for data collection has been helpful. Learning from each remote visit has also been key: one recommendation when assessing verbal children is to provide instruction to the caregiver via Bluetooth headphone so that the child does not overhear the instructions. Suffice it to say, pragmatic remote assessment is not a simple adjustment of in-lab methods; however, the return on investment is the ability to continue to collect data on critical tasks during a global pandemic, and ultimately the possibility to provide families with options in the future for how they participate in research. This will have important implications for the design of the HBCD assessment strategy.

A final consideration when adapting behavioral assessments for in-home administration is the coding of the observations. Detailed coding schemes often require carefully controlled assessment so that individual nuances in the participant’s behavior can be noted and distinctly tied to an event—this is less evident when coding more naturalistic interactions and potentially more complicated in remote assessment, where camera angles are not ideal and distractions are likely present in the home environment (e.g., siblings, pets, television). Researchers also have to concede a level of control of administration when caregivers are given this task, as caregivers naturally make adjustments to language and instruction to meet their child’s level (Vigil, Hodges, & Klee, 2005), often a negative for assessment purposes. One possibility for coding may be to focus on global ratings of children’s behavior, which have been shown to align with more detailed moment-by-moment coding but may also be more forgiving of variations in assessment administration, or to have simple coding schemas that can be done in real-time (Adamson, Bakeman, Deckner, & Nelson, 2012; Bontinck, Warreyn, Meirsschaut, & Roeyers, 2018).

Our workgroup specifically discussed pragmatic assessments during two of our meetings, prior to the COVID outbreak. First, Elizabeth Planalp shared the coding scheme used in her research with colleague Hill Goldsmith, where Research Assistants (RAs) provide global codes of children’s temperament and behavior (e.g., approach/avoidance, social/shyness, positive and negative affect, exploration, attention, engagement). These codes are based on the RAs own perceptions of the child during a lab visit that includes various tasks and interactions (see Gagne, Van Hulle, Aksan, Essex, & Goldsmith, 2011). Similar protocols used in other studies have found that RA ratings assessed in the lab or at home are associated with children’s behaviors and adjustment (e.g., Moore, Planalp, Van Hulle, & Goldsmith, 2020; Smith-Donald, Raver, Hayes, & Richardson, 2007; Volbrecht & Goldsmith, 2010). These types of ratings are pragmatic because they do not add burden to the participant and the codes are recorded directly after the assessment, or as part of an assessment during natural breaking points. Typically, two RAs code the same child during the same visit to establish reliability, and then reliability checks are done at planned intervals. Such coding requires some training and a detailed coding manual, but is a pragmatic technique for saving time while collecting objective data. Second, we had a presentation on computer adaptive testing (CATS) by Richard Gershon, an expert on using technology to improve the impact and accuracy of measurement tools. Using tools like CATS can be beneficial in studies with a multitude of assessments because CATS minimizes assessment duration by tailoring surveys using algorithms to shorten the number of questions asked based on previous answers. Using CATS and short forms of surveys can lessen participant burden, and is a pragmatic solution for large-scale studies like HBCD.

Conclusions and Thinking Ahead: Laying the Groundwork for a Robust HBCD

In a recent systematic review of empirical infant neuroimaging studies, Azhari et al. (2020) concluded that the field of neuroscience needs larger samples across developmental periods that include under-represented populations (p. 101389). We agree, and hope that strategies laid out here contribute to the HBCD study actualizing its potential to address such imperatives. A landmark study like HBCD has the ability to reveal not only patterns of typical brain development across infancy and early childhood, but also modifiable factors even before birth that influence brain development and risk and resilience trajectories (Brown et al., in press). We argue that we must consider neurological development “in context,” and examine variables that affect concurrent and prospective development and behavior within the context of ecological settings in which development is shaped. Indeed, development in infancy and early childhood occurs within caregiving relationships and through experiences and social interactions (Fitzgerald, Weatherston, & Mann, 2011), and neurodevelopment is best understood using a multi-method approach that captures such influences over time. Moreover, in infancy and early childhood, direct assessments are necessary and are the gold standard in the field (despite added time and cost). Direct assessments will help link structural and functional changes in the brain to developmental tasks and behavior. As noted previously, there are limitations of parent report, and young children cannot serve as informants on behavior and relationships until they are older (Wakschlag et al., 2005) thus, the need for developmental coherence and direct assessments (our first 2 principles).

It is always a balancing act (and at times, a wrestling match!), when large complex consortia deliberate over the nature and timing of multi-level assessments to include in a study. As researchers, we tend to want to study all facets of a phenomena despite limited time, participant burden, and cost. Thus, using a pragmatic framework (Principle 3) like the one proposed here will help to make decisions systematically, rather than based on what has always been done without considering best and new options. On the other hand, close attention to the requisite developmental nuance in predictors and outcomes is absolutely essential to trace normative patterns as well as individual differences in process and outcome in the rapidly changing developmental landscape of early childhood. Including strengths-based assessments (Principle 4) and focusing on resilience as an outcome, as well as protective factors, is important for understating positive developmental trajectories and for framing the study for participants and community stakeholders. If we focus only on risk and negative outcomes, we are missing much of the developmental story. Examining positive trajectories and resilience promoting factors can help identify leverage points for intervention. We can also learn from other large-scale studies such as ECHO (Blackwell et al., 2018; Bush et al., 2020) and ABCD (Luciana et al., 2018). We had workgroup members with experience on such studies, and with other multi-site, longitudinal studies and their perspectives were invaluable in preparing our recommendations. Indeed, the need for varied expertise, excellent leadership, and shared engagement in making study decisions with a common vision will aid in the success of studies like HBCD. As we discussed previously, the planning phase of HBCD has allowed for the creation of shared principles and innovative ideas that will positively influence the next phase of HBCD.

It must be noted that not all working members necessarily agree with every recommendation on our proposed list of measures. Ultimately workgroup leadership had to incorporate diverse viewpoints and synthesize for decision-making. Despite this, we underscore that the workgroup unanimously endorsed the strategic principles presented in this paper. The selection of actual measures will be finalized in the next phase of the study, and selection will be based on many factors such as time, study goals, participant burden, and cost. Nevertheless, the potential for studies like HBCD to elucidate and understand risk and resilience processes across time, and to lay the foundation for primary prevention is unparalleled. Strategic assessment frameworks, like the one proposed here, will help to ensure it realizes its tremendous potential.