Converging neural and behavioral evidence for a rapid, generalized response to threat-related facial expressions in 3-year-old children.

Electrophysiological studies on adults suggest that humans are efficient at detecting threat from facial information and tend to grant these signals a priority in access to attention, awareness, and action. The developmental origins of this bias are poorly understood, partly because few studies have examined the emergence of a generalized neural and behavioral response to distinct categories of threat in early childhood. We used event-related potential (ERP) and eye-tracking measures to examine children's early visual responses and overt attentional biases towards multiple exemplars of angry and fearful vs. other (e.g., happy and neutral) faces. A large group of children was assessed longitudinally in infancy (5, 7, or 12 months) and at 3 years of age. The final ERP dataset included 148 infants and 132 3-year-old children; and the final eye-tracking dataset included 272 infants and 334 3-year-olds. We demonstrate that 1) neural and behavioral responses to facial expressions converge on an enhanced response to fearful and angry faces at 3 years of age, with no differentiation between or bias towards one or the other of these expressions, and 2) a support vector machine learning model using data on the early-stage neural responses to threat reliably predicts the duration of overt attentional dwell time for threat-related faces at 3 years. However, we found little within-subject correlation between threat-bias attention in infancy and at 3 years of age. These results provide unique evidence for the early development of a rapid, unified response to two distinct categories of facial expressions with different physical characteristics, but shared threat-related meaning.

Introduction

Threat-biased attention refers to the preferential perception of threat-related emotional signals due to their affective salience (Morales et al., 2016, Todd et al., 2012). Decades of electrophysiological studies on adults have provided evidence for a rapid detection of affectively salient facial expressions (particularly angry and fearful expressions) as well as prioritized allocation of attention to these expressions (Hinojosa et al., 2015). Threat-biased attention develops from early childhood (Bayet and Nelson, 2019), and maladaptive development of this attentional mechanism conveys risks for later socioemotional problems (LoBue and Pérez-Edgar, 2014, Pérez-Edgar et al., 2011, Pérez-Edgar et al., 2010). A well-documented behavioral change in threat-biased attention occurs in early childhood, such that from infancy to childhood there is shift from a specific bias towards fear (e.g., preferential looking to fearful faces) to a more generalized attentional bias toward threatening signals or negative emotions (e.g., anger and fear) by three years of age (Burris et al., 2019, Leppänen et al., 2007). However, little is known about the neural substrates of these behavioral changes, which in turn hinders our understanding of the brain-behavioral relationship in the development of threat-biased attention. This is attributable to a scarcity of longitudinal research on the development of neural correlates of threat-biased attention and how they are accompanied by the behavioral changes in early childhood (Fu and Pérez-Edgar, 2019).

Studies on threat-biased attention using behavioral measures have suggested differentiated developmental patterns between children’s responses to angry and fearful faces. For example, infants between 7 and 12 months of age exhibit preferential looking to fearful faces (Kotsoni et al., 2001, Peltola et al., 2009), as well as longer dwell time (DT) on fearful vs. other (e.g., angry and happy) faces, i.e., greater difficulty in disengaging from them (Leppänen et al., 2018, Peltola et al., 2013). Although an advantage in the speed of attention orienting to angry vs. happy faces has also been observed in 8- to 14-month-old infants (LoBue and DeLoache, 2010), this pattern has not been consistently observed in the first two postnatal years (Burris et al., 2017, LoBue et al., 2017) and infants do not dwell longer on angry as compared to happy faces (Leppänen et al., 2018, LoBue and DeLoache, 2010). Threat-biased attention continues to develop beyond infancy and children start to formalize a valence-based categorization of negative emotional expressions by three years of age. Specifically, children at 3 years of age and beyond show an attentional bias toward both fearful and angry faces compared to other emotions (e.g., happy and neutral) in a variety of behavioral tasks (Bayet et al., 2018, Gao and Maurer, 2009, Gao and Maurer, 2010, Burris et al., 2019). It is plausible that infants discriminate emotions relying on facial configurations and features (e.g., “toothiness” in an angry face, the white areas in fearful eyes); while older children start to categorize facial configurations into different groupings based on their appraisal of the meaning (i.e., emotion) of the configurations from 3 years of age (see (Barrett et al., 2019) for review and discussion). To better resolve this pervasive confound in developmental studies, a longitudinal investigation of emotion perception and its neural correlates with the same stimuli and experimental design is critical.

The recording of the event-related potential (ERP) is the most commonly used tool to examine the neural correlates of facial emotion processing in children. The effect of affective valence on infant brain activation has been inferred from several ERP components, including the N290, P400 and Nc components. The N290 component differentiates between faces and non-face objects in infants and has been speculated to be a precursor to the face-sensitive N170 component in adults (Halit et al., 2004, Nelson and McCleery, 2008). The Nc is a reliable neural index of child sustained attention and attention allocation (Richards, 2003). The P400 was initially identified as another face-sensitive ERP component (de Haan and Nelson, 1999), but more recent studies have shown that its functional significance is more comparable to the Nc, such that greater P400 amplitude was observed in response to salient stimuli and during sustained attention than inattention (Guy et al., 2016, Xie and Richards, 2016). Mixed results have been reported on the sensitivity of these ERP components to facial emotions in infancy. There are studies reporting a fear-bias (i.e., greater ERP amplitudes in response to fear than other emotions) in infants between 7 and 12 months (Leppänen et al., 2007, Hoehl and Striano, 2008, van den Boomen et al., 2019, Xie et al., 2019), while others have found an anger-bias (Xie et al., 2019, Kobiella et al., 2008), a happy-bias (Jessen and Grossmann, 2015), or no difference to multiple exemplars of angry and fearful relative to happy faces (Vanderwert et al., 2015).

How neural responses to facial emotions change from infancy to three years of age, when the categorization of threat-related emotions emerges, is not yet fully understood. To the best of our knowledge, only one study has tested the ERP responses to threat-related faces in 3- to 4-year-old children (N = 20), showing a greater negative-going ERP deflection to fearful relative to neutral faces at around 320 ms in the left temporal-occipital electrodes (Dawson et al., 2004). The evidence from other modalities, e.g., functional near-infrared spectroscopy (fNIRS) or functional magnetic resonance imaging (fMRI) is also scarce, perhaps due to the difficulty in testing 2- to 3-years old children with neuroimaging tools. Further, there is a dearth of studies examining how ERPs relate to looking-behavior measures, cross-sectionally or longitudinally (Fu and Pérez-Edgar, 2019). Therefore, the development of brain-behavior relationship in threat-biased attention and the consistency of this bias across measures in early childhood remain unclear.

In the current study, we investigated children’s ERP correlates of facial emotion processing along with their looking behaviors in an attention disengagement task drawing from a large longitudinal sample. Our final sample included a large group of children who participated in an emotion disengagement eye-tracking task at 5, 7 or 12 months of age (N = 272) and were re-tested at 3 years of age (N = 334); and half of them participated in an ERP experiment for facial emotion processing (N = 148 infants and 132 3-year-olds) before the eye-tracking task during the same visit. We addressed three novel questions about the development of threat-biased attention: 1) Will children’s ERP and behavioral responses to facial emotions converge on a generalized bias toward threat-related emotions (fear and anger) at 3 years of age at the group-level? 2) Whether the biases seen in infancy predict a bias toward threat in the same measures at 3 years? 3) Whether the ERP and behavioral manifestation of the threat-biased attention are correlated, consistent with the existence of a unified (or integrated) neural system? The first question was addressed by using new eye-tracking and ERP data collected at 3 years of age. The second question was addressed by combining data collected during infancy (published in (Leppänen et al., 2018, Xie et al., 2019)) with the new data collected at 3 years. To answer the third question, we applied correlation analyses and a support vector machine-regression (SVR) technique to assess whether children’s behavioral performance can be predicted by their ERP data.

Methods

Participants and design

This study involves a longitudinal cohort of children (N = 774) who visited the lab at 5 (N = 215), 7 (N = 246) or 12 (N = 313) months of age and had their second lab visit at 3 years of age (N = 408). At the first study visit in infancy, participants were randomly assigned to two equally sized groups, and underwent the assessment of either ERP or functional Near-Infrared-Spectroscopy (fNIRS) responses to facial expressions, followed by the same eye-tracking-based assessment of disengagement times. Children were kept in the same subgroups in infancy and at 3 years. The present analyses used data from the ERP subgroup for analyses examining the neural correlates of facial emotion processing. The details about the fNIRS task can be found in (Bayet et al., 2020). Data from all available participants were used for the analyses examining disengagement times as this task did not differ across the subgroups.

There were 33 children who were initially enrolled but later excluded from the study because 1) we found out later about maternal prenatal medication use (N = 11); 2) the children were diagnosed with autism (N = 18), or others clinical conditions (e.g., preexisting genetic condition N = 1; hydrocephalus N = 1; absence seizures N = 1; having a brain tumor N = 1). Thus, the remaining infants were all typically developing, born full-term and had no pre- or peri-natal complications. The families were recruited from a densely populated state of the United States. The demographic information (e.g., children’s age, sex and race/ethnicity; family combined income, and parental education) of the participants is described in Supplemental Information (SI Table 1).

The final ERP analysis for the 3-year-old children involved 132 participants with at least 10 valid trials per experimental condition (Xie et al., 2019). An additional 84 children were tested but excluded from the analysis due to fussiness (i.e., not completing the experiment), or excessive artifacts (e.g., eye and/or body movements) that resulted in insufficient number of trials. This attrition rate (38.9%) is much lower than the rate for the infant ERP analysis in the current cohort (55.4%; (Xie et al., 2019)). The final eye-tracking analyses for the 3-year-old children included 334 participants with successful calibration for > 3 points and at least 3 valid trials per experimental condition (Leppänen et al., 2018). An additional 78 participants were tested but excluded from the eye-tracking analysis for not meeting one or more of the inclusion criteria. The final sample of the infant ERP data included 148 participants (5 mos: 48; 7 mos: 49 and 12 mos: 51), and the final eye-tracking data included 272 participants (5 mos: 62; 7 mos: 109 and 12 mos: 101). Details about the infant data have been described in the two prior studies (Leppänen et al., 2018, Xie et al., 2019). The ERP and eye-tracking tasks are described in the following paragraphs.

Ethics statement

Parents of the participants provided written informed consent before each of the child’s study visits, and ethical permission for the study was obtained from the Institutional Review Board at Boston Children’s Hospital.

Stimuli and procedure

ERP task

The ERP task was a passive-viewing paradigm of emotional face stimuli (Fig. 1 a). This ERP task was implemented during both the visits in infancy and at 3 years of age. The participants were sitting (infants were sitting on their parent’s lap) at approximately 65 cm from a 17-in. Tobii T120 Eye Tracking Monitor (Tobii Technology, Dan-deryd, Sweden) in a dark room. At the visit in infancy, the participants viewed a maximum of 150 images of full intensity emotional faces taken from the NimStim Face stimulus set (Tottenham et al., 2009), with 50 images for each emotional category – angry, fearful and happy. At the 3-year visit, the children viewed the full intensity happy, angry, and fearful faces, and, additionally, images of neutral faces, as well as 40% intensity angry and fearful faces (50/category). The 40% intensity fearful and angry faces were added to examine whether 3-year-old children are able to perceive threat-related expressions when the intensity is low. The 40% intensity faces were created by morphing neutral with 100% intensity angry and fearful faces using MorphX (http://www.norrkross.com/software/morphx/morphx.php) with 160 predefined points. Distortions created by morphing were fixed in Photoshop. These stimuli have been justified in a prior behavioral study (Bayet et al., 2018). For both the infant and 3-year visits, the emotional faces in the ERP task were chosen from five female models per child, race-matched with the child’s mother, based on parent report of the mother’s expressed race. The size of the stimuli was 16.5 × 14 cm (14.3° × 11.2° vertically and horizontally) and presented on a gray background in a random order using E-prime 2.0 (Psychology Software Tools, Sharpsburg, PA). Each stimulus was presented for 1000 ms and the interstimulus interval ranged between 650 and 950 ms. The luminescence of the screen was matched for each stimulus. At the 3-year visit, the task was framed as a “Finding Nemo” game to keep the children engaged, such that a still image from the Finding Nemo movie was presented after each block of 15 trials. Children were not required to respond in any way but were told they could tell the experimenter when they saw Nemo. Attractors were additionally used when children looked away from the presentation.

Fig. 1.

ERP (A) and eye-tracking (B) paradigms.

In the ERP task at the 3 year visit, the stimuli were 100% intensity “happy”, “angry” and “fearful” faces (Tottenham et al., 2009), and 40% intensity “anger40” and “fear40” faces. In the eye-tracking task at the 3 year visit, the stimuli were 100% intensity happy, angry and fearful faces, or a matched non-face pattern (the 40% intensity expressions and neutral faces were not used in the eye-tracking task).

Eye-tracking task

After the ERP task (or fNIRS task for participants not in the ERP subgroup), an eye-tracking task designed to assess attention disengagement times was administered (Fig. 1 b). The same eye-tracking paradigm was used for the visits in infancy and at 3 years of age, with slightly different parameters as described below. The child was sitting in the same place for the attention disengagement task. The same Tobii eye tracker and Eprime 2.0 software were used for stimulus presentation. Eye tracker calibration was conducted at the beginning of the task. Each trial started with a dynamic attention-grabbing stimulus on the center of the screen. Once the child’s fixation was on the attention grabber, a stimulus was presented on the center of the screen for 4000 ms. This center stimulus was randomly chosen from four types of images, a non-face pattern, angry, fearful or happy faces (1 model/child). The decision to include happy but not neutral faces in the eye tracking assessment was motivated by the need to keep the task short enough for children and based on prior studies consistently showing that disengagement does not typically differ for happy and neutral faces in infants (e.g., (Peltola et al., 2018)). Thus, to make it compatible with the infant ERP paradigm, happy faces were included as a “baseline” condition for the DT measure for angry and fearful faces. A second (target) stimulus (13.0° × 3.5°) was presented with a 1000 ms (infants) or 200 ms (three-year-olds) onset asynchrony laterally on the left of right size of the screen with 13.6° eccentricity and remained on the screen for 300 ms. This target stimulus was a geometric shape that could be either a black and white vertically arranged checkboard pattern, e.g., circles, lines, or diamonds (infants) or a colorful vertically arranged pattern, e.g., circles and pentagons (three-year-olds). These age-specific changes were made to the task to render it more attractive for the 3-year-old group.

EEG recording, processing and measurement

Continuous scalp EEG was recorded from a 128-electrode HydroCel Geodesic Sensor Net (HGSN; Electrical Geodesic Inc.) and references online to a single vertex electrode (Cz). Channel impedances were kept at or below 100 kΩ and signals were sampled at 500 Hz.

The infants’ and 3-year-olds’ EEG recordings were preprocessed in MATLAB (R2018a, the Mathworks, Inc.) following the same processing stream, which has been justified in recent EEG/ERP studies (Xie et al., 2019, Xie et al., 2019) and involves functions adopted from the MADE pipeline (Debnath et al., 2020). The continuous EEG data were filtered using a Hamming windowed FIR filter with a passband of 0.3–30 Hz. The filtered data was then segmented into 1-s epochs (trials) with 100 ms pre- and 900 ms post-stimulus onset. Independent component analysis (ICA) was conducted, and then SASICA (Chaumon et al., 2015) was used to identify and remove artificial components that are related to eye movements, blinks, and focal activity. The EEG epochs were then inspected for artifacts (EEG > 100 μV or EEG < −100 μV). Channel interpolation was conducted using a spherical spline interpolation if there were fewer than 18 (15%) electrodes that were missing or had bad data (Luyster et al., 2014, Righi et al., 2014). There were 7.74 (SD = 2.95) channels interpolated for each epoch on average per participant. In addition, children’s looking behavior was judged offline trial by trial based on the video recording – whether she/he was fixating on the stimulus presentation for the first 500 ms without of an eye movement or blink. Trials were excluded from further analysis if this criterion was not met. The processed data were re-referenced to the average reference. These artifact detection and rejection criteria were the same between infant and 3-years-old ERP analyses. The mean number of “clean” trials was not different among the 6 emotional categories and is listed as following: anger (M = 22.68, SD = 10.59), fear (M = 22.68, SD = 10.05), happy (M = 22.35, SD = 10.09), anger40 (M = 22.91, SD = 10.01), fear40 (M = 22.69, SD = 10.29), neutral (M = 22.72, SD = 10.00).

The HGSN electrodes were grouped into 9 virtual clusters to cover the scalp regions that are commonly used to examine the N290, P400 and Nc components in infants and young children (Fig. 3 a): (Nc: “Frontal_Z (F_Z)”, “Central_L (C_L)”, “Central_Z (C_Z)” and “Central_R (C_R)”; N290: “Temporal-Occipital_L (TO_L)”, “Occipital-Inion_L (OI_L)”, “Occipital-Inion_R (OI_R)”, and “Temporal-Occipital_R (TO_R)”; and P400: “TO_L”, “OI_L”, “Occipital-Inion_Z (OI_Z)”, “OI_R” and “TO_R”). These channel clusters were the same as those used in (Xie et al., 2019). The peak latency and amplitude of the N290 component were assessed in the time window of 190 to 350 ms. The N290 peak amplitude was corrected for the pre-N290 positive peak as what was done in (Xie et al., 2019) to reduce the effect of positive or negative trends on the child N290 component (Guy et al., 2016, Kuefner et al., 2010). The peak latency and mean amplitude of the P400 component, as well the mean amplitude of the Nc component, were assessed in the time window of 350 to 650 ms based on previous ERP studies on face and emotion processing in children (e.g., (Leppänen et al., 2007, Kobiella et al., 2008, Vanderwert et al., 2015, Conte et al., 2020)). The Nc component does not have a distinguishable (pronounced) peak; therefore, its peak latency was not assessed (this is consistent with most child ERP studies).

Fig. 3.

A. Electrode clusters used for assessing the components; B. Mean ERP amplitude difference between threat- (anger and fear) and non-threat-related (happy and neutral) emotions in the N290 and P400/Nc time windows. Paired t-tests were conducted for each electrode (N = 124) on the EEG net, and all p-values were then adjusted with an FDR of 0.05. The asterisks mark the electrodes that showed a significant difference between the threat and non-threat conditions after the FDR adjustment. The posterior yellow (“threat” > “non-threat”) area is the “P400 region”, and the central and frontal blue (“threat” is more negative than the “non-threat”) area is the “Nc region”; C. ERP waveforms in response to different emotions;

Eye-tracking processing and assessment

The eye-tracking data for infants and 3-years-old children were processed with a validated pipeline (Leppänen et al., 2015) in MATLAB (R2018a, the Mathworks, Inc.). Trials were retained for further analyses when all of the following preset criteria were met: 1) a sufficient fixation on the central stimulus (> 70% of the time) during the time preceding gaze shift or the end of the analysis period (i.e., 1000 ms after the lateral stimulus onset); 2) sufficient number of valid samples in the gaze data so that there were no gaps longer than 200 ms; 3) valid information about the eye movement from the central to the lateral stimulus (i.e., the eye movement did not occur during a period of missing gaze data). This combination of criteria has been justified to provide valid eye-tracking measures in previous studies with children (Leppänen et al., 2018).

The variable, DT, was calculated as the duration of fixation dwell time on the central face before the saccade to the lateral target between 150 ms and 1000 ms following the target onset, and then converted to a normalized dwell time index score. The value of the index varies between 0 (saccade at 150 ms after the onset) and 1 (no saccade by 1000 ms after the onset). This DT index is comparable to the measure of saccade latency with the exception that DT does not exclude trials without gaze shift, which makes is preferable for young children given that the probability of no gaze shifts among them is relatively high than older children and adults (Leppänen et al., 2015). We tested the correlations between DT and saccade latency for 3 years of age and found they were highly correlated for all three conditions: angry (r(266) = .824, p < .001), fearful (r(266) = .830, p < .001) and happy (r(266) = .825, p < .001) faces.

Machine learning analysis

We applied a linear SVR method to determine if behavioral anger- and fear-bias scores could be predicted by the corresponding bias-scores of ERP measures (amplitudes and latencies). A five-fold cross-validation was performed. Specifically, the participants were randomly divided into 5 folds (cohorts). For each round, 4 folds were used as the training set and the remaining one was treated as the testing set. As a result, each participant’s DT (anger or fear) bias-score was predicted once. This five-fold cross-validation was then repeated for 100 times, swapping which participants were used for training and testing. Each participant ended up with 100 predicted bias-scores from the permutations, which were then averaged to obtain the final predicted scores. Finally, Pearson correlations were used to determine whether the predicted bias-scores were correlated with the observed (real) bias-scores for the participants (Finn et al., 2015). The weights (relevance) of the features were calculated with following formula and averaged across the 100 permutations: weight(w) = sv_coefT × SVs, in which sv_coefT is the transformed support vector coefficients and SVs are the support vector matrix from the training model. This technique will also allow us to interrogate the machine learning models to understand what (temporal and spatial) aspects of the ERP data may be particularly important for predicting children’s behavioral performance.

Results

Question 1: behavioral and ERP indices of threat-biased attention at group-level

We first addressed the question of whether children’s ERP and behavioral responses show evidence for a generalized bias toward threat-related emotions (i.e., fear and anger) at 3 years of age.

Eye-tracking analysis

The final analysis of the averaged dwell time (DT) toward the central face before a saccade to the target, included 334 36-month-old children (F = 151). A repeated-measures ANOVA was performed to test differences in DT as a function of emotion condition (within-subject factor: happy, fearful and angry faces) at 36 months. There was a main effect of emotion, F (2,331) = 33.498, p < .0001, ηp2 = .092. Post-hoc comparisons with a false discovery rate (FDR) of 0.05 adjustment using the Benjamini & Hochber procedure (Benjamini et al., 2001) showed that the DTs on angry and fearful faces were longer than the DTs on happy faces (Fig. 2). Since the children either participated in an ERP or fNIRS experiment before the eye-tracking task, we also included prior experiment (between-subject factor: ERP or fNIRS) in the ANOVAs. No interaction was found between prior experiment and emotion condition for DT, F (2,331) = .146, p = .451. Thus, the eye-tracking data for both cohorts were pooled in subsequent analyses.

Fig. 2.

Dwell times (DT) for facial expression in infancy (data reanalyzed from (Leppänen et al., 2018)) and 3 years of age. The y-axis refers to the ratio of the DT on the central face before the saccade to the lateral target between 150 and 1000 ms following the target onset to 1000 ms. The value of the index varies between 0 (saccade at 150 ms after the onset) and 1 (no saccade by 1000 ms after the onset). Multiple comparisons were adjusted with an FDR of 0.05. * adjusted p < 0.05, ** p < 0.01, *** p < 0.001. The 5, 7, and 12-month data have been reanalyzed from

The infant eye-tracking data has been published in a prior study (Leppänen et al., 2018), but they were analyzed with different statistical methods. In the current study, we re-analyzed the infant data with a repeated-measures ANOVA to test DT as a function of age (between-subject factor: 5, 7 and 12 months) and emotion (within-subject factor: happy, fearful and angry faces). Results regarding the interaction of age and emotion condition and the main effect of age are reported in the SI document (also see Fig. 2).

ERP analysis

The ERP measures at 36 months (N = 132, F = 70) were analyzed with repeated measures ANOVAs involving two within-subject factors: emotion condition (6 levels: “angry”, “angry40”, “fearful”, “fearfull40”, “happy” and “neutral”) and channel cluster (4 clusters for the N290 and Nc, and 5 clusters for the P400; see the Method section). “Angry40” and “fearful40” refer to angry and fearful expressions with 40% intensity; “angry”, “fearful” and “happy” refer to facial expressions with 100% intensity (which are the same three types of stimuli used in the eye-tracking task); and “neutral” refers to neutral faces. The ERP components for the “angry”, “fearful”, “happy” and “neutral” conditions are illustrated in Fig. 3, and the plots and comparisons for all 6 conditions can be found in Supplemental Information (SI) Figs. 1 and 3.

N290 results: The analyses of the N290 amplitude and latency showed no significant main effects of emotion condition (amplitude: F (5,127) = 1.556, p = .171, ηp2 = .012; latency: F (5,127) = 1.335, p = .247, ηp2 = .010) or its interaction with channel cluster (amplitude: F (5,127) = .253, p = .939, ηp2 = .002; latency: F (5,127) = .717, p = .611, ηp2 = .005) (Figs. 3 & 4).

Fig. 4.

Bar graphs for ERP amplitudes (A) and latencies (B) by emotion at 36 months. Multiple comparisons were adjusted with an FDR of 0.05. * adjusted p < 0.05, ** p < 0.01, *** p < 0.001.

P400 results: The analysis of the P400 amplitude revealed a main effect of emotion condition, F (5,127) = 4.449, p = .001, ηp2 = .033. Post-hoc comparisons showed that the P400 amplitude was greater in response to angry and fearful faces with 100% intensity (i.e., angry and fearful) compared to the other 4 conditions (angry40, fearful40, happy and neutral) (Figs. 3 and 4). There was no difference found among angry40, fearful40, happy and neutral conditions (SI Figs. 1 and 3). There was also a main effect of channel cluster, F (2.769,128) = 263.2, p < .001, ηp2 = .668, with the Occipital-Inion clusters showing greater P400 amplitudes than the Occipital-Temporal clusters (Fig. 3). The analyses of the P400 latency also showed a main effect of emotion condition, F (5,127) = 5.143, p < .001, ηp2 = .038, as well as same patterns of differences between conditions, such that compared to the other four facial expressions, angry and fearful expressions of 100% intensity elicited longer P400 latency (Figs. 3 and 4; SI Fig. 3). There was no difference found among angry40, fearful40, happy and neutral conditions (SI Figs. 1 and 3).

Nc results: The analysis of the Nc amplitude revealed similar results than the P400 analysis, such that there was a main effect of emotion condition, F (4.637,127) = 10.23, p < .001, ηp2 = .072. Compared to the angry40, fearful40, happy and neutral facial expressions, angry and fearful expressions of 100% intensity elicited greater Nc amplitude (Figs. 3 and 4; SI Fig. 3).

Interaction between emotion and channel cluster.

There were no interactions between emotion condition and channel cluster for the P400 amplitude, latency and the Nc amplitude, ps > .05. The SI Fig. 2 shows the topographical maps for the ERP activation from stimulus onset to 800 ms following the onset with a 50 ms step, illustrating the scalp distribution and the changes of these ERP components.

Re-analysis of infant ERP data.

The infant ERP data used in the current study have been reported in a prior study (Xie et al., 2019). Herein, we reprocessed them with the processing pipelines used for the 36-month-old data in order to perform the longitudinal analyses reported in the next section. Results are reported in the SI document.

In sum, we found that children’s eye-tracking and ERP (the P400 and Nc) responses converged on a general pattern of attentional bias towards threat-related emotions at group level at 3 years of age. There was no difference between the ERP responses to anger and fear at this age. The biased responses to angry and fearful faces were not only distinguished from happy faces, but from neutral faces and negative emotions with low intensities as shown in the ERP results.

Question 2: correlations between within-subject changes in threat-biased attention

We further tested whether the magnitude of the bias towards threat-faces in infant (5, 7 and 12 months) behavioral and ERP responses correlated with that at 36 months of age. Anger- and fear-bias scores in the behavioral and ERP measures (N290 and P400 amplitudes and latencies, Nc amplitude) were respectively calculated with the following formulae for each participant to obtain normalized differences between conditions: Anger-bias score = (Anger − Happiness) / (Anger + Happiness); Fear-bias score = (Fear − Happiness) / (Fear + Happiness). The bias-scores for each ERP component were calculated for each individual channel cluster (used in machine learning tests) and also were averaged across channel clusters (to reduce the number of tests in the following regression and correlation analyses). The absolute values of the N290 and Nc amplitudes were used, and thus for all behavioral and ERP measures, greater (more positive) bias scores represent a greater magnitude of the bias towards fear or anger.

Pearson’s Correlation analysis showed that behavioral (DT) anger- and fear-bias scores were positively correlated within a testing session in infancy, r(256) = .576, p < .001, and at 36 months of age, r(332) = .590, p < .001. Since the anger- and fear-bias showed different patterns in infancy, we did not combine them for each individual to get a composite “threat-bias” index; rather, we analyzed them separately. The analysis of correlations between the anger- and fear-bias scores for the ERP features also showed that they were moderately correlated: 1) N290 amplitude (in infancy and at 36 months respectively): r(148) = .281, p < .01, r(132) = .594, p < .001; 2) P400 amplitude: r(148) = .427, p < .001, r(132) = .423, p < .001; 3) Nc amplitude: r(148) = .616, p < .001, r(132) = .412, p < .001; 4) N290 latency: r(148) = .443, p < .001, r(132) = .527, p < .001; 5) P400 latency: r(148) = .442, p < .001, r(132) = .450, p < .001.

There were 136 participants with clean DT data both in infancy and at 36 months of age, and there were 50 subjects with clean ERP data both in infancy and at 36 months of age. Regression analyses with FDR correction for the p-values revealed no association between the bias scores in infancy and at 36 months for neither the behavioral (fear-bias_DT: β = −.076, p = .816; anger-bias_DT: β = −.192, p = .200) or the ERP measures (fear-bias_N290: β = −.022, p = .882; fear-bias_P400: β = −.055, p = .882; fear-bias_Nc: β = .263, p = .260; anger-bias_N290: β = −.095, p = .816; anger-bias_P400: β = −.116, p = .816; anger-bias_Nc: β = .036, p = .882). The beta values reported here were standardized, and the p-values were adjusted with FDR of .05. These results indicate that there is little within-subject stability in the neural and behavioral manifestation of the threat-bias in early childhood. We further tested the possibility of a moderation effect of age at testing (i.e., 5, 7 or 12 months) on the relation between the threat-bias scores in infancy and at 36 months of age. Analyses revealed no difference among the three infant age groups, ps > .05.

Question 3: brain-behavior relationship in threat-biased attention across ages

The last set of analyses addressed the question of whether there is brain-behavior relationship in threat-bias in infancy and at 36 months of age. We first calculated the correlations between the behavioral and ERP bias scores (averaged across channel clusters), separately for infants and 36-month-old children. The FDR adjusted p-values are reported here. The analyses for infants’ data revealed no correlation between ERP and behavioral bias-scores. Specifically, the anger-bias score for DT was not correlated with that for the 1) N290 amplitude, r(63) = .079, p = .540; 2) N290 latency, r(63) = −.099, p = .440; 3) P400 amplitude, r(63) = −.046, p = .720; 4) P400 latency, r(63) = .162, p = .203; or 5) Nc amplitude, r(63) = .139, p = .278; and the fear-bias score for DT was not correlated with the ERP measures either: 1) N290 amplitude, r(63) = −.059, p = .648; 2) N290 latency, r(63) = .085, p = .506; 3) P400 amplitude, r(63) = −.236, p = .063; 4) P400 latency, r(63) = .−.015, p = .907; or 5) Nc amplitude, r(63) = .063, p = .622.

In contrast, the analyses for the 36-month-olds’ data showed significant correlations between the ERP and behavioral bias scores. The anger-bias score for DT was correlated with the anger-bias scores for the N290 (r(108) = .238, adjusted p = .033), P400 (r(108) = −.201, p = .045) and Nc (r(108) = −.237, p = .032) amplitudes and the N290 latency, r(108) = .207, p = .045); and the fear-bias score for DT was correlated with the fear-bias score for the N290 amplitude (r(108) = .325, p = .005).

We further investigated whether children’s anger- and fear-bias scores in ERP measures could respectively predict the behavioral anger- and fear-bias scores using a linear SVR model (LIBSVM toolbox v3.24) (Chang and Lin, 2011). A five-fold cross-validation was performed and permutated for 100 times, swapping the participants assigned to the training and testing groups. The bias-scores for ERP amplitudes and latencies in different channel clusters were used as predictive features. Finally, Pearson correlations were used to determine whether the predicted bias-scores of the behavioral measures using the ERP bias-scores, were correlated with the observed (real) scores (Finn et al., 2015). We found that anger- and fear-bias scores for ERP amplitudes (13 features) respectively predicted the behavioral anger (r(108) = .510, p < 10e−7) and fear-bias (r(108) = .474, p < 10e−6) scores (Fig. 5). The prediction accuracy was lower when ERP latencies were used (9 features) [anger-bias: (r(108) = .302, p = .0014) and fear-bias: (r(108) = .280, p = .0031)], and the combination of ERP amplitudes and latencies (22 features) resulted in comparable results with ERP amplitudes only [anger (r(108) = .502, p < 10e −7) and fear-bias (r(108) = .454, p < 10e −6)].

Fig. 5.

(A). Machine learning results. These figures showed that the behavioral bias-scores predicted with the bias-scores in ERP amplitudes are significantly correlated with the observed behavioral bias-scores. (B). Weights in the machine learning (SVR) training models for different ERP features in different channel clusters (see method section for these acronyms). Since we repeated the five-fold cross validation 100 times, the violin plots show the distribution of the weights across repetitions.

The weights (“relevance or importance” of the ERP features in SVR prediction) for each ERP feature are shown in Fig. 5 B. In the model of predicting individual behavioral anger-bias scores, the N290 amplitudes in three out of the four channel clusters (OI_L, OI_R and TO_R) showed positive weights significantly larger than zero, and the N290 latencies in OI_L and TO_R also showed high positive weights in predicting the anger-bias scores. Similar patterns have been shown for the N290 features in the model predicting behavioral fear-bias scores. This means that children who showed (relatively) greater N290 responses to angry and fearful compared to happy faces were more likely to show longer attentional DT on these threat-related faces. In contrast, the P400 and Nc amplitudes in some of the Occipital-Inion (e.g., OI_L and OI_Z) and frontal-central clusters (C-R and F_Z) showed negative weights bigger than 0.5, i.e., were more important, in predicting the anger-bias scores. However, the patterns shown in the P400 and Nc channel clusters were less consistent than those for the N290 clusters. Future research may conduct source localization analysis to investigation the contribution of different brain regions to the prediction of child threat-bias behaviors.

Discussion

This study is the first to investigate the development of brain-behavior relationship in child emotion processing and threat-biased attention. We utilized ERP and eye-tracking measures, as well as machine learning techniques, to achieve three research objectives. Our results indicate that a) child brain and behavioral responses to facial emotions converge on a pattern of attentional bias toward threat-related emotions (fear and anger) by 3 years of age (with no differentiation between or bias towards one or the other of these expressions); b) there is little correlation between the magnitude of the threat-biased attention in infancy and at 3 years of age, suggesting substantial changes at the individual level; c) brain-behavioral relationship is well established by 3 years of age, as children’s ERP data predicted the degree of threat-bias in their behavioral performance. However, this relationship is not apparent in the first year of life.

The present study demonstrates that child behavioral and brain responses to anger and fear are differentiated during infancy but converge on a comparable pattern by three years of age. As was first reported 40 years ago (Nelson et al., 1979) and numerous times since then (e.g., (Peltola et al., 2013)), the behavioral findings showed a clear fear-bias emerged at 7 months of age (Fig. 1), which is consistent with the N290 changes in the right hemisphere as reported in a prior study ([31]; also see SI Fig. 4). In contrast, there was no anger-bias in the eye-tracking measure in infancy, a finding consistent with prior behavioral studies that have also reported a lack of anger-bias in the first 2 years of life (LoBue and DeLoache, 2010, Burris et al., 2017). However, infants did show greater P400 and Nc amplitudes in response to angry than fearful and happy faces (Xie et al., 2019) (SI Fig. 4). Together, these results suggest there is a developmental change in the coding of facial expressions during the first three years of life. One possibility is that children may first discriminate facial expressions based on low-level (e.g., featural) information in infancy (e.g., eyes in fearful faces) instead of broader affective categories. Subsequently, children begin to code facial expressions on the bases of their affective meaning, and this is when they begin to exhibit commonalities in their neural and behavioural responses to physically distinct exemplars of threat-related facial expressions (see (Barrett et al., 2019)for discussion).

Several accounts since early 1990s have suggested that threat-biased behaviors indicate heightened allocation of attention to threat-relevant signals and that cognitive appraisal of the negative emotional expressions takes a longer time and more processing resources than that of positive and neutral emotional expressions (e.g., (Morales et al., 2016, Buss et al., 2019, Taylor, 1991)). The current ERP results provide direct empirical evidence for this hypothesis by showing elevated P400 and Nc amplitudes and longer P400 latency in response to anger and fear compared to the other types of emotions at 3 years of age (Figs. 3–4). The P400 and Nc components in early childhood have been associated with attention allocation and sustained attention (Guy et al., 2016, Xie and Richards, 2016), and their cortical sources have been localized to brain regions (e.g., posterior and anterior cingulate cortex) related to brain alertness and attention monitoring (Xie et al., 2019, Posner et al., 2014, Reynolds and Richards, 2005). The elevated activation in attention-related ERP components suggests that the emergence of threat-biased processing does not only rely on a discrete development of the face and emotion networks. Instead, it may reflect the onset of functional connections across circuits involved in emotion processing, e.g., between emotion and attentional networks (Casey et al., 2016, Leppanen and Nelson, 2009). It is noteworthy that in our study the 3-year-old children did not show a threat-bias in their ERP responses when negative expressions were presented with 40% intensity (SI Fig. 3). This finding is consistent with a prior behavioral study showing that unlike adults, 3 years old children could only accurately categorize negative emotions and understand the emotional meaning when the intensity was above 60–80% (Bayet et al., 2018), meaning that emotion perception is still undergoing refinement during development at this age. One limitation of the current study was that there were no facial expressions with low intensities (< 100%) applied in the eye-tracking task. Future research may test whether eye gaze patterns (i.e., measured via eye-tracking paradigms) to threat-related facial expressions also change with intensity in children at this age.

Previous studies with infants and young children have shown little correlation between within subject changes in threat-biased attention (Burris et al., 2017, Peltola et al., 2018, Yrttiaho et al., 2014). For example, no correlation in fear-bias was found between 5 and 7 months or between 7 and 24 months in two longitudinal studies, meaning that infants who showed a fear-bias at earlier ages might not show it later on and vice versa[38, 58]. The present study, with its large longitudinal sample, provides converging evidence for the non-correlated developmental changes in threat perception in early childhood. Our finding and others together highlight the substantial changes in child behaviors and neural circuitries related to threat perception at individual level, as many psychological processes have developmental trajectories that vary across individuals (Molenaar, 2004). However, these null results should not be interpreted as ruling out the existence of a link between the behavioral and neural indices of threat-bias attention in infancy and older ages. The lack of association may also reflect the unreliability of the measures used to assess attentional biases. Behavioral and EEG measures used in threat-biased attention tasks (e.g., the dot-probe attention task) are known to suffer from low internal reliability and test-retest stability as reported in a prior study with adults, although the internal stability was found to be better for the ERP indicators of threat bias (Kappenman et al., 2014). While we cannot rule out the possibility that similar factors attenuated the associations in the current study as well, it is worth noting that the fear and anger bias scores were significantly correlated in the current study, and this was true for both behavioral and ERP measures. This raises the possibility that the measures used in children have better internal stability than those used in adults, and that low reliability may not be the only reason for the absence of longitudinal associations in the measures. There were only fifty children who eventually had valid ERP data in infancy and at 3 years of age, which might also have limited the power for us to detect longitudinal correlations between the ERP features of threat-biased attention in childhood.

The current findings provide novel insights into the development of brain-behavior relationship in threat-biased attention in the first three years of life. The brain–behavior relationship was strongest at 3 years of age, suggesting that children’s behavioral responses to facial emotions and their neural substrates become more stable and reliable with age. However, this by no means suggests that there is no brain-behavior relationship in infancy. Instead, the current findings might imply that the relationship is still developing and difficult to detect in infancy, potentially due to the large amount of variability in infant data and the dramatic changes in the patterns of their behavioral and brain responses to facial emotions.

Children’s looking time toward facial emotion is likely to be determined by both the early registration of visual and affective salience, and subsequent prioritization of attention to specific expressions. This is suggested by the analyses showing that the bias towards threat in the N290, P400, and Nc components was correlated with the magnitude of children’s behavioral bias scores. The bias in the N290 component (a neural index of the early visual processing or registration of a face) was the strongest individual feature in predicting the bias in overt attentional dwell time for threat-related emotions. To our knowledge this is the first study showing a correspondence in the N290 and overt attentional measures in early childhood, consistent with similar results in adults (Zhang et al., 2012) and the hypothesis that a rapid enhancement of activity in the visual representation areas may be one of the key mechanisms mediating attentional biases in humans (Vuilleumier, 2005). The positive association between the behavioral threat-bias score and the N290 latencies suggests that children who take a relatively longer time to register or initially process negative emotions (i.e., anger and fear) tend to have a longer disengagement latency. It is worth noting that there was no emotion effect on the N290 response at the group-level, which highlights the value of examining individual differences to identify the neural processes that are important in mediating children’s behavioral biases. In contrast, the behavioral bias score was found to be negatively associated with that for the P400 and Nc components, which means that children who allocates less attention to the negative compared to positive emotions might need a longer time to fully process the face, i.e., have a longer looking time toward the negative emotions (Colombo and Mitchell, 2009). Future research may justify this hypothesis with older children using simultaneously recorded ERP and eye-tracking measures.

Increased attentional biases toward threat in childhood has been associated with later problems in socioemotional development (e.g., anxiety disorders and behavioral inhabitation) (LoBue and Pérez-Edgar, 2014, Pérez-Edgar et al., 2011, Pérez-Edgar et al., 2010). However, it remains unclear whether this behavioral pattern reflects differences in cognitive style that predispose individuals to social-emotional disorders, or just current emotional state. Future research should investigate whether heightened attentional biases for threat at 3 years or even in infancy conveys risks for later socioemotional problems. This kind of work will further shed light on the targets for early interventions for anxiety- and depression-associated behaviors.

In summary, the current study systemically demonstrates clear differences in neural and behavioral responses to facial expressions between infancy and three years of age in a large longitudinal sample. Findings reported here suggest that child emotion perception and its ERP correlates undergo substantial changes from infancy to 3 years of age, and threat-biased attention changes from a specific bias towards fear in infancy to a more generalized bias toward both negative emotions (i.e., anger and fear) at 3 years of age. This study additionally provides the first empirical evidence for brain-behavior relationship in child threat-biased attention and how it develops with age. These findings may pave the way for future research to assess whether deviated development of threat-biased attention in childhood will lead to socioemotional problems and anxiety-related disorders later in life.