Rapid and multiple-stage activation of the human amygdala for processing facial signals.

Human faces transmit multiple valuable signals, and neuroimaging studies have shown that the amygdala is active in response to facial stimuli. However, little has been known about the temporal profile of amygdala activation during facial signal processing until recently. Here we review three recent studies conducted by our group in which we recorded amygdala intracranial electroencephalography in humans. The subjects were engaged in tasks that required automatic processing of faces, eye gazes and emotional expressions. Time-frequency statistical parametric mapping analyses revealed that the amygdala showed gamma-band activation in response to emotional expressions, gazes and faces, with peak latencies at about 100 ms, 200 ms and 250 ms, respectively. These results suggest that: (1) the amygdala performs multiple-stage processing in response to these facial signals using different visual input routes, and (2) amygdala activation for processing all of these facial signals is rapid, which could be prior to or simultaneous with conscious awareness of faces.

Introduction

Human faces transmit multiple signals, such as faceness, eye gaze and emotional expression, all of which indicate valuable information during social interactions. Under evolutionary pressures, decoding of such facial signals would have helped humans take immediate collective action to tackle biologically significant events, such as encountering predators. Consistent with this notion, behavioral studies have demonstrated that the processing of facial signals is rapid and efficient. For example, faces were detected more rapidly than other stimuli,, eye gazes automatically triggered attentional shifts, even without awareness, and emotional facial expressions were efficiently detected, and processed without conscious awareness.,

Several neuroimaging studies have explored the neural mechanisms involved in implementing these rapid facial signal processes. These studies have reported that, in addition to the visual cortices such as the superior temporal sulcus and fusiform gyrus, the amygdala is activated by facial stimuli. For example, the amygdala showed stronger activity while subjects viewed faces compared with control stimuli such as mosaics,, the amygdala became active in response to others’ gazes, even when the gaze stimuli were presented without awareness and the amygdala was more activated by emotional facial expressions than by neutral expressions,, even in response to subliminally presented stimuli.,

Based on these findings, some researchers have speculated that the amygdala plays a crucial role in the rapid processing of facial signals. For instance, the amygdala has been proposed to rapidly process faces, eyes and emotional expressions. This appears to be consistent with animal data, suggesting that the amygdala has an phylogenetic history associated with detecting biologically important signals., This hypothesis is also in accordance with anatomical evidence that the amygdala receives visual input via subcortical pathways involving the pulvinar and superior colliculus, bypassing the cortical visual areas.,

However, empirical data on the temporal aspects of face processing in the amygdala has been lacking. Hence, it has remained unclear whether the amygdala can be rapidly activated in response to facial stimuli and, if so, whether the amygdala can process multiple facial signals following a similar temporal profile. Although, as mentioned above, some neuroimaging studies reported amygdala activation in response to subliminal faces,,, brain activation to briefly presented stimuli might occur at later processing stages. Anatomical evidence of subcortical input also does not decisively support rapid amygdala activation because the amygdala also receives visual input via the cortical visual areas., Some magnetoencephalography (MEG) and electroencephalography (EEG) studies have reported temporal information regarding amygdala activation during facial signal processing., However, uncertainty regarding whether the activity of deep, complex brain structures such as the amygdala can be estimated from scalp signals remains.

Information about the timing of amygdala activity during face processing is valuable because it can confirm or disconfirm the hypothesis that the amygdala rapidly processes facial signals. Furthermore, because several MEG/EEG studies have reported the temporal activity profiles of other brain regions related to face processing (e.g., the first face-specific EEG component occurs in the cortical visual areas at 150–200 ms), such information about the amygdala could provide clues about its functional relationships with other brain regions. Additionally, because several MEG/EEG studies have revealed relationships between the temporal stages of brain activity and psychological functions (e.g., brain activity related to the conscious perception of visual stimuli occurs at about 200–300 ms), the temporal information on amygdala activity would be suggestive of its psychological correlates.

Intracranial EEG recordings provide direct information regarding the temporal profile of amygdala activity. Although a few previous intracranial EEG studies have examined this issue, these studies did not fully analyze the time-frequency domains., We recently conducted a series of intracranial EEG studies in which we recorded amygdala activation in response to faces, eye gazes and emotional expressions. Here we review the results of these studies. Based on our findings, together with data from other anatomical, neuroimaging and electrophysiological studies, we propose a model of the temporal profile of amygdala activation responsible for rapid processing of facial signals.

Our Studies

We conducted a series of experiments with six patients suffering from pharmacologically intractable focal epilepsy. The patients received intracranial implantation of electrodes for preoperative investigation. Anatomical assessments using magnetic resonance imaging confirmed that the electrodes were successfully placed into the amygdala of all subjects. Field-potential data recorded in the amygdala were analyzed using statistical parametric analyses for time-frequency maps. We hypothesized that significant activation in response to facial signals would occur in the gamma frequency band (> 30 Hz), because a previous study demonstrated that this frequency was relevant for assessing amygdala activity.

In the first study, we tested amygdala activity while subjects observed faces (Fig. 1, upper left). As control stimuli, mosaics and houses were also presented. To test the inversion effect of faces, upright and inverted images were presented. Subjects were engaged in a dummy task to detect infrequently presented target stimuli. The results showed that the amygdala showed significant gamma-band activity, peaking around 250 ms, in response to both the contrasts between faces and mosaics and those between faces and houses, irrespective of whether the images were upright or inverted (Fig. 1, upper right).

Figure 1. Examples of stimuli (left) and statistical parametric maps of the gamma-band range in which we observed significant activation (right) in studies 1 (upper), 2 (middle) and 3 (lower). Blue crosses indicate the location of activation foci.

In the second study, to investigate amygdala activity in response to eye gazes, facial stimuli depicting only the eye region were presented (Fig. 1, middle left). To investigate the effect of gaze direction, averted and straight gazes were used. Mosaics made from these stimuli were used as control stimuli. As in the first experiment, the subjects detected dummy targets. The comparison between eyes and mosaics, regardless of their directions, showed a significant cluster of gamma-band activity peaking at 200 ms (Fig. 1, middle right).

In the third study, we examined amygdala activity when subjects viewed fearful, happy and neutral facial expressions (Fig. 1, lower left). The subjects were asked to discriminate the gender of the faces. The contrast between fearful and neutral facial expressions revealed a significant gamma-band activation in the amygdala at 50–150 ms with a peak at 135 ms (Fig. 1, lower right).

Discussion

Our results demonstrated the different temporal profiles of amygdala gamma-band activity in response to faces vs. mosaics/houses, eye gazes vs. mosaics and fearful vs. neutral facial expressions. Specifically, the results showed that the amygdala first processes emotional facial expressions, at about 100 ms; it then processes eye gazes at about 200 ms; finally it processes faces, at about 250 ms. These data indicate that the amygdala performs facial signal processing at multiple time stages.

These results raise the possibility that the amygdala processes different facial signals based on information from different visual routes. Anatomical studies have indicated that, as mentioned above, the amygdala receives visual inputs via the cortical visual areas related to face processing,, as well as the subcortical routes including the pulvinar and superior colliculus., Regarding the processing of emotional expressions, a previous neuroimaging study in a blindsight patient showed that the amygdala was activated while the patient viewed emotional facial expressions even after damage to the entrance of the cortical visual areas. Regarding the processing of faces and gazes, several previous MEG/EEG studies demonstrated that the face-specific visual analysis in the visual cortices occurred first during 150–200 ms., Some MEG/EEG studies also showed that cortical visual area activity at this time stage reflected the processing of eye gazes. Taken together, these findings suggest that the amygdala performs the processing for emotional facial expressions using coarse visual inputs via the subcortical pathways, and processes eye gazes and faces with fine-grained visual information processed through the cortical visual areas (Fig. 2, left and center).

Figure 2. Our hypothetical model of the temporal profile of amygdala activation involved in facial signal processing. (Left) At about 50–150 ms, the amygdala processes emotional expression in faces using visual information transmitted via the subcortical visual structures. (Center) At about 200–250 ms, the amygdala processes eye gaze and face using visual information from the cortical visual areas. (Right) At about 200–300 ms, the amygdala transmits modulation signals to the cortical visual areas, which modulates conscious awareness.

At the same time, our results showed that the evident gamma-band activation of the amygdala was commonly elicited before 300 ms for the processing of faces, eye gazes and emotional facial expressions. Do the data support the hypothesis that the amygdala rapidly processes facial signals? What are the implications for the psychological functions of amygdala activity before 300 ms?

Several MEG/EEG studies have provided clues about the answers to these questions, indicating that neural activity at around 200–300 ms is associated with the conscious perception of visual stimuli. For example, a previous study showed that the EEG component corresponding to visual awareness of faces emerged at around 260–310 ms. These studies have consistently reported the enhanced negative deflection for seen vs. unseen stimuli in the posterior cortices, and such topography is consistent with neuroimaging evidence showing that the conscious perception of faces is related to activation of the cortical visual areas. Based on these data, our results suggest that the amygdala processes facial signals prior to or simultaneous with the conscious awareness of faces, which is the most rapid time stage introspectively accessible (Fig. 2, left and center).

In addition, several lines of research suggest that an important psychological function of this rapid amygdala activity in response to facial stimuli is to modulate the conscious awareness of faces. Neuroanatomical studies have shown that the amygdala has widespread projections to the visual cortices. Neuroimaging studies have demonstrated that amygdala activity modulated the activity of facial signal processing in the visual cortices, such as gaze direction and emotional expression. Regarding the temporal profiles of these modulatory effects on the cortical visual areas, several EEG studies have shown that negative deflection in the posterior cortices at about 200–300 ms was facilitated by emotional vs. neutral facial expressions., This heightened posterior negative deflection for emotional, compared with neutral, facial expressions was shown to be correlated with more rapid detection of emotional than neutral facial expresions. In this context, our data suggest that rapid facial signal processing in the amygdala might modulate the conscious awareness of faces, which has been demonstrated in several behavioral studies (Fig. 2, right).

However, it must be noted that several issues related to neurocognitive processes discussed remain unsettled. For example, debate regarding the visual input routes to the amygdala and whether the subcortical structures can process visual information about facial signals in a way that activates the amygdala persists., It remains possible that indirect pathways including other cortical regions, such as the posterior parietal cortex, may be responsible for the modulatory effect of the amygdala on the visual cortices. Further studies are necessary to clarify these issues.

In summary, our intracranial EEG recording studies have demonstrated that the gamma-band activation in the amygdala was elicited by emotional expressions, gazes and faces, with peak latencies at about 100, 200 and 250 ms, respectively. These results suggest that the amygdala performs multiple-stage processing for these facial signals using different visual input routes. The data also indicate that amygdala activation for processing all of these facial signals is rapid, which could be prior to or simultaneous with conscious awareness of the faces.