Magnocellular and parvocellular pathway contributions to facial threat cue processing.

Human faces evolved to signal emotions, with their meaning contextualized by eye gaze. For instance, a fearful expression paired with averted gaze clearly signals both presence of threat and its probable location. Conversely, direct gaze paired with facial fear leaves the source of the fear-evoking threat ambiguous. Given that visual perception occurs in parallel streams with different processing emphases, our goal was to test a recently developed hypothesis that clear and ambiguous threat cues would differentially engage the magnocellular (M) and parvocellular (P) pathways, respectively. We employed two-tone face images to characterize the neurodynamics evoked by stimuli that were biased toward M or P pathways. Human observers (N = 57) had to identify the expression of fearful or neutral faces with direct or averted gaze while their magnetoencephalogram was recorded. Phase locking between the amygdaloid complex, orbitofrontal cortex (OFC) and fusiform gyrus increased early (0-300 ms) for M-biased clear threat cues (averted-gaze fear) in the β-band (13-30 Hz) while P-biased ambiguous threat cues (direct-gaze fear) evoked increased θ (4-8 Hz) phase locking in connections with OFC of the right hemisphere. We show that M and P pathways are relatively more sensitive toward clear and ambiguous threat processing, respectively, and characterize the neurodynamics underlying emotional face processing in the M and P pathways.

Introduction

Evaluation of threat must be both rapid and accurate for adaptive functioning. An evolutionarily adaptive response to threat cues would be a combination of relatively automatic and deliberative processes that enable reflexive responses to clear threat signals (e.g. fleeing an attacker) while also inhibiting context-inappropriate responses during the extended evaluation of more ambiguous threat cues (e.g. fleeing from someone who is seeking help). Recent findings suggest that visual stimuli associated with threat may be processed differently by the major visual streams––the magnocellular (M) and parvocellular (P) pathways (Kveraga, 2014; Carretié et al., 2017; Im et al., 2017, 2018). An emerging hypothesis posits that reflexive processing of clear threat cues may be predominantly associated with the phylogenetically older, coarser and action-oriented M pathway, while reflective, sustained processing of threat ambiguity may preferentially engage the slower, analysis-oriented P pathway (Adams et al., 2012; Kveraga, 2014; Adams and Kveraga, 2015). Indeed, recent direct comparisons of M vs P pathway involvement in threat perception using functional magnetic resonance imaging (fMRI; Kveraga, 2014; Im et al., 2017, 2018) support this more nuanced picture, in that both M and P pathways are involved in threat processing, but seem to be attuned to different aspects of it. Specifically, clear, congruent threat cues in both face and scene images appear to be processed preferentially by the M channel, and ambiguous or incongruent threat cues appear to activate the P pathway more (Kveraga, 2014; Im et al., 2017, 2018). Similarly, the M pathway recently has been shown to be more sensitive than the P pathway to exogenous distractors when they were biological threat stimuli (pictures of spiders) rather than neutral stimuli (Carretié et al., 2017).

The human face can signal threat via combinations of cues, the primary of which are facial expression and eye gaze direction, which reflect both the affective state of the observed and the source or target of that emotion (Adams et al., 2010). The ‘shared signal hypothesis’ predicts that when expression and eye gaze direction are congruent, in that they together forecast the same behavioral tendency to approach or avoid, the processing of social signals conveyed by the face should be facilitated (Adams et al., 2006; Nelson et al., 2013). Adams and Kleck (2003, 2005) found that direct gaze, an approach signal, facilitated processing efficiency of approach-oriented facial expressions (e.g. anger and joy). In contrast, averted gaze, an avoidance signal, facilitated the processing of avoidance-oriented facial expressions (e.g. fear and sadness). These results have been replicated with dynamic threat displays (Sander et al., 2007), in a diffusion model of decision-making and reaction time (Benton, 2010), by examining reflexive orienting to threat (Fox et al., 2007), with EEG (Rigato et al., 2013), and in an attentional blink paradigm by Milders et al. (2011). Such interaction effects have also been demonstrated in fMRI studies examining amygdala responses to congruent and incongruent facial threat cues (Sato et al., 2004; Hadjikhani et al., 2008; N’Diaye et al., 2009) and most recently in in a magnetoencephalography (MEG) study (Cushing et al., 2018).

Using fMRI, we have recently shown that congruent facial threat cues (fear with averted eye gaze) evoke greater amygdala activation when presented to the M pathway, and incongruent threat cues (fear with direct eye gaze) evoke greater amygdala activation and performance when biased to the P pathway (Im et al., 2017). Furthermore, with increasing observer anxiety, perceptual accuracy was enhanced for M-biased clear threat cues, but impaired for ambiguous threat cues, coupled with hemispheric amygdala lateralization for M-clear and P-ambiguous cues (Im et al., 2017). While these findings support our hypothesis of differential engagement of M and P pathways with clear and ambiguous threat cues, the neurodynamics and connectivity in the M and P pathways mediating this processing have not been described.

In the present study we aimed to elucidate the dynamics and connectivity mediating the processing of these threat cues by presenting facial threat cues biased to the M and P pathways (Kveraga et al., 2007) while recording the magnetoencephalogram in 57 subjects. We examined activity in and between a number of key regions in the extended face processing network: the fusiform gyrus, posterior superior temporal sulcus (pSTS), orbitofrontal cortex (OFC), amygdaloid complex and the earliest cortical visual region V1. The contributions of these regions to face perception in general, as well as to social communication and gaze perception, have been well established in many studies (e.g. Puce et al., 1995; Wicker et al., 1998; Haxby et al., 2002; Hooker et al., 2003; Hardee et al., 2008). The fusiform cortex has been implicated in face perception (Kanwisher et al., 1997; McCarthy et al., 1997) and is additionally known to be sensitive to gaze (George et al., 2001) along with the superior temporal sulcus (STS; Hoffman and Haxby, 2000), and both have been shown to respond to emotional expression (Harris et al., 2012). STS, OFC and amygdala comprise the major nodes of the proposed ‘social brain’ (Baron-Cohen et al., 1999), and pSTS specifically has been shown to specialize in inferring intentionality from social cues (Nummenmaa and Calder, 2009; Rhodes et al., 2012). Importantly, the amygdala has been the primary focus of previous investigations into facial fear and eye gaze (Adams et al., 2003, 2012; Hadjikhani et al., 2008; Van Der Zwaag et al., 2012; Im et al., 2017, 2018; Cushing et al., 2018), and this set of brain regions was the focus of our recent MEG investigation into how exposure duration modulates the sensitivity and neurodynamics of congruent and incongruent facial threat cues (Cushing et al., 2018).

Our primary hypothesis was that M-biased stimuli would result in greater early processing of congruent threat cues, while stimuli presented to the P pathway should elicit relatively stronger processing of incongruent threat cues. Based on previous findings implicating the amygdala in differential sensitivity to threat cue congruency (Adams et al., 2012; Im et al., 2017, 2018; Cushing et al., 2018), we particularly expected to observe this in connectivity with the amygdaloid complex. As no clear spectral separation between congruent and incongruent cues was observed during facial threat cue processing in Cushing et al. (2018), we had no a priori hypothesis regarding sensitivity in a particular range of frequencies to threat cue congruency. Conversely, based on our previous MEG findings showing distinct temporal stages of congruent and incongruent threat cue processing (Cushing et al., 2018), we expected to observe earlier processing in the M pathway and later processing in the P pathway.

Methods

Participants

This research was performed in accordance with the guidelines and regulations set forth in the Declaration of Helsinki and was approved by the Institutional Review Board of the Massachusetts General Hospital (MGH). MEG and behavioral data were collected from 73 participants [28 males, mean age (s.d.) = 26.6 (6.8)] with normal or corrected to normal vision who completed the study for monetary compensation ($50). Enrollment of 73 participants ensured an ultimately even number of participants completed an anatomical MRI portion of the exam, as 5 participants had already been determined unwilling or unable to complete the follow-up MRI scan. Potential subjects were screened via a questionnaire to make sure they were eligible for MEG recording and subsequent MRI structural scans and had no history of mental illness or use of psychoactive medication. Their informed written consent was obtained according to the protocol approved by the Institutional Review Board of MGH. Of the 68 participants that completed both MEG and MRI sessions, 5 were excluded due to data loss during collection due to technical issues, 3 due to excessive trial rejection from eye blinks or poor signal, 2 due to poor task performance (defined as below 60% accuracy in any one condition) and 1 due to neurological abnormalities discovered during the follow-up anatomical MRI scan, resulting in 57 participants entered into the final analyses.

Stimuli

Experimental face stimuli included eight models (four male) from the Pictures of Facial Affect (Ekman, 1976), eight models (four male) from the NimStim Emotional Face Stimuli database (Tottenham et al., 2009) and eight models (four male) from the FACE database (Ebner et al., 2010), each displaying a neutral or fearful expression. Faces with an averted gaze had the gaze averted either leftward or rightward. An example of each emotional expression by eye gaze pairing was taken from each model and rendered as a two-tone image under all biasing schemes (unbiased, M-biased and P-biased), resulting in 288 unique stimuli. Stimuli were presented with Psychtoolbox (Brainard, 1997; Pelli, 1997) in Matlab (MathWorks Inc., Natick, MA). All images were 294 × 400 pixels at a screen resolution of 1280 × 800 pixels. Stimuli were rear-projected onto a translucent screen placed 160 cm from the seated participant to create a 61.5 × 38.5 cm display. Stimuli measured 14.1 × 19.2 cm, subtending ~5.1o of visual angle horizontally and 6.9o vertically.

In order to create stimuli biased toward M and P neurons, we utilized their respective specific response properties in designing the stimuli. M retinal ganglion cells have a higher contrast gain and are relatively color-insensitive compared to P cells. These differences in contrast gain are maintained at low levels of mean retinal illumination (Purpura et al., 1998). Opposingly, P cells are highly sensitive to chromatic contrasts while there is a noticeable dropout of M response at chromatic isoluminance (Lee et al., 1988). Additionally, P cells have less contrast gain and cannot resolve luminance contrast below 8% (Tootell et al., 1988a). M cells are also known to selectively respond to low spatial frequencies (Derrington and Lennie, 1984; Tootell, et al., 1988b) while P cells respond more to high spatial frequencies (Merigan et al., 1991; Tootell et al., 1988b).

Despite the fact that simultaneous use of spatial frequency and chromatic contrast manipulations have been used to successfully localize M and P inputs to the lateral geniculate nucleus (LGN) using fMRI (Denison et al., 2014), P cells actually appear to act as low-pass filters when presented with isoluminant red–green stimuli (Gegenfurtner and Kiper, 2003). Consequently, simultaneous manipulation of these neurophysiological properties may not necessarily be complementary. Previous studies examining M and P contributions to emotional processing have utilized low and high spatial frequency manipulations (Vuilleumier et al., 2001, 2003; Méndez-Bértolo et al., 2016). However, in general, both chromatic and spatial frequency have been used to bias the M and P pathways (Steinman et al., 1997; Denison et al., 2014). In particular, the same strategy of using chromatic/luminance over spatial frequency manipulations that we use here has been used to investigate perceptual processing dysfunction in schizophrenia (Schechter et al., 2003) as well as to reinstate global processing in simultanagnosia patients with parietal lesions (Thomas et al., 2012). Importantly in this latter study, it was the P-biased red/green stimuli that allowed patients with damage to the dorsal/M pathway to perceive global forms, indicating such biasing is effective at shifting perceptual processing away from the M pathway as it would typically happen from retinal input. There is evidence that M neurons can detect red–green borders (Lee and Sun, 2009), but it did not seem to prevent biasing the P pathway in these patients despite intact subcortical structures and pathways (Tamietto and De Gelder, 2010). Building on this and our successful use of these exact manipulated images in fMRI studies investigating pathway contributions to emotional processing (Im et al., 2017, 2018), we use these same image manipulations here.

Black and white two-tone images were created for each face to serve as unbiased stimuli (high luminance contrast). Pathway-biased stimuli were then created from these two-tone images using standard techniques from other studies relying on the neuronal properties described above which have successfully investigated M and P pathway contributions during a variety of tasks (Steinman et al., 1997; Schechter et al., 2003; Butler et al., 2007; Kveraga et al., 2007; Im et al., 2017). M-biased stimuli were created by means of a luminance defined two-tone stimulus at ~3.5% Weber contrast. P-biased stimuli were created by converting the two-tone face images to isoluminant intensities of red and green. Isoluminance was individually calibrated for each participant using heterochromatic flicker photometry. The detailed procedures for creating these stimuli can be found in supplemental information (SI) methods. Participants completed both pre-tests while placed in the MEG dewar immediately before the MEG recording began to ensure identical viewing conditions for the main task.

Experimental design and statistical analysis

Task design

Participants were exposed to 288 experimental trials and 96 blank trials in four runs of 96 trials each in an event-related design. Trials were 2.5 s long, starting with a randomized 200–400 ms of attending to a red fixation cross centered on the screen. The face stimulus was then focally presented for 1 s, followed by a blank screen for 1100–1300 ms. Subjects were instructed to indicate whether the face displayed a fearful or neutral expression as quickly and accurately as possible. Button mapping was counterbalanced across subjects between the index and middle digits of the participant’s right hand. Feedback was briefly displayed following the blank post-stimulus period indicating whether the participant had correctly or incorrectly identified the emotion of the face or had taken too long to respond. A response later than 1.5 s post-stimulus was considered late (Figure 1A).

Fig. 1

Task design and behavioral performance. (A) Experimental trial design beginning with 200–400 ms of attending to a central red fixation followed by stimulus (examples below) presentation for 1 s. Participants were instructed to indicate via key press as quickly and accurately as they could whether the presented face appears fearful or neutral. Participants were then shown a green central fixation inversely timed to the pre-stimulus jitter, then presented feedback on whether their response was correct, incorrect or if they took too long to respond (maximum RT allowed was 1.5 s post-stimulus). Trials were always 2.5 s. (B) RTs of participants responding to M-biased (gray) vs P-biased (red) congruent (bright hues) and incongruent (dark hues) facial threat cues. Upper horizontal black bar indicates a significant interaction between pathway bias and facial threat cue congruency (F(1,56) = 16.98, p<0.001). Lower horizontal black bar indicates significant difference in RT for incongruent facial threat cues (t(56)=3.36, p=0.0014). Participants were faster to respond to M-biased congruent cues and P-biased incongruent cues, supporting our main hypothesis of congruent cues being preferentially processed by the M pathway and incongruent cues being preferentially processed by the P pathway.

MEG acquisition

MEG recordings were obtained with a 306-channel Neuromag Vectorview whole-head system (Elekta Neuromag, Stockholm, Sweden) with 204 planar gradiometers and 102 magnetometers enclosed in a magnetically shielded room with a shielding factor of 250 000 at 1 Hz (Imedco AG, Haegendorf, Switzerland). Only gradiometers were used in the experimental analysis due to less noise being present in these sensors. Four head position indicator (HPI) electrodes were affixed asymmetrically to each participant’s forehead and the mastoid processes to monitor head position in the dewar at the beginning of the recording session. Digitizer data were collected for each participant’s head on a Polhemus FastTrack 3D system within a head coordinate frame defined by anatomical landmarks (left preauricular area, right preauricular area and the nasion). HPI positions were marked within this frame, and 150–200 points on the scalp and the face were saved for use in co-registering with a multi-echo structural MRI of the subject. Eye movements and blinks were monitored via four electrooculography (EOG) electrodes: two vertical electrodes on the left eye (one placed just above the eyebrow, the other on the upper cheekbone just below the eye) and two horizontal electrodes (placed at the edges of the outer canthi). Cardiac activity was recorded via electrocardiography (ECG) using electrodes placed on the left and right upper chest (two total). All data from MEG sensors and EOG and ECG electrodes were sampled at 600.615 Hz and were band-pass filtered at 0.1–200 Hz. Recordings were stored for offline analysis.

Data pre-processing and averaging

Pre-processing and averaging of all recordings were performed with the minimum-norm estimate (MNE) analysis package (Gramfort et al., 2014) as well as MNE-Python (Gramfort et al., 2013) and custom scripts in Python and Matlab. Noise from external sources was excluded by application of signal-space projection (Tesche et al., 1995; Uusitalo and Ilmoniemi, 1997). Sensors that were visibly noisy during the recording were noted by the researchers and excluded from the analysis. For timecourse analysis, a low-pass filter of 40 Hz was applied, and recordings were epoched from 200 ms before stimulus onset to 1300 ms post-stimulus. For time-frequency analysis, no filter was applied to the data, and recordings were epoched from 500 ms before stimulus onset until 1440 ms post-stimulus onset. Wider windows than necessary were taken in both cases to prevent filtering and other edge artifacts, but only time points from 0 to 500 ms were entered into statistical analyses for both analyses. Rejection parameters were set at 4000 fT/cm for gradiometers and 800 uV for EOG. Any epoch where any of these limits were exceeded was excluded from further analysis. A further data quality inspection was performed during pre-processing, and any noisy or flat channels that were not picked up during the recording, but resulted in the rejection of 20% or more of epochs, were excluded from analysis to prevent unnecessary epoch rejection.

Source localization

Individual participants’ anatomical brain images for source localization of MEG activity were obtained with multi-echo structural MRIs on a 1.5 T Siemens Avanto 32-channel ‘TIM’ system. Brain images were reconstructed, and triangulated brain surfaces generated via the watershed algorithm with the Freesurfer analysis package (Fischl et al., 2004a). A decimated dipole grid was fitted to the inflated white matter surface in the shape of an icosahedron recursively divided five times to generate a 20 480-point grid. Two source spaces were created for each participant: one surface source space of the whole cortical surface based on the dipole grid described above and one volume source space of the bilateral amygdala as segmented by Freesurfer (due to this segmentation being absent in the surface source space). Two forward solutions were then calculated, one for each source space, but with both using the same geometry dependent solution calculated from the single-compartment boundary-element model. Sources closer than 5 mm to the inner skull surface were omitted from the forward solution in all cases. The MRI-head coordinate transformation for each subject was supplied to the forward model by aligning the digitizer data obtained in the original recording session (see MEG acquisition) with a high-resolution head surface tessellation constructed from the MRI data. The inverse operator was prepared with a loose orientation constraint parameter of 0.2 in order to improve localization accuracy (Lin et al., 2006). A depth-weighting coefficient of 0.8 was also set for the inverse operator to lessen the tendency of MNEs to be localized to superficial currents in place of deep sources. MEG data were source localized onto the entirety of each source space using a λ2 regularization parameter based on Signal-to-Noise Ratio (SNR, set to 3) equal to 1/(SNR2). Evoked cortical activation was quantified spatiotemporally by taking only the radial component from a three-orientation source (x y z) at each vertex of the triangulated source space in the form of dynamic statistical parametric maps (dSPMs) based on an inverse solution regularized with an SNR of 3. DSPMs are a statistical representation of significant activity from each source per time point calculated by noise normalization on the estimated current amplitude (MNE) of a given source according to noise covariance between sensors calculated during a baseline period of 200 ms pre-stimulus (Dale et al., 2000). The noise covariance estimation model was selected automatically according to rank for each participant (Engemann and Gramfort, 2015). For time-frequency analysis, MEG data were source localized in an identical manner, but on a trial-by-trial basis, using the MNE (Hämäläinen and Ilmoniemi, 1994) method with the same inverse operators and regularization parameters described above (one surface-based, one volume-based).

Region of interest (ROI) selection and definition

We analyzed neural activity in and between a number of key regions in the extended face processing network: the fusiform gyrus, pSTS, OFC, amygdaloid complex and the earliest cortical visual region V1. The role of these regions in the perception of faces, in addition to social communication and gaze perception, has been well documented across many studies (e.g. Puce et al., 1995; Wicker et al., 1998; Haxby et al., 2002; Hooker et al., 2003; Hardee et al., 2008). Additionally, we examined this same extended face processing network in our MEG study documenting exposure duration effects on facial fear and eye gaze congruency sensitivity (Cushing et al., 2018).

The ability to source localize subcortical activity from MEG data has been questioned in the MEG literature (Stephen et al., 2005; Riggs et al., 2009; Attal et al., 2012), but there is accumulating evidence that MEG activity can indeed be localized to the amygdala’s subcortical nuclei (Attal et al., 2007; Cornwell et al., 2008; Dumas et al., 2010, 2011, 2013; Attal and Schwartz, 2013; Styliadis et al., 2014). However, given the reliable amygdala activation in all of the previous studies (Breiter et al., 1996; Whalen, 1998, 2004; Vuilleumier et al., 2001; Whalen et al., 2001; Adams et al., 2003, 2010, 2012; Zald, 2003; Sergerie et al., 2008; N’Diaye et al., 2009; Lee et al., 2010; Van Der Zwaag et al., 2012) investigating this question using fMRI, it is possible that some of the MEG activity localized to the amygdala may be generated subcortically. However, physiologically, a likelier source of MEG activity in this region is the surrounding periamygdaloid cortex (PAC), which is heavily involved in the inputs and outputs of deeper subcortical amygdala nuclei, in addition to many other cortical regions (Aggleton, 2000; Pitkänen et al., 2000). Because we are unable to distinguish whether this MEG activity may arise from subcortical nuclei in the amygdala and/or the surrounding PAC in the present study, we will remain agnostic as to the exact generators of the signal and refer to the region as the amygdaloid complex.

For all regions of interest (ROIs) except the amygdaloid complex, functional surface labels were generated on each individual’s surface source space based on anatomical constraints. Anatomical constraints for V1, fusiform gyrus and OFC were taken from the ‘BA’ (Hinds et al., 2008), ‘aparc.a2009s’ (Destrieux et al., 2010) and ‘aparc’ (Fischl et al., 2004b) cortical parcellation atlases, respectively. Because of an observed tendency for parcellations of the STS by Freesurfer to extend posteriorly beyond the true STS into inferior sulci, as tracked in the MRI volume, we generated our own anatomical constraint for pSTS. Using the ‘mne_analyze’ software from the MNE package, the STS was traced in standard space on the ‘fsaverage’ inflated surface and split into equal thirds, with the most posterior third taken as pSTS. This pSTS surface label was then morphed from the standard ‘fsaverage’ space to each individual participant’s source space to be used as the pSTS anatomical constraint. For each of these anatomical constraints, a functional label was derived therein, based on activity averaged from all conditions, i.e. activity that was independent of trial type. Using this condition-independent activity, the maximally activated source was isolated along with any neighboring vertices reaching at least 60% of that maximum activation. These vertices were then saved as the final label to be used in the final statistical analyses. Our amygdaloid complex ROI was defined as the entire volume source space generated from the automatic segmentation of the amygdala.

Phase-locking analysis

Using custom scripts in Matlab, the Phase-Locking Value (PLV) between regions was calculated for the following connections: V1-fusiform gyrus, V1-amygdaloid complex, V1-OFC, fusiform gyrus-amygdaloid complex, fusiform gyrus-OFC, fusiform gyrus-pSTS, amygdaloid complex-OFC, pSTS-amygdaloid complex and pSTS-OFC. Rather than examining all possible connections, we selected connections which we thought to be of a priori functional interest. We wanted to examine the full reciprocal connectivity between the proposed ‘social brain’ of OFC, (p)STS and amygdala (Baron-Cohen et al., 1999). In addition, we wanted to document the interaction of this social network with the critical face processing area in the fusiform gyrus. For connectivity with early visual cortex (V1), we selected three terminal nodes that could elucidate how facial threat information is relayed through the brain. We examined V1 functional connectivity with both the amygdaloid complex and OFC. Due to their massive reciprocal connections, there is some uncertainty about what is the initial target of the early visual system during facial threat perception. Testing the connectivity of these with V1 would elucidate whether fast M projections reach OFC early to be fed back into the face processing stream as suggested by Kveraga et al. (2007) or whether connectivity with the amygdala is more important. Similarly, examining connectivity of the early visual cortex with the face-sensitive fusiform gyrus would help to elucidate what facial information (congruency, emotion and gaze) is conveyed between the earliest visual processing areas and this critical face processing area.

The PLV is a number between 0 and 1 (1 being perfect synchrony) that represents a magnitude-normalized measure of the phase angle consistency between regions for a particular time point at a particular frequency, serving as a proxy measure of functional connectivity (Lachaux et al., 1999). This number was obtained by spectral decomposition of each epoch at each time point for each frequency, using a complex Morlet wavelet transformation with a cycle-width of 5, to yield a 2D matrix (frequency × time) of complex numbers. We analyzed frequencies from 4 Hz to 70 Hz.

Behavioral data analysis

Reaction time (RT) and accuracy were computed from each participant’s response for each experimental trial. All responses were submitted to a parametric repeated-measures analysis of variance (ANOVA) according to a 3 (Pathway bias: unbiased, M-biased and P-biased) × 2 (Emotion: fear and neutral) × 2 (Gaze: direct and averted) factorial design using the ‘JASP’ statistical software (JASP Team, 2017). The whole of these ANOVA results is reported in SI results. In addition, to test our hypothesis of enhanced processing of M-biased compared to P-biased congruent and P-biased compared to M-biased incongruent threat cues, both RT and accuracy were compared for these conditions with a 2 × 2 ANOVA.

MEG data analysis

All statistics were computed using non-parametric cluster-level permutation tests based on 3000 permutations with a critical α-value of 0.05, following Maris and Oostenveld (2007). Cluster mass was determined by summing statistical values of adjacent suprathreshold pixels within a cluster rather than counting the number of adjacent suprathreshold pixels/time points (or any other cluster-weighting method). The null distribution was built by permuting data and storing the largest cluster mass observed in the permuted data. In both the time and time-frequency domains, all time points between 0 and 500 ms were submitted to analysis, with all frequencies (4–70 Hz) included for the frequency domain. We characterized the effects of our experimental variable manipulation on neurodynamics with a non-parametric repeated measures ANOVA design (see SI methods), the results of which we report in full in SI results. However, as we approached this experiment with the specific hypothesis that M-biased stimuli would result in greater early processing of congruent threat cues, while stimuli presented to the P pathway should elicit relatively stronger processing of incongruent threat cues, we performed a non-parametric t-test of M-biased compared to P-biased congruent and incongruent threat cues for our planned comparisons to explore our main hypothesis (see SI methods). The interaction between eye gaze on fearful faces and M/P was not directly tested via an ANOVA as it would be largely insensitive to the expected temporal differences between M and P processing, making the non-parametric t-test the more appropriate test for our specific hypothesis.

Results

Behavioral results

Unbiased stimuli

For unbiased stimuli, we found no significant difference in accuracy for unbiased compared to M-biased faces [t(56) = −1.426, P = 0.47, Bonferroni corrected] or to P-biased faces [t(56) = 2.021, P = 0.137, Bonferroni corrected]. We observed a main effect of pathway bias on RT [F(2,112) = 32.498, P < 0.001, η2p = 0.367] with participants responding on average 20 ms faster for unbiased faces compared to M or P-biased faces (both Ps < 0.001, Bonferroni corrected). In short, participants were faster but not more accurate at identifying emotion in unbiased compared to pathway-biased stimuli.

Pathway-biased stimuli

In our analysis of M- vs P-biased congruent facial threat cues we observed a significant interaction on RTs between pathway bias and facial threat cue congruency [F(1,56) = 16.98, P < 0.001, η2p = 0.233]. Participants responded significantly faster to P-biased vs M-biased incongruent facial threat cues [t(56) = 3.35, P = 0.0014] and had a non-significant tendency to respond faster to M-biased vs P-biased congruent facial threat cues [t(56) = 1.34, P = 0.18; Figure 1B]. These findings provide behavioral evidence for an M advantage for congruent facial threat cues and a P advantage for incongruent facial threat cues. For accuracy, there was no significant interaction between threat cue congruency and pathway bias [F(1,56) = 0.208, P = 0.650, η2p = 0.233].

MEG results

Unbiased stimuli in general evoked higher activations compared to M-biased stimuli across our ROIs in both hemispheres (SI Tables S1 and S2, SI Figure S1A). Similarly, unbiased stimuli elicited higher phase locking in many connections primarily involving the early visual cortex (V1) and fusiform gyrus (SI Tables S3 and S4, SI Figure S3). In brief, unbiased stimuli evoked significantly higher activation in, and connectivity between, our ROIs in the face processing network.

As the primary hypothesis of this investigation was an M advantage for the processing of congruent threat cues (averted-gaze fear) and a P advantage for processing incongruent threat cues (direct-gaze fear), we performed contrasts of M-biased vs P-biased congruent cues and incongruent cues. Contrasts were performed in all phase-locking connections, as well as dSPM activation in all ROIs (SI Figure S2). Additional results outside this main hypothesis are reported in the SI Material including full ANOVA results for both ROI activations and functional connectivity (SI Tables S1–S4, SI Figures S1–S4).

Phase locking: M-biased vs P-biased congruent facial threat cues

Comparing M-biased vs P-biased congruent facial threat cues (averted-gaze fear), we found an early M advantage in the connectivity between the fusiform gyrus and amygdaloid complex of the right hemisphere (Figure 2). M-biased congruent facial threat cues evoked significantly stronger phase locking compared to P-biased congruent facial threat cues from ~50 to 400 ms (P = 0.0054, Figure 2). Note that in this comparison we also observed significantly stronger phase locking for the P-biased cues in the θ and α ranges. However, as we observed a main effect of pathway bias (see SI Table 3) and stronger phase locking to P-biased incongruent facial threat cues compared to congruent facial threat cues (P = 0.0048) in the same spectro-temporal range in the same connection, this stronger phase locking to P-biased congruent facial threat cues is likely the result of pathway biasing in general rather than facial threat cue congruency. We saw evidence of a similar early β M advantage in the connectivity of the amygdaloid complex and OFC in the left hemisphere from ~0 to 200 ms (P = 0.0156). P-biased congruent facial threat cues evoked stronger phase locking between left fusiform gyrus and pSTS at ~0–250 ms in the θ range (P = 0.0132). No main effect of pathway was observed in this connection (SI Table 3) so it is more likely that this P advantage is related to the threat cue congruity.

Fig. 2

Phase-locking results. Time-frequency maps depicting results of planned non-parametric cluster-based t-test of M-biased (blue) vs P-biased (red) clear, congruent (averted-gaze fear, ‘top’) and ambiguous, incongruent (direct-gaze fear, ‘bottom’) facial threat cues. Significant and marginally significant clusters are marked with a letter and traced in white. Corresponding Monte-Carlo P-values for marked clusters based on 3000 permutations listed below each time-frequency plot. Note that frequency on the y-axis is plotted logarithmically. Greek letters on the right y-axis denote canonical frequency ranges demarcated by dashed black lines across time-frequency plots. Color bars to the right of each vertical pair of time-frequency plots denote statistic being plotted and pixel value color mapping.

In brief, we found an early M advantage for congruent facial threat cues in β phase locking involving bilateral amygdaloid complex with right fusiform gyrus and left OFC. Additionally, we saw a P advantage in the connectivity between left fusiform gyrus and pSTS, indicating possible fusiform lateralization by pathway biasing.

Phase locking: M-biased vs P-biased incongruent facial threat cues

In our other comparison of M-biased vs P-biased incongruent facial threat cues, we observed an early P advantage for incongruent facial threat cues concentrated in the θ range. In the connectivity between OFC and the amygdaloid complex in the right hemisphere (P = 0.018) as well as fusiform gyrus and amygdaloid complex in the left hemisphere (P = 0.0002), we observed significantly stronger phase locking to P-biased compared to M-biased incongruent facial threat cues in the θ range from ~0 to 500 ms (Figure 2). Additionally, P-biased incongruent threat cues evoked stronger phase locking compared to M-biased incongruent threat cues in left V1 and OFC (P = 0.018). As we documented no interactions or main effect of pathway in the connectivity between amygdaloid complex and OFC (see SI Table 3), the stronger phase locking to P-biased incongruent facial threat cues is likely due to the incongruity of the cue. However, as we did observe higher phase locking to P-biased compared to M-biased stimuli in general (see SI Table 3) in the connectivity between left fusiform gyrus and amygdaloid complex as well as left V1 and OFC, it is possible that this is a result of pathway biasing in general, independent of signal congruity. An additional non-parametric t-test comparing P-biased congruent facial threat cues to P-biased incongruent facial threat cues revealed there were no differences in phase locking between the cues in left V1 and OFC (P = 0.283), indicating that the effect is indeed likely caused by pathway biasing rather than threat cue congruity. However, in the connectivity between left fusiform gyrus and amygdaloid complex, a significant difference in phase locking to P-biased incongruent threat cues compared to congruent threat cues was observed in the same spectro-temporal range (P = 0.0086) indicating the effect in this connection is likely dependent on facial threat cue congruity.

In summary, we observed an early (0–400 ms) increase in β-band phase locking for congruent facial threat cues presented in M-biased form in the connectivity between right fusiform and amygdaloid complex as well as left amygdaloid complex and OFC. In addition, during the same time period we saw a P advantage in θ phase locking for incongruent facial threat cues in the connectivity between left fusiform gyrus and amygdaloid complex as well as between the right amygdaloid complex and OFC. Given that interregional phase locking is thought to be indicative of the strength of communication between the regions, this finding provides support for our hypothesis that the M pathway preferentially processes congruent threat cues while the P pathway preferentially processes incongruent threat cues, as M-biased congruent and P-biased incongruent facial threat cues resulted in stronger initial phase locking than their opposing pathway counterparts. Moreover, the finding is consistent with our recent fMRI findings with an identical paradigm and an overlapping set of subjects (Im et al., 2017). Lastly, our results provide preliminary evidence of a spectral separation between M and P pathway-related processing as the initial response to M-biased cues occurred in the β band, while the initial response to P-biased cues occurred in the θ band for most significant functional connectivity increases.

Discussion

The primary goal of this study was to test the hypothesis (Adams et al., 2012) that the M and P visual pathways in humans are attuned to different aspects of threat cues embedded in facial expression and eye gaze. To accomplish this, we recorded the MEG of participants who viewed intact (unbiased), M- or P-biased faces that wore either a fearful or a neutral expression, with either direct or averted eye gaze. This design was employed to test our specific prediction that congruent threat cues (fear faces with averted gaze), which point to the source of danger (Hadjikhani et al., 2008), would engage the action-oriented M pathway relatively more, while the analysis-oriented P pathway would be more engaged by incongruent (direct-gaze fear faces) threat cues, which leave the source of threat ambiguous (Adams et al., 2017). This hypothesis was generally supported by both our behavioral and MEG results. Behaviorally, we found an interaction between visual pathway and the congruence of threat cues in the face, such that responses to M- and P-biased fear faces were about equally fast when eye gaze was averted, but P-biased fear faces were responded to significantly faster than M-biased faces when the gaze was direct. Connectivity with the amygdaloid complex was increased in OFC and fusiform gyrus for M-biased faces when the threat cues were congruent (fear with averted gaze) and for P-biased faces when the cues were incongruent (fear with direct gaze). Functional connectivity also increased between left fusiform gyrus and pSTS for P-biased congruent faces. This could be a result of the fusiform gyrus’ role as a key node in the largely P-fed ventral stream, highlighting the potential of parallel but interacting pathways for efficient processing of facial threat cues. These results largely support and extend the findings of a number of previous studies examining this topic (Adams et al., 2012; Van Der Zwaag et al., 2012; Im et al., 2017; Cushing et al., 2018) by demonstrating different attunement of the M and P pathways to processing congruent and incongurent threat cues, respectively.

The finding of preferential attunement of the M pathway to clear, congruent threat cues is consistent with previous investigations of affective blindsight. Patients with cortical blindness due to striate cortex lesions can process both emotional expression and eye gaze, indicating processing outside the primary visual striate cortex (Burra et al., 2013, 2017), and are sensitive to facial emotion congruency (De Gelder et al., 2001). Importantly, this processing is dependent upon low spatial frequencies, suggesting the M pathway performs this unconscious processing (Burra et al., 2013, 2017). These findings support the present results and underscore the potential for them to be supported by an early subcortical route that bypasses V1 (Tamietto and De Gelder, 2010). Furthermore, connections between these subcortical structures on this proposed subcortical route are compensatorily enhanced in patients with V1 damage demonstrating such affective blindsight (Tamietto et al., 2012), indicating these regions likely are involved in the processing of affective stimuli in typical vision (though see Pessoa and Adolphs, 2010). However, another possible explanation is that results are driven by rapid M projections traveling the dorsal stream to OFC, which then connects to the amygdala (Kveraga et al., 2007). As some MEG evidence indicates a subcortical route to the amygdala would be insensitive to spatial frequency or emotional affect (Garrido et al., 2012; McFadyen et al., 2017), perhaps cortical M projections drive the present results. Conversely, other studies utilizing intracranial recordings have indicated a subcortical route may be specifically sensitive to low spatial frequency fearful faces (Méndez-Bértolo et al., 2016). Future investigations using methodology equipped to examine contributions from the pulvinar thalamic nucleus and superior colliculus in threat perception could shed light on this topic.

Another novel finding was the observed frequency separation in the phase locking evoked by M- and P-biased stimuli. While we did not expect a priori to observe a frequency band separation for faces presented in M- or P-biased form, we do find it worth noting that the early M advantage tended to occur primarily in the higher (13–30 Hz) β band, while phase locking in response to P-biased stimuli was centered in the lower (4–8 Hz) θ band. Notably, these frequency ranges happen to correspond to the temporal flicker frequencies used to bias low-level stimuli to either the M or P pathways in other studies (e.g. Denison et al., 2014 used 15 Hz and 5 Hz temporal flicker stimuli to bias the M and P pathways, respectively), suggesting that there may be a relationship between the oscillatory responses of these M and P neurons and the stimulus features to which they are sensitive. This observed frequency separation may be additionally explained by recent findings that feedforward activity is represented by θ-band synchronization while feedback activity is associated with β-band synchronization (Bastos et al., 2015). Previous findings have implicated the M pathway in top-down facilitation of object recognition (Kveraga et al., 2007). So, considering our findings here of a β-band M advantage for clear, congruent threat cues, it may be the case that the facilitated processing of congruent signals that the ‘shared signal hypothesis’ seeks to explain is enabled through similar rapid top-down facilitation by the M pathway as found by Kveraga et al. (2007).

While our hypothesis of an M advantage for the processing of congruent facial threat cues and a P advantage for the processing of incongruent facial threat cues was supported, the processing differences that we observed were separated primarily spectrally and spatially rather than temporally. The finding that amygdala reactivity to gaze on a fearful face is modulated by exposure duration (Adams et al., 2012; Van Der Zwaag et al., 2012; Cushing et al., 2018) initially suggested these congruency sensitivities could result from early and late processing differences focusing on coarse to fine levels of processing, respectively (Goffaux et al., 2011). However, the temporally overlapping results reported here suggest that a faster emerging, ‘reflexive’ and slower, ‘reflective’ processing is perhaps too simplistic. Rather, processing in the M and P channels appeared to proceed more or less in parallel, consistent with the idea of the retinal image being filtered by parallel, selective channels to accomplish somewhat different visuospatial functions (Hughes et al., 1996).

In summary, presenting fearful faces in P-biased form resulted in a processing bias toward incongruent facial threat cues (direct-gaze fear) manifesting in shorter RT and increased connectivity in the θ frequency range, compared with congruent threat cues (averted-gaze fear). This supports the idea that the analysis-oriented P pathway is geared relatively more toward processing threat ambiguity. Presenting M-biased faces resulted in greater phase locking for M stimuli in the β range for congruent facial threat cues between the right amygdaloid complex and fusiform gyrus, as well as between the left amygdaloid complex and OFC. These findings support the hypothesis that the coarser and more action-linked M pathway has a processing bias for clear threat cues. These findings characterize for the first time the brain dynamics and connectivity evoked by M and P pathway-biased facial threat cues in MEG and describe highly temporally resolved and spectrally specific connectivity patterns in the extended face processing network, with an M pathway bias in the β frequency range for congruent facial threat cues and a P pathways bias in the θ frequency range for incongruent facial threat cues.

Funding

This work was supported by the National Institutes of Health [R01MH101194 to K.K. and R.B.A.J., K01MH084011 to K.K., R01MH107797 to Avniel Ghuman, instrumentation grants 1S10RR023401, 1S10RR019307, 1S10RR023043]; and the National Center for Research Resources [P41RR14075 to A.A. Martinos Center].

Conflict of interest. None declared.

Click here for additional data file.