The effect of constraining eye-contact during dynamic emotional face perception-an fMRI study.

Eye-contact modifies how we perceive emotions and modulates activity in the social brain network. Here, using fMRI, we demonstrate that adding a fixation cross in the eye region of dynamic facial emotional stimuli significantly increases activation in the social brain of healthy, neurotypical participants when compared with activation for the exact same stimuli observed in a free-viewing mode. In addition, using PPI analysis, we show that the degree of amygdala connectivity with the rest of the brain is enhanced for the constrained view for all emotions tested except for fear, and that anxiety and alexithymia modulate the strength of amygdala connectivity for each emotion differently. Finally, we show that autistic traits have opposite effects on amygdala connectivity for fearful and angry emotional expressions, suggesting that these emotions should be treated separately in studies investigating facial emotion processing.

Le surveillant, on l’appelle le Bouillon, quand il n’est pas là, bien sûr. On l’appelle comme ça, parce qu’il dit tout le temps: « Regardez-moi dans les yeux », et dans le bouillon il y a des yeux. Moi non plus je n’avais pas compris tout de suite, c’est des grands qui me l’ont expliqué. Le Bouillon a une grosse moustache et il punit souvent, avec lui, il ne faut pas rigoler.

We call our supervisor Broth, when he is not there, of course. We call him like that because he says all the time: ‘Look me in the eyes’, and in the broth, there are eyes. I did not either understand right away, but the older kids explained it to me. Broth has a big mustache and he often punishes, with him you do not laugh.

Rene Goscinny, Le Petit Nicolas, Chapitre 3 ‘Le Bouillon’.

Introduction

The eyes of others truly hold a special status in human social perception, and eye contact serves as an important non-verbal channel for communication and social interaction throughout life (Darwin, 1872/1965). Four-day-old newborns distinguish between a face looking toward them and a face looking away (Farroni ), and from age three months and onwards, humans look more at a person's eyes than at other parts of the face (Haith ). Still, even healthy adults do not look at other people’s eyes all of the time during face perception (Janik ). In the present study, we sought to unravel how people neurophysiologically react when they are constrained to look at others consistently in the eyes compared to when they are free to allocate their gaze pattern during perception of faces with different emotional expressions. We also took an individual differences approach to this issue, since it has been shown that what counts as a ‘comfortable’ amount of eye contact varies between otherwise healthy individuals (Kleinke, 1986; Binetti ).

The presentation of faces activates a network of brain areas that have been coined as ‘the social brain’ (Kennedy and Adolphs, 2012). The social brain consists of the amygdala network (amygdala and orbitofrontal cortex), the mentalizing network (posterior superior temporal sulcus, temporal pole, posterior cingulate and dorso-median prefrontal cortex), the empathy network (insula and anterior cingulate), and the action-perception network (inferior frontal gyrus and superior parietal lobule). In addition, face-specific areas are located in the inferior occipital gyrus [referred as IOG—or occipital face area, OFA (Rossion )] and the fusiform gyrus [referred as the fusiform face area, FFA (Kanwisher )].

Activation of the fusiform gyrus by faces is modulated by where people are looking when face stimuli are presented. In their study published in 2007, Morris and colleagues demonstrated that activation in the fusiform gyrus correlates with the amount of fixation in the eyes (Morris ). Burra ) reported that the amygdala is also sensitive to the eye-contact effect, even in absence of primary visual cortex activation. The amygdala modulates activation of cortical areas during face perception, and the interaction between amygdala and the fusiform gyrus is an important component of face perception (Vuilleumier ; Vuilleumier and Pourtois, 2007). The amygdala is strongly connected with the prefrontal cortex (Ghashghaei ) and the dynamic interaction of the amygdala with the social brain modulates the recognition and modulation of affective states (Banks ; Miyahara ). Anatomical connectivity between the amygdala and brain areas involved in emotion perception is modulated by trait anxiety (Greening and Mitchell, 2015) and its functional connectivity can be pharmacologically altered during social-emotional processing (Gorka , 2015). Whether amygdala connectivity varies depending on the emotion presented and to which extent it is modulated not only by anxiety but also by other personality traits has, to our knowledge, not been specifically studied.

In their landmark paper published in 2009, Senju and Johnson (2009b) coined the term ‘eye contact effect’ to describe the modulation of cognitive processing following the perception of eye contact with another human. They demonstrated that eye contact can modulate activity in the social brain network, and developed the hypothesis that this is accomplished through the detection of perceived direct gaze by brain areas of the subcortical route. Recent studies have come to support the subcortical hypothesis, suggesting that automatic establishment of eye contact is mediated by fast subcortical pathways (Rothkirch ).

Direct eye contact is indeed a powerful signal—that also conveys different meanings depending on the facial expression of the person. Many studies examining the effect of eye-contact have not addressed the interaction between gaze and emotion (Calder ; George and Conty, 2008; Hietanen ; Wieser ; Helminen ), but those who did (Roelofs ; Soussignan ) concluded that reactivity and behavior was depending on the self-relevance of the combination of gaze and emotion in terms of approach or avoidance. When looking at the effect of gaze, its interaction with the emotional expression needs to be considered, as a direct gaze will have a completely different effect on the viewer depending on the facial expression it is presented with: a smiling face with a direct gaze can be interpreted as an invitation for further interactions, while an angry face with direct gaze signals a potential threat (Darwin, 1872/1965; Argyle and Cook, 1976; Kleinke, 1986; Baron-Cohen, 1995; Emery, 2000).

Emotional processing can be affected by several personality traits, such as anxiety, alexithymia, and the presence of autistic traits. Here, we tested whether these personality traits may affect how constrained gaze is perceived, for all emotions, as well as for specific emotional expressions; we also examined how amygdala connectivity is modulated by these personality traits, using questionnaires specifically addressing each of them. Previous studies have indeed demonstrated that anxious individuals have shortened viewing time in the eye-region (Daly, 1978; Garner ; Moukheiber ), and recent work from Myllyneva ) has revealed increased autonomic arousal and shorter self-controlled viewing time for direct gaze in adolescents with social anxiety disorders compared with controls. If we think of anxiety traits as a continuum in the population, we would expect that anxiety would be potentiating the effect of constraining the attention to the eye region of emotional faces and that this effect would be maximum for anger given that this emotion represents a direct threat for the observer. We first aimed to examine the effect of constraining gaze in the eye-region when looking at faces, by comparing the exact same stimuli in a free-viewing condition and in a condition where typical healthy participants were specifically asked to look at a cross-situated in the eye-region. Our second aim was to examine the effect of distinctive facial expression on differences in brain activation induced by constraining gaze, by looking at each facial expression separately. We used short movies of morphed facial expressions. We chose to use dynamic facial stimuli, as they are more ecological and are known to elicit increased activity in the social brain (Arsalidou ). It is important to note that we did not look at the effect of gaze direction, as all stimuli had a direct gaze.

In summary, we hypothesized that constrained eye-contact would increase activation in the social brain, and lead to increased connectivity between the amygdala and the rest of the social brain (Kawashima ; Huijgen ). We also hypothesized that anxiety, alexithymia and autistic traits would modulate functional connectivity between brain regions.

Materials and methods

Participants

The Lausanne University Hospital ethics committee approved all procedures. All adult participants gave written informed consent before the start of the study. Minor participants provided assent and one of their parents gave written consent. All procedures followed the Declaration of Helsinki. Twenty-five healthy participants were enrolled in the study. They had no history of psychiatric or neurological disorders. Five subjects were excluded from the data analysis due to excessive movement or for not performing the task during the scan. Thus, 20 participants (17 males, 23.5 years ± 8.14 (mean age ± SD), range 12.7–42.9) were included in the final analysis. Performance Intelligence Quotient (PIQ) was assessed using the Wechsler Nonverbal Scale of Ability (Wechsler and Naglieri, 2006) or the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999) and all participants had a PIQ in the normal range (mean: 113.4 ± 10.7).

Levels of anxiety were evaluated with the State-Trait Anxiety Inventory (STAI) (Spielberger ) in adults and with the Revised Children’s Manifest Anxiety Scale (RCMAS-2) in adolescents (Reynolds ). In addition, the Toronto Alexithymia Scale (TAS) was used in its adult and children version (Bagby ). Autistic traits were evaluated in all participants using the Autism Spectrum Quotient (AQ) self-report questionnaire (Baron-Cohen , 2006).

Visual stimuli and design

Twenty-four movies were created from the NimStim database (Tottenham ) representing morphs of facial expressions from NEUTRAL to FEAR, HAPPY or ANGER with Morph Age Pro (Creaceed). Each movie lasted for 5 s, and consisted of a dynamic morph lasting 3 s, followed by 2 s of the final emotional expression. Morphs of NEUTRAL were also created by creating a left-to-right morph between mirror images of neutral faces, in order to also have a dynamic component in this condition. A red fixation cross was present for 1 second between each movie, as well as in blocks of 6 s at the beginning and the end of the presentation, plus 7 short blocks of 3 s each interspersed with the face blocks. During a run, 2 blocks of 48 s of each expression were presented in a pseudo-randomized way. Two versions of these movies were created, with one version containing a red fixation cross in the region of the eyes. Each participant viewed both versions (CROSS and NO-CROSS) during the scanning session (that also comported other tasks not reported in the present manuscript). The order of the CROSS and NO-CROSS versions was counterbalanced across participants, so that about half of them saw the stimuli with NO-CROSS first. Participants were instructed to carefully look at the videos and, in order to monitor their attention, to press a button every time they saw a blue cross between the stimuli, which happened four times during CROSS and four times during NO-CROSS. The stimuli presented during CROSS and NO-CROSS were identical, the only difference was the presence of a fixation cross during the CROSS version.

Imaging data acquisition and analysis

Anatomical and functional MR images were collected in all participants with a 12-channel RF coil in a Siemens 3T scanner (Siemens Tim Trio, Erlangen) at the Centre d’Imagerie BioMédicale in Lausanne. Anatomical images were acquired using a multi-echo magnetization prepared rapid gradient echo sequence (ME-MPRAGE: 176 slices; 256x256 matrix; 1x1x1 mm voxels, echo time (TE): TE1: 1.64 ms, TE2: 3.5 ms, TE3: 5.36 ms, TE4: 7.22 ms; repetition time (TR): 2530 ms; flip angle 7°). Functional data were obtained using an echo planar imaging (EPI) sequence (47 AC-PC slices, 3 × 3 × 3 mm voxels, 64 × 64 matrix; FOV: 216; TE: 30 ms; TR: 3000 ms; flip angle 90°) lasting 425 s.

Functional MRI data processing, as well as preprocessing was carried out using FSL 5.0.2.2. Non-brain tissue was removed from high-resolution anatomical images using Christian Gaser's VBM8 toolbox for SPM8 (Ashburner ) and fed into feat. Data were motion corrected using MCFLIRT and motion parameters added as confound variables to the model. Residual outlier timepoints were identified using FSL’s motion outlier detection program and integrated as additional confound variables in the first-level general linear model (GLM) analysis. Preprocessing included spatial smoothing using a Gaussian kernel of 8 mm, grand-mean intensity normalization and high-pass temporal filtering with sigma = 50.0 s.

Subject-level statistical analysis was carried out for the contrasts [ALL EMO > FIXATION], as well as [NEUTRAL > FIXATION], [HAPPY > FIXATION], [ANGRY > FIXATION], and [FEAR > FIXATION] using FILM with local autocorrelation correction for both the CROSS and the NO-CROSS runs. Registration to high-resolution structural images was carried out using FLIRT. Registration to MNI standard space was then further refined using FNIRT nonlinear registration. For each subject, a fixed-effect analysis was used to compare activation for each emotion in the [CROSS > NO-CROSS] and the [NO-CROSS > CROSS] conditions. Group-level analyses for the [CROSS vs NO-CROSS] conditions were performed using mixed effects GLM analysis using FLAME 1 + 2 with automatic outlier detection. In modeling subject variability, this kind of analysis allows inference about the population from which the subjects are drawn. FMRI data processing was carried out using FEAT (FMRI Expert Analysis Tool) version 6.00, part of FSL FMRIB’s software library, www.fmrib.ox.ac.ul/fsl). Z statistic images were thresholded using clusters determined by Z > 2.3 and a corrected cluster significance threshold of P = 0.05 (Worsley, 2001). Cluster-corrected images were displayed on a standard brain (fsaverage for the surface and MNI-template for the volume).

Correlation analyses

Correlations were computed between full brain activation for each emotion and age, as well as with TAS, STAI trait, STAI state, and AQ. In addition, we examined activation and Spearman correlations in a series of anatomical ROIs belonging to the social brain, as defined by Kennedy and Adolphs (Kennedy and Adolphs, 2012). These consisted, for the amygdala network, of the amygdala (AMY) and the orbitofrontal cortex (OFC); for the mentalizing network of the right posterior STS, the temporal pole (TP), the posterior cingulate (post Cing), and the dorso-median prefrontal cortex (dmPFC), for the empathy network of the insula and the anterior cingulate (ant Cing), and for the action–perception network of the right inferior frontal gyrus (IFG) and the right superior parietal lobule (SPL). Activation was also examined in face specific areas including the right occipital fusiform gyrus and the right inferior occipital cortex (IOG).

Anatomical masks were created from the Harvard Oxford cortical and subcortical atlases For each subject, the mean percentage BOLD signal change was extracted for each of those regions of interest for the contrast [CROSS > NO-CROSS] for [ALL EMOTIONS], as well as [NEUTRAL], [HAPPY], [ANGRY], and [FEAR] conditions, using the Featquery tool in FSL.

PPI analyses

A psychophysiological interaction (PPI) was conducted to examine the effect of the presence of a fixation cross in the eye region for each emotion on amygdala functional connectivity with the rest of the brain. The relationships between strength of amygdala functional connectivity difference between CROSS and NO-CROSS conditions with STAI state and STAI trait scores, TAS and AQ (O'Reilly ) were also examined. Seed regions were anatomically defined in the left and right amygdala in MNI space, and transformed into each subject space. Time-courses of mean activity were extracted for the left and the right amygdala for each subject using fslmeants. Task specific changes were examined for each subject for each emotion and for each condition, using the mean-centered task time course and the demeaned seed ROI time course as described in (O'Reilly ). Then, within subject fixed-effect comparison was done for each emotion for each condition. Higher-level statistics were conducted using mixed-effect FLAME 1 + 2 for the CROSS vs. NO-CROSS condition for each emotion separately. Finally, we examined the correlations for each emotion between behavioral scores and the CROSS vs NO-CROSS condition PPI interaction.

Eye-tracking data of spontaneous face viewing

A subset of the participants (N = 13) took part in a parallel eye-tracking experiment of emotional face. In this experiment, similar NO CROSS face stimuli were shown while gaze patterns were recorded with a Tobii T 120 eyetracker during the first three seconds of presentation (Åsberg Johnels ). Analyzing this data allow us to get a picture of how the current sample spontaneously allocate their attention to the eyes in a non-constrained presentation. Proportional averaged fixation durations show that the participants viewed the eyes of the faces for 79, 77, 79, and 64% of the total gaze time for neutral, fearful, angry, and happy faces, respectively. This result is consistent with previous research in showing that neurotypical and healthy adults look more at people's eyes than at other parts of the face, although less than 100% of the time [around 70% of the time in (Pelphrey )]. It also mirror previous results in showing somewhat less eye gaze to happy facial expressions compared with other emotional expressions (all differences in eye gaze for happiness compared to the other emotions were P < 0.06 according to Wilcoxon signed ranks tests). This is due to the fact that a smiling mouth attracts relatively more attention (Eisenbarth and Alpers, 2011). Supplementary Table S14 specifies gaze patterns for each emotion to the eyes, the mouth as well as to non-eyes/non-mouth areas of the screen (i.e. the rest of the screen).

Results

Behavioral results

Anxiety scales

The adults (n = 15) had a mean T score of 46.13 ± 6.78 on the State-Trait Anxiety Inventory (STAI) trait scale (range: 33–57), and of 45.73 ± 5.93 on the STAI state scale (range: 34–54). Adolescents (n= 5) had a mean T score of 54.6 ± 6.87 on Revised Children’s Manifest Anxiety scale (RCMAS) (range: 49–66). We are reporting T-scores, as there are no clear-cut points for the anxiety scales. These scores mean that all participants had very low to moderate levels of anxiety, and none of them met the cut off score for severe anxiety.

Toronto alexithymia score (TAS)

The TAS 20 uses a 5-point Likert scale in adults, and a 2-point Likert scale in children and adolescents. In order to combine both, we used a transform from a 2 to 5 scale using the formula, x’ = 2x + 1, where x is the initial score and x’ the transformed score. The mean score for the group (n = 20) was 40.0 ± 12.3 (range: 17–59). None of the individuals met criteria for alexithymia in this study (cut off score ≥ 61), despite the individual variation we observed.

Autism spectrum quotient (AQ): None of the participants scored above the cutoff of 26 (mean: 12.85± 5.71, range 4–22) (Woodbury-Smith ).

Neuroimaging

Effect of age. We performed whole brain and ROI analyses with and without age as a regressor, and did not find a difference. Therefore, we are only reporting the analysis without the covariate of age in text, figures and tables.

Whole brain analyses

All-emotions combined

Free viewing of the stimuli [NO-CROSS > CROSS] did not elicit any increased activation compared to constrained viewing.

Adding a fixation cross to the stimulus [CROSS > NO-CROSS] significantly increased brain activation in a number of areas of the social brain. These consisted of facial visual processing areas (IOG, Fusiform gyrus) as well as areas from the amygdala network (AMY and OFC), the mentalizing network (posterior STS, temporal pole, dmPFC), the empathy network (INS, ACC) and the action perception network (IFG, IPL). In addition, activation was observed in the ventral striatum, and in the cerebellum (Figure 1 and Table 1).

Fig. 1.

Map of activation showing the effect of having a fixation cross in the eye region during the perception of dynamic emotional stimuli, with all conditions (NEUTRAL, HAPPY, ANGRY and FEAR) combined. Data are shown at P < 0.05, corrected.

Table 1.

Activation for the CROSS > NO CROSS CONDITION, all emotions analyzed together

CROSS>NO CROSS, all emotions
				MNI coordinates
Brain area	side	# voxels	Z-MAX	X	Y	Z
		21177
Inferior occipital gyrus	R		6.89	48	−64	−14
	L		4.11	−46	−78	−14
Lateral occipital cortex	R		4.29	42	−88	0
	L		6.49	−38	−84	22
Precuneus	L		5.71	−6	−76	34
	R		4.31	8	−74	44
Occipital Fusiform Gyrus	R		5.34	40	−62	−12
	L		4.73	−34	−78	−16
Intraparietal sulcus	L		4.74	−26	−80	30
	R		3.85	34	−74	34
Superior parietal lobule	R		5.21	32	−60	48
	L		4.87	20	−56	62
Posterior superior tem poral cortex	R		3.62	58	−20	0
	L		3.17	−58	−38	8
Orbitofrontal cortex	R	10129	5.58	42	20	−6
	L		3.65	−42	20	−12
Inferior prefrontal cortex	R		5.22	38	34	32
Temporal pole	R		4.81	50	14	−8
	L		4.32	−54	18	−8
Dorsolateral prefrontal cortex	R		4.22	28	58	18
	L		2.77	−42	40	18
Inferior frontal gyrus opercularis	R		4.65	58	20	18
Inferior frontal gyrus triangularis	R		3.98	46	30	4
Precentral gyrus	R		3.90	54	0	32
Insula	R		4.62	30	24	−4
	L		4.11	−42	18	−4
Putamen	R		3.07	18	12	−10
	L		2.85	−16	14	−4
Accumbens	L		3.60	−12	8	−10
	R		2.52	10	16	−6
Amygdala	R		3.08	16	−6	−14
		2542
Posterior cingulate	R		5.00	4	−28	42
	L		3.05	−12	−24	40
Anterior cingulate	R		4.37	4	0	36
	L		3.42	−4	26	30
Paracingulate gyrus	R		3.89	6	10	44
	L		3.85	−2	14	40
Precentral cortex	L		3.59	−30	−6	44
		1138	5.33	−28	56	15
Cerebellum VIIIa	L		4.28	−28	−44	−52
	R		3.53	30	−58	−56
Cerebellum VIIIb	R		3.08	16	−58	−54

Correlations with behavioral data and ROI data analyses are reported in supplementary material.

Differences between emotions

When each emotion was considered separately, we observed similar general patterns of activation, although different levels of activation were present for each emotion. The amount of significantly activated voxels (Z > 2.3, corrected cluster significance of P = 0.05) was the highest for ANGRY (4 cluster adding to 46’089 voxels, Zmax = 4.46), followed by NEUTRAL (5 clusters adding to 25’872 voxels, Zmax = 4.41), HAPPY (5 clusters adding to 22’606 voxels, Zmax = 4.22) and FEAR (5 clusters adding to 18’973 voxels, Zmax = 4.11) (Supplementary Figure S1 and Table S1, See supplementary material for more details).

Correlation with behavioral measures

STAIs: Levels of anxiety status predicted activation in the left TPJ for ANGRY and HAPPY faces, as well as in the SMA for ANGRY faces (Table 2). There were no correlations between STAIs and the rest of the brain for NEUTRAL and for FEAR faces.

Table 2.

Activation for the positive correlations with behavioral measures for the CROSS > NO CROSS condition for each emotion separately, Z > 2.3, P = 0.05

		ANGRY					NEUTRAL					HAPPY					FEAR
Brain area	Side	# voxels	Zmax	MNI			# voxels	Zmax	MNI			# voxels	Zmax	MNI			# voxels	Zmax	MNI
				X	Y	Z			X	Y	Z			X	Y	Z			X	Y	Z
STAI-S Positive correlation
TPJ	L	1170	4.09	−54	−32	28						1006	3.6	−54	−36	32
SMA	L/R	1665	3.77	2	−10	66
STAI-t Positive correlation
Insula	R	895	3.42	42	26	0	664	3.34	42	28	−2
	L	720	3.31	−36	10	4
SMA	L/R	1030	3.95	−16	6	70
DLPFC	L	731	4.31	−30	44	12											681	3.48	−28	44	14
TAS Positive correlation
Insula	L	889	3.12	−32	12	6
	R	1172	3.5	36	14	2
AQ Positive correlation
Precentral gyrus												868	3.75	50	8	42

STAIt. Levels of anxiety traits predicted activation in the right anterior insula for NEUTRAL, the left dlPFC and bilateral anterior insula and the SMA for ANGRY, and the left dlPFC for FEAR. There were however no correlations between STAIt and HAPPY.

TAS. Levels of alexithymia trait predicted activation in the insula, bilaterally, for ANGRY faces. There were no other correlations between TAS and the rest of the brain for the other emotions.

AQ. Levels of autistic traits predicted activation in the precentral cortex and the middle frontal gyrus for HAPPY faces, but not for other emotions.

PPI analysis results

NEUTRAL. PPI analysis revealed that the amygdala had increased functional connectivity in the CROSS vs NO-CROSS condition for neutral faces with the bilateral pericalcarine cortex, IOG, FFA, inferior temporal cortex, left STS, left TPJ, left superior frontal gyrus, PAG and the cerebellum (bilateral crus I and II, left VIIIb, left IX, vermis VI and right VI; Figure 2; Supplementary Table S7).

Fig. 2.

PPI analysis. Maps of increased connectivity between the amygdala and the rest of the cortex for NEUTRAL (top left), HAPPY (top right), ANGRY (bottom left) and FEAR (bottom right). Red-to-yellow depicts increased connectivity between amygdala and the rest of the brain for the CROSS condition, and cyan-to-blue depicts increased connectivity between the amygdala and the rest of the brain for the NO-CROSS condition. All data are shown at P < 0.05, corrected.

HAPPY. PPI analysis revealed that the amygdala had increased functional connectivity in the CROSS vs NO-CROSS condition for happy faces with the pericalcarine, cuneal and lateral occipital cortices bilaterally, the right STS, the bilateral orbitofrontal and medial prefrontal cortices, the right IFG, the right temporal pole and, as well as between the accumbens and the caudate bilaterally (Supplementary Table S8).

ANGRY. PPI analysis revealed increased connectivity between the amygdala and the hypothalamus, the thalamus, the left insula and the bilateral primary motor and somatosensory cortices for the CROSS vs NO-CROSS condition (Supplementary Table S9).

FEAR. For fearful faces, there were no differences in amygdala functional connectivity for the CROSS > NO-CROSS condition. However, there was an increase in amygdala functional connectivity for the NO-CROSS >CROSS condition in the bilateral orbitofrontal cortex, right medial prefrontal cortex, in bilateral anterior cingulate, bilateral insula, as well as in the right temporal pole, right superior frontal gyrus and occipital pole (Supplementary Table S10).

PPI analysis results—correlations with behavioral

All behavioral scores examined predicted the amount of functional connectivity of the amygdala with the rest of the cortex, however differently for each of the four emotions (Figures 3 and 4). More details are given in the supplementary material.

Fig. 3.

PPI analysis—correlation with STAI trait. Maps of increased connectivity between the amygdala and the rest of the cortex correlated for trait anxiety level for NEUTRAL (top left), HAPPY (top right), ANGRY (bottom left) and FEAR (bottom right). Red-to-yellow depicts increased connectivity between amygdala and the rest of the brain for the CROSS condition, and cyan-to-blue depicts increased connectivity between the amygdala and the rest of the brain for the NO-CROSS condition. All data are shown at P < 0.05, corrected.

Fig. 4.

PPI analysis—correlation with AQ. Maps of increased connectivity between the amygdala and the rest of the cortex correlated for trait anxiety level ANGRY (left) and FEAR (right). Red-to-yellow depicts increased connectivity between amygdala and the rest of the brain for the CROSS condition, and cyan-to-blue depicts increased connectivity between the amygdala and the rest of the brain for the NO-CROSS condition. All data are shown at P < 0.05, corrected.

Correlation between eye-tracking data and behavioral data. See Supplementary Material.

ROI analyses . The results of the ROI analyses are given in the Supplementary Material.

Discussion

In the present study, we investigated whether constraining participants’ attention to the eyes of dynamic emotional faces would enhance their brain response in the regions involved in perceiving social signals. In accordance with our hypothesis, we show that the activity in social brain regions is enhanced for all emotions when attention is constrained to the eye region. It is important to stress that the current study only includes healthy neurotypical adults who spontaneously already look more at the eyes than at other parts of people’s faces. Previous studies have shown, however, that the amount of time people spend looking at the eyes is less than 100% (Janik ; Pelphrey ), and that there are individual differences in what counts as ‘comfortable’ eye gaze time (Binetti ). Given that eye contact is so pivotal in human social interactions, and that information about abnormal eye contact is used during diagnostic for clinical conditions such as autism and schizophrenia, it is surprising that ‘normal’ eye contact behavior remains so ill understood (Binetti ). From this perspective, the current study contributes with new, valuable insights. Specifically, even in a group who spontaneously orient towards the eyes, constraining the gaze pattern to be located stably in the eye region produces neural activation patterns in the social brain markedly different from when these individuals are free to allocate their visual attention during face perception. Moreover, this constrained eye contact-effect has somewhat different signatures depending on the emotional expression. Finally, we also show that this effect is of importance for the purpose of understanding individual differences, since the magnitude of the effect for different emotions was modulated by individual differences in anxiety, alexithymia as well as autistic traits.

It might be surprising that the constrained eye contact effect was evident for all emotions including for NEUTRAL faces. A possible mechanism for this effect is that direct gaze is processed prior to affect evaluation and potentiates the process, as suggested by ERP studies showing that mutual eye-contact evokes an early response at 85 ms already, while the effect of the emotional expression is observed at around 115 ms (Pizzagalli ; Eimer and Holmes, 2002; Eger ; Klucharev and Sams, 2004). Still, although all emotions yielded more activation in the social brain in the constrained fixation condition, the effect was most pronounced for ANGRY faces. This is unsurprising, given that an angry face looking directly at us is considered particularly powerful (Tiedens, 2001) and elicits strong arousal (Garfinkel ). Angry faces are detected more efficiently (Fox ) as they potentially represent a biological threat (Stussi ). When we think of ourselves as less powerful than our angry social partner (e.g. child vs adult), this social partner represents a direct threat (Ewbank ), and our typical reaction is to avert our gaze in order to avoid confrontation (Marsh et al., 2005). In fact, when parents are angry and berate their kids, they very often say: ‘look me in the eyes’. Angry faces with direct gaze are indeed more self-relevant as they may signal a danger of being attacked (Sander ). They are processed more rapidly when they display a mutual gaze (Adams and Kleck, 2005; Graham and LaBar, 2007; Sander ; Bindemann ).

Constraining participants to look into the eyes of NEUTRAL faces had the next strongest effect on brain activation. NEUTRAL faces are often ambiguous, and can be perceived as emotionally negatively valenced (Thomas ; Donegan ; Somerville ; Iidaka ; Lobaugh ) and even threatening in socially anxious individuals (Yoon and Zinbarg, 2007, 2008). Here, we also observed that self-report of anxiety traits predicted activation in the right anterior insula, for NEUTRAL faces, an area involved in interoception and the representation of conscious emotional experience (Hogeveen ).

Constraining participants to look into the eyes of FEAR faces had the least effect. Given that fearful faces automatically attract attention in the eye-region (Schyns ), the eye-region might already have been fixated in the free-viewing condition and constraining the fixation there would therefore have little effect. In typical participants, it has been suggested that the amygdala triggers reflexive orientation towards fearful eyes (Gamer and Buchel, 2009; Gamer ). In addition, individuals do not direct their attention in the eye-region after amygdala damage, which prevents them to recognize fear (Adolphs ). However, it has also been shown that averted compared with direct gaze facilitate the identification of fear, in direct contrast to what is the case for more ‘approach-oriented emotions’ such as anger and happiness (Adams and Kleck, 2003).

Personality traits modulating constrained emotional face perception

Anxiety has been found to bias emotional processing (Rossignol ). Previous studies have reported that individuals with high levels of social anxiety tend to fixate less in the eye region especially when they process emotional faces (Horley ; Schulze ). In individuals with social anxiety disorder (SAD), gaze avoidance is present for both positive and negative social stimuli (Weeks ). This avoidance is associated with a stronger orienting towards gaze in angry faces in anxious individuals (Holmes ), demonstrating heightened sensitivity for these ‘threat’ stimuli. Although we measured general trait anxiety in this study, as opposed to specifically social anxiety, those are strongly associated. Imaging studies of SAD patients have shown that direct eye contact (vs. averted) in neutral faces increased activity in the vmPFC, the parahippocamal gyrus and the posterior cingulate (Schneier ) compared to controls, and adolescents with anxiety disorders have enhanced autonomic reaction and self-evaluated arousal during the perception of direct gaze (Myllyneva ). In the current study, a subset of participants were eye-tracked during perception of similar face stimuli as those used in the free-viewing condition in the scanner. Importantly, we found the association between diminished spontaneous eye gaze and anxiety traits described in previous research, while no clear associations were seen between eye gaze and either alexithymic or autistic traits. Even though these findings are dampened by the small sample size and that fact that the eye-tracking were performed outside the scanner, the results highlight a more general possibility that the modulation of brain activity in the constrained versus the unconstrained condition could reflect differences in eye fixations that occur during unconstrained face perception. Given that we find robust modulations in brain activity as a function of all these personality traits, an important avenue for future research would be to parse the brain-behavior associations between specific patterns of both spontaneous and constrained eye contact and a range of personality traits in larger groups, including in clinical samples.

In accordance with our hypothesis, we found that anxiety potentiated the effect of constraining the attention to the eye region of emotional faces - especially for anger because it represents direct threat.

Alexithymia, a personality trait characterized by the difficulty to identify and describe emotions, has been associated with deficits in detecting and matching emotional facial expression in healthy populations [for review, see Grynberg ]. People suffering from alexithymia are impaired at recognizing others’ facial expressions (Parker ; Mann ; Prkachin ), possibly because of their atypical attention to the eyes in faces (Bird ). None of the participants had TAS scores that would lead to a diagnosis of alexithymia (61 and above). Yet, we found positive correlations between alexithymia scores and level of activity in the anterior insula for all four emotions for the CROSS vs NO-CROSS comparison, indicating that the more difficulties participants had with identifying their own emotion, the more they showed increase in insula activation when having to sustain direct gaze vs looking at faces with no constraints. The strongest correlation between alexithymia and brain activation was found for the ANGRY condition and the more ambiguous NEUTRAL condition.

In addition, for the ANGRY and the NEUTRAL condition we found that alexithymia scores were positively correlated with the strength of connectivity between the amygdala and the mPFC, the STS, the OFC and the vmPFC, as well as the ventral striatum, the thalamus and the brainstem. The present results therefore support the view that alexithymia and anxiety both play an important role in gaze perception (Cook ).

Autistic traits. There has been a debate as to whether the social and communication impairments of those with autism result from their difficulty attending the eye region [for review see Senju and Johnson (2009a)]. In the present study conducted in the general population, the only correlation between social brain activation and autistic traits was found in the fusiform gyrus when all emotions were combined. The fusiform gyrus is involved in processing invariant aspects of faces [see Haxby for a review]. The reason why some people with high levels of autistic traits attend less to the eye region of faces is unclear but could be due to an active avoidance of this facial area due to its arousing properties [see Tanaka and Sung (2016) for an in-depth description of this hypothesis].

The PPI analysis was performed to specifically look at the influence of constrained direct gaze on amygdala connectivity with the rest of the brain for different emotional expressions. In her pioneering study, George et al. (2001) reported that direct gaze (in neutral faces) increases the functional connectivity between the amygdala and the fusiform gyrus, consistent with the concept of the modulation of the fusiform gyrus by amygdala activation. Using fMRI, Sato et al. demonstrated that the left amygdala showed increased activation to angry faces with a direct gaze compared with angry averted or neutral direct or averted (Sato .) Here, we show that constraining gaze to the eye region has a very different effect depending on which emotion is processed, increasing connectivity in different areas of the brain for NEUTRAL, HAPPY and ANGRY faces, while reducing it for FEAR. For NEUTRAL faces, as expected we observed increased connectivity for the CROSS vs NO-CROSS condition essentially in the face-processing network, whilst for HAPPY faces, effects were observed not only in the visual cortex, but also in areas involved in emotional and reward processing, including the ventromedial and orbital prefrontal cortex, the temporal pole and ventral striatum. Constraining gaze into the eyes of ANGRY faces increased the connectivity of the amygdala with the somatosensory and motor cortex, in the region of the faces, as well as in the hypothalamus, the thalamus and the insula. Activity in the hypothalamus may represent a stress response to having to look into the eyes of a threatening face, and it is noteworthy that ANGRY faces were the only emotion where we could observe increased amygdala connectivity with the hypothalamus. Note that the same areas (insula and hypothalamus) are involved in social regulation of neural response to threat (Coan ).

A limitation of this study is that we could not collect eye-tracking while the participants were engaged in face perception in the scanner. Such information would have helped us better characterize the participants’ behavioral response to the constrained gaze manipulation. Even so, our data show that constraining typically developed participants to look consistently into the eyes of neutral and emotional dynamic face stimuli remarkably alters brain activation. They show that this effect is the most pronounced for threatening social stimuli (anger). Finally, our data demonstrate that amygdala connectivity varies in function of the emotional expression presented, and is modulated by behavioral traits such as alexithymia, anxiety and autistic traits.

Supplementary data

Supplementary data are available at SCAN online.

Click here for additional data file.