Neural correlates of "social gaze" processing in high-functioning autism under systematic variation of gaze duration.

Direct gaze is a salient nonverbal signal for social interest and the intention to communicate. In particular, the duration of another's direct gaze can modulate our perception of the social meaning of gaze cues. However, both poor eye contact and deficits in social cognitive processing of gaze are specific diagnostic features of autism. Therefore, investigating neural mechanisms of gaze may provide key insights into the neural mechanisms related to autistic symptoms. Employing functional magnetic resonance imaging (fMRI) and a parametric design, we investigated the neural correlates of the influence of gaze direction and gaze duration on person perception in individuals with high-functioning autism (HFA) and a matched control group. For this purpose, dynamically animated faces of virtual characters, displaying averted or direct gaze of different durations (1 s, 2.5 s and 4 s) were evaluated on a four-point likeability scale. Behavioral results revealed that HFA participants showed no significant difference in likeability ratings depending on gaze duration, while the control group rated the virtual characters as increasingly likeable with increasing gaze duration. On the neural level, direct gaze and increasing direct gaze duration recruit regions of the social neural network (SNN) in control participants, indicating the processing of social salience and a perceived communicative intent. In participants with HFA however, regions of the social neural network were more engaged by averted and decreasing amounts of gaze, while the neural response for processing direct gaze in HFA was not suggestive of any social information processing.

Introduction

One of the core deficits in autism spectrum disorders (ASD) concerns the adequate interpretation of nonverbal behaviors, an ability that is essential for successful social interactions between humans (Baron-Cohen et al., 1999; Centelles et al., 2013; Ogai et al., 2003). In particular, gaze behavior serves important functions in social encounters by facilitating the understanding of another person's mental states and allowing for the coordination of attention and activities (Argyle and Cook, 1976; Argyle and Dean, 1965; Emery, 2000; Kleinke, 1986; Pierno et al., 2008; Schilbach et al., 2010). For instance, the direction of perceived gaze is important, with direct gaze expressing interest and the intention to communicate (Argyle and Cook, 1976; Argyle and Dean, 1965; Emery, 2000; Kampe et al., 2003; Kleinke, 1986).

However, behavioral studies have repeatedly demonstrated that direct gaze does not elicit the so-called “eye contact effect” in individuals with ASD. This means that perceived eye contact is neither preferred by nor does it modulate cognition and attention in persons with ASD (for a review, see Senju and Johnson, 2009a). Moreover, they are impaired in reading others' mental states from the eye region (Baron-Cohen, 1997; Baron-Cohen et al., 1997, 2001a). Thus, it has been suggested, that such gaze processing deficits in ASD result from an impairment to extract socially relevant information from the eye region, hence indicating that social cues are less intrinsically salient for autistic persons (Nation and Penny, 2008; Pelphrey et al., 2005a; Ristic et al., 2005; Senju and Johnson, 2009a).

In search of the neural correlates of the processing of social gaze, neuroimaging studies have focused to a large degree on the processing of gaze direction in various contexts. Electrophysiological evidence has robustly indicated differential neural activity for direct gaze versus averted gaze (Conty et al., 2007; Gale et al., 1975; Hietanen et al., 2008a, 2008b; Senju et al., 2005). FMRI studies have further explored the specific brain regions involved in processing gaze direction (for reviews, see Grosbras et al., 2005; Itier and Batty, 2009; Nummenmaa and Calder, 2009; Senju and Johnson, 2009b). In a recent review, Senju and Johnson (2009b) summarize that a total of six regions have been reported to show differential activity between direct and averted gaze, namely the fusiform gyrus (FG), the posterior superior temporal sulcus (pSTS), the dorsomedial prefrontal cortex (dmPFC), the orbitofrontal cortex (OFC) and the amygdala. These regions are known to be part of the so-called “social neural network” (SNN), which is involved in conscious mental inference and evaluation of social stimuli (Frith, 2007; Gallagher and Frith, 2003; Van Overwalle and Baetens, 2009; Vogeley and Roepstorff, 2009). To our knowledge, only two fMRI studies have investigated the neural processing of direct compared to averted gaze in individuals with ASD relative to a control group (Pitskel et al., 2011; von dem Hagen et al., 2013). Both studies confirmed a network of SNN regions sensitive to direct gaze versus averted gaze in typically developing participants. On the other hand, the SNN was not preferentially active when perceiving direct gaze in participants with ASD.

However, dynamic aspects of gaze behavior have not been investigated comprehensively so far, despite the fact that they are known to modulate the communicative content transmitted by the eyes (Argyle and Cook, 1976; Kleinke, 1986; Kuzmanovic et al., 2009). For instance, a complex source of social information is the duration of perceived eye contact. In order to adequately interpret it, more elaborate mentalizing abilities are required (Eskritt and Lee, 2007). Humans learn to use relative gaze duration towards different objects in the environment to infer other people's preferences only during later developmental stages (Einav and Hood, 2006; Montgomery et al., 1998).

To our knowledge, this is the first investigation of the processing of both gaze direction and duration in adults with high-functioning autism (HFA) and a matched control group. For this purpose, the current study made use of a parametric design and a person perception task, previously introduced by Kuzmanovic and colleagues (2009). Participants watched dynamically animated faces of anthropomorphic virtual characters while undergoing fMRI, and were asked to rate on a four-point scale how likeable they perceived each virtual character to be. To estimate the impact of gaze direction and gaze duration on person perception, these variables were systematically manipulated. We assumed that, in the control group, direct compared to averted gaze would activate the pSTS, a region that has been robustly linked to the perception of gaze behavior (Bristow et al., 2007; Calder et al., 2002; Ethofer et al., 2011; Kuzmanovic et al., 2009; Pelphrey et al., 2004; Pitskel et al., 2011; von dem Hagen et al., 2013; Wicker et al., 2003) and that increasing gaze duration would engage the medial prefrontal cortex, a region associated with the evaluation of social stimuli (Amodio and Frith, 2006; Zysset et al., 2002). We further assumed that these effects would be weaker or absent in participants with HFA, given that direct gaze may hold less salience for them.

Materials & methods

Subjects

A group of 13 HFA individuals and a group of 13 matched control persons participated in this study (see Table 1). All subjects were right-handed, as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971), reported normal or corrected-to-normal vision and were naïve with respect to the purpose of the study.

Table 1

Demographic and neuropsychological data.

	Age	AQ	WST	Gender (m/f)
HFA (n = 13)	31.23 ± 4.87	38.31 ± 4.05	108.46 ± 8.1	9/4
CON (n = 13)	30.23 ± 3	13.85 ± 3.63	108.92 ± 9.23	9/4
t-Test	p = .536	p < .001	p = .893	–

Mean values and the respective standard deviations are displayed; HFA = high-functioning group; CON = control group; WST = German multiple-choice verbal IQ test (“Wortschatztest”); AQ = Autism Spectrum Quotient.

The 13 HFA participants (9 male) were between 24 and 39 years of age (M = 31.23, SD = 4.87) and were diagnosed and recruited in the Autism Outpatient Clinic at the Department of Psychiatry of the University Hospital of Cologne in Germany. HFA, as part of the autism spectrum, is characterized by sociocommunicative impairments on the one hand but intact non-social cognitive capacities on the other (Klin, 2006). Moreover, the brain structure of individuals with HFA appears to be less impaired compared to other conditions within the spectrum. For instance, investigations carried out in our group revealed only limited local areas with cortical thinning, especially in the left posterior superior temporal sulcus (Scheel et al., 2011) and no difference in the size of the corpus callosum (Tepest et al., 2010). As part of a systematic assessment, the diagnoses were confirmed by clinical interviews according to ICD-10 criteria by two specialized physicians and were supplemented by extensive neuropsychological assessment. The sample included patients with the diagnoses Asperger syndrome/high-functioning autism with an at least average Full Scale IQ (FSIQ > 85, measured using Wechsler Adult Intelligence Scale, WAIS). Thus, we henceforth use the term HFA to refer to individuals with ASD and a high intellectual level of functioning. None of the HFA participants were taking any psychotropic medications except for two who were taking an antidepressant medication (Citalopram 40 mg/day and Cymbalta 30 mg/day, respectively). Additionally, three HFA participants reported episodes of depression in their past medical history. As depression is a common co-morbidity in HFA (Lehnhardt et al., 2011; Stewart et al., 2006), they were not excluded from the sample.

The 13 control participants (9 male) were between 24 and 36 years of age (M = 30.23, SD = 3) and were recruited online from the under- and graduate students at the University of Cologne in Germany. They reported no history of psychiatric or neurologic disorders, and no current use of any psychoactive medications. In order to avoid clinically significant autistic traits in the control sample, control participants were included only if scoring less than 22 on the Autism Quotient (AQ) (Baron-Cohen et al., 2001b).

Intelligence in both diagnostic groups was assessed by the German multiple-choice verbal IQ test (“Wortschatztest”, WST; see Table 1). Known to provide a valid and time-effective estimate of intelligence (Lehrl et al., 1995; Satzger et al., 2002; Schmidt and Metzler, 1992), the WST has been used in previous studies for matching purposes (David et al., 2010, 2011; Kuzmanovic et al., 2011; Scheel et al., 2011; Schilbach et al., 2012).

Written informed consent was obtained from all participants and they were informed of the necessary safety precautions involving fMRI experiments prior to the scanning session. All participants received a monetary compensation for their participation of 15 Euros per hour. The study was conducted with the approval of the local ethics committee of the Medical Faculty of the University of Cologne.

Stimuli & design

The current paradigm has a two by three factorial design with the two factors (a) “gaze direction”, varied on two levels (direct or averted) and (b) “gaze duration”, varied on three levels (1, 2.5 and 4 s). The stimulus material was made up of dynamic displays of 20 computer-generated faces (10 male, 10 female) created using the commercially available 3D animation software package Poser 6.0 (Curious Labs Inc., Santa Cruz, USA). Virtual characters were used instead of real faces due to their advantage of a high degree of standardization and systematic manipulability, which constitute important prerequisites enabling the investigation of subtle nonverbal signals such as gaze behavior (Bente et al., 2001a, 2001b; Vogeley and Bente, 2010). Each trial began with the display of a face, the gaze of which was initially averted. After a short blink (150 ms), the character directed its gaze toward the participant and after a variable period of time (depending on the condition, either 1, 2.5 or 4 s), the virtual character looked again away by shifting its gaze back to the initial position (see Fig. 1). The duration of the initial and final averted gaze within a direct gaze trial was adjusted according to the respective duration of the direct gaze condition in order to establish an equal total duration of 5.65 s for all animations (see Fig. 1). Conditions with direct gaze were complemented by a condition in which the virtual character expressed averted gaze throughout, i.e. it did not include any gaze shifts away from the initial position. To keep the conditions comparable and to maintain the natural appearance, the eye-blink occurred in the averted gaze condition as well. The task required participants to watch each animation and evaluate the likeability of the presented animated characters on a four-point likeability scale, with the response options 1 (“very dislikeable”), 2 (“rather dislikeable”), 3 (“rather likeable”) and 4 (“very likeable”).

Fig. 1

A. Experimental design. B. An example of a virtual face stimulus and a sample direct gaze trial. The participants' task was to observe and rate the perceived likeability of each face on a 4-point scale.

Experimental procedure

An experimental trial consisted of a stimulus presentation lasting for 5.65 s, followed by a four-point likeability rating scale lasting for 1 s. Further, each trial entailed two randomly jittered inter-stimulus intervals (ISIs): one between each stimulus presentation and the following rating scale (applied ISI durations: 1.55 s, 1.75 s, 2.25 s and 2.5 s; mean ISI 2 s) and the other between single trials to increase condition-specific BOLD signal discriminability (Dale, 1999; Serences, 2004) (applied ISI durations: 5.4 s, 6.33 s, 7.2 s and 8.1 s; mean ISI: 6.75 s). An average trial lasted for 15.4 s. Each of the twenty stimulus faces was provided in two versions (head orientation towards right or left side), summing up to a total of 160 trials. The experiment was conducted in an event-related fashion and split into two runs each lasting for 20 min. Both runs consisted of equivalent numbers of condition-specific events, shown in randomized order. The sequence of the two runs was randomized as well. A break of approximately 2–4 (two to four) min was taken between runs.

Prior to the fMRI experiment all participants were introduced to the task by a standardized instruction and practice session presented on a computer screen outside the MRI environment. None of the stimuli used in the introduction were used in the subsequent fMRI experiment. Participants were told that they would see short animations of virtual faces which they should watch carefully and that, after each animation, they would be asked “How likeable did the face appear to you?”, to respond by pressing one of four buttons corresponding to a four-point scale which would appear on screen. Additionally, subjects were instructed to focus on the fixation cross between trials and to rate on the displayed scale as intuitively and quickly as possible.

To balance for lateralized motor-related activations, participants alternately used the right or left hand across runs. The stimulus presentation and response recording were performed by the software package Presentation (version 12.2; Neurobehavioral Systems, Inc., www.neurobs.com/) and responses were assessed using four buttons of a MR-compatible handheld response device (LUMItouch™, Photon Control Inc., BC, Canada).

Data acquisition

Functional magnetic resonance imaging (fMRI) was performed on a Siemens 3 T whole-body scanner, which was equipped with a standard head coil and a custom-built head holder for movement reduction (Siemens TRIO, Medical Solutions, Erlangen, Germany). For the fMRI scans we used a T2*-weighted gradient echo planar imaging (EPI) sequence with the following imaging parameters: TR = 2200 ms, TE = 30 ms, field of view = 200 × 200 mm2, 36 axial slices, slice thickness 3.0 mm, in-plane resolution = 3.1 × 3.1 mm2. Each session consisted of 574 volumes preceded by 4 additional volumes allowing for T1 magnetic saturation effects. These 4 images were discarded prior to further image processing.

Behavioral data analysis

The subjects' rating scores for each condition level were mean averaged. Subsequently, the overall effect of gaze duration on individual ratings as well as group differences were tested using SPSS (PASW Statistics 18) by a two way mixed analysis of variance (ANOVA) with group (HFA vs control) as a between-subject factor and direct gaze duration (codes 1 to 3 for the different gaze durations) as a within-subject factor. If Mauchly's test indicated that the assumption of sphericity was not fulfilled, degrees of freedom were corrected using the Greenhouse–Geisser estimates of sphericity. Planned polynomial contrasts were applied for trend analyses. Pairwise comparisons (Bonferroni corrected post-hoc tests) were performed to better characterize the nature of the significant main effect of gaze duration. The trials with averted gaze were excluded from this analysis as their primary purpose was to provide a control condition for the fMRI paradigm (i.e. a “high-level baseline”). Nevertheless, paired sample t-tests were performed to test whether the averted gaze condition was rated significantly different compared to the direct gaze conditions. All effects are reported as significant at p < .05.

FMRI data analyses

FMRI data were spatially preprocessed and analyzed using SPM5 (Wellcome Department of Imaging Neuroscience, London, UK) implemented in Matlab 6.5 (The MathWorks, Inc., Natick, USA). After the functional images were corrected for head movements using realignment, the mean functional image for each participant was computed and coregistered to the Montreal Neurological Institute (MNI) reference space using the unified segmentation function in SPM5. The ensuing deformation was subsequently applied to the individual functional volumes. Functional images were then spatially smoothed with an isotopic Gaussian filter (8 mm full width at half maximum) to meet the statistical requirements of further analyses and to account for macroanatomical interindividual differences across participants.

The data were analyzed using a General Linear Model as implemented in SPM5. The analysis followed a combined categorical-parametric design that allowed us to characterize different forms of responses to direct gaze: (i) the categorical response to the presence of direct or averted gaze (DG vs AG and AG vs DG) and (ii) the parametric response to varying gaze durations within the direct gaze condition by identifying brain regions where activations increase or decrease linearly with increasing direct gaze duration (DGd + and DGd −).

At the single subject level, conditions DG and AG were modeled separately using a boxcar reference vector convolved with the canonical hemodynamic response function. Events were defined by onsets of corresponding stimulus presentations, whereas durations always amounted to 5.65 s according to the duration the virtual character was present on screen. Within this categorical framework, the effect of DGd was modeled as a linear parametric modulation of the hemodynamic response to DG by the corresponding duration (1, 2.5, 4 s). Taken together, two types of events (AG, DG) and one event parameter of interest (DGd) were included in the statistical analysis at the single subject level. Additionally, another two regressors were added to the model (one for either hand). Here, the duration of all response events amounted to 1 s according to the time the rating scale was present on screen. Head movement estimates were included as regressors to remove movement-related variance from the image time series. Thereby, all events were computed against resting baseline by weighting only the regressor corresponding to that particular event with “1” and all other regressors with “0”. Only in the case of response events, both hand regressors were weighted with “1”.

The performed single-subject contrasts were then fed into the 2nd level group analysis using a flexible factorial ANOVA (factors: group, condition and subject), employing a random-effects model (Penny et al., 2003). First, the group-level analysis evaluated which brain regions were differentially active while watching direct gaze versus averted gaze (and vice-versa) for the control group and the HFA group, together and separately. The following t-contrasts were computed: (i) DG > AG, (ii) AG > DG, (iii) HFA_DG > HFA_AG, (iv) HFA_AG > HFA_DG, (v) CON_DG > CON_AG, and (vi) CON_AG > CON_DG. Second, the main effect of gaze duration was calculated. The following t-contrasts were computed for both groups separately and together: (i) DGd +, the positive effect of gaze duration, that is, brain regions with increased neural activation corresponding to increases in perceived gaze duration, and (ii) DGd −, the negative effect of gaze duration, that is, brain regions with increased neural activation corresponding to decreases in perceived gaze duration. Significant Group × Condition interactions ((DG > AG) × (CON > HFA) and (DG > AG) × (HFA > CON)) were investigated in order to see whether the effect of stimulus condition varied as a function of group membership.

At the group level, all effects are reported as significant at p < .05, corrected for multiple comparisons at the cluster level (pFWEcorr) with p < .001, uncorrected, at the voxel level (Friston et al., 1996). Functional activations were anatomically localized by using the brain atlas by Duvernoy (1999) and the SPM anatomy toolbox, version 1.7 (Eickhoff et al., 2005), implementing a maximum probability map. Group activation maps were superimposed on an SPM canonical T1-weighted image. Reported coordinates refer to maximum values in a given cluster according to the standard MNI template.

Eye tracking data

Due to technical difficulties with the recording hardware, eye tracking could not be performed reliably during fMRI and eye movements of the participants could hence not be considered. However, we were interested in investigating whether individuals with HFA and control persons would differ in the visual exploration of faces while performing the likeability rating task. Therefore we tested a gender-, age- and verbal intelligence-matched sample consisting of a group of 6 high-functioning individuals with ASD (4 male; mean age 32.7 years, standard deviation (SD) = 3.6 years) and 6 control participants (5 male; mean age 28.8 years, SD = 3.5 years) in a follow-up experiment. Eye movements were monitored at a frequency of 50 Hz and recorded using TOBII systems eyetracking technology. For the statistical analysis the eye tracking data were first inspected in order to remove saccades and identify fixations. To this end, a MATLAB (Version 7.1, MathWorks, Natrick, MA) dispersion-based identification algorithm was developed. This algorithm uses a Dispersion-Threshold Identification approach and determines fixations based on both a priori defined dispersion and duration criteria (Falkmer et al., 2008; Salvucci and Goldberg, 2000). To detect potential fixations, the algorithm uses a sliding window method (Salvucci and Goldberg, 2000), which encompasses a minimum number of chronological data points and checks whether the criteria are met. Further, facial regions of interest (ROIs) were defined. These areas were based on the core facial features such as forehead, eyes, nose, mouth including the chin area, as well as a category for the rest of the face. Mean fixation frequencies were calculated and a mixed design ANOVA was performed for each ROI. The analysis was performed both with absolute as well as with relative fixation frequencies (i.e. fixation frequencies towards a particular ROI relative to the total fixation frequencies to the whole face). A two-factorial mixed design ANOVA was used for each ROI separately, with the repeated-measures variable “gaze duration” and the between group variable “group”.

Results

Behavioral results

The behavioral analysis revealed no main effect of gaze duration (F(2, 48) = 1.1, p = .34) or group (F(1, 24) = 2.92, p = .1); however the interaction effect between the two factors gaze duration and group approached significance (F(2, 48) = 2.83, p = .07). When looking at the two groups separately, a significant main effect of gaze duration was only found in the control group (F(1.13, 13.49) = 6.74, p < .05), but not in the HFA group (F(2, 24) = .67, p = .52; see Fig. 2). The pairwise comparisons within the control group showed a significant difference between mean likeability ratings for the 1 s versus 2.5 s condition (p = .006) and a trend toward significance between the 1 s and 4 s condition (p = .08). In addition, for control participants, polynomial contrasts revealed both a significant linear trend (F(1,12) = 6.41, p < .05) and a significant quadratic trend (F(1,12) = 11.82, p < .005) for the gaze duration condition in the control group. In the HFA group, neither of these trends was significant. Across both groups however, paired-samples t-tests showed that the averted gaze condition was rated significantly lower than the 1 s (t(25) = − 2.78, p < .05) and 2.5 s conditions (t(25) = − 2.6, p < .05). The difference between the averted gaze and the 4 s direct gaze condition only approached significance (t(25) = − 1.87, p = .07).

Fig. 2

The plot illustrates the effects of gaze duration on likeability ratings. The scores on the y-axis indicate the mean of stimuli ratings. A score of 1 refers to rating a face as “dislikable” and one of 4 as “likeable”. Error bars show 1 standard error of the mean.

Neural results

First, we identified brain regions in each group of participants that responded more strongly to direct gaze compared to averted gaze (DG > AG) as shown in Fig. 3 and Table 2. In the control group, activity was localized bilaterally in the STG, the pSTS, and the MT/V5 area, as well as the left paracentral lobule. Furthermore, in the right hemisphere, the supramarginal gyrus/TPJ, the PCun and the insular cortex responded more strongly to direct than to averted gaze. In HFA individuals, the same contrast yielded activations solely in the right pSTS.

Fig. 3

A. Differential neural activity for observing direct compared to averted gaze in control participants. B. Differential neural activity for observing direct compared to averted gaze in HFA participants. C. Differential neural activity associated with the group × gaze interaction; plots illustrate corresponding contrast estimates obtained for the four stimulus categories for three different local maxima: right PCun (11, − 50, 60), left mOFC (− 2, 48, − 21) and left TPJ (− 44, − 65, 20). Error bars represent confidence intervals. D. Differential neural activity for observing averted compared to direct gaze in HFA participants. The principally activated voxels are overlaid on the mean structural anatomic image of the 26 participants: p < .001, cluster-level corrected; DG = direct gaze; AG = averted gaze; CON = control group; HFA = high-functioning autism group; PCun = precuneus; mOFC = medial orbitofrontal cortex; TPJ = temporoparietal junction.

Table 2

Effects of gaze direction.

Region	Cluster-level		Side	MNI coordinates			T
	Size	pFWE-corr		x	y	z
Gaze direction
DG > AG controls
MT/V5	1411	0.000	R	47	− 68	− 2	6.47
MT/V5	1077	0.000	L	− 45	− 72	2	5.57
Rolandic operculum	783	0.000	R	48	2	6	5.24
Insula			R	41	8	3	5.02
Precuneus	390	0.004	R	12	− 51	66	4.95
Paracentral lobule?	1064	0.000	L	− 5	− 35	60	4.13
Superior temporal gyrus	322	0.023	L	− 51	− 32	8	4.44
Superior temporal gyrus	1237	0.000	R	57	− 41	12	4.44
Temporoparietal junction/supramarginal gyrus			R	47	− 36	23	4.42
Posterior superior temporal sulcus			R	65	− 47	15	3.95
DG > AG HFA
Posterior superior temporal sulcus	283	0.040	R	66	− 45	6	4.73
AG > DG HFA
Posterior cingulate cortex/precuneus	4642	0.000	R	11	− 60	23	5.69
Temporoparietal junction	734	0.000	R	45	− 63	23	5.47
Fusiform gyrus	356	0.015	L	− 32	− 33	− 18	5.20
Parahippocampal gyrus			L	− 26	− 36	− 12	4.02
Middle frontal gyrus	310	0.028	L	− 38	23	48	5.12
Middle temporal gyrus	1182	0.000	L	− 59	− 12	− 15	4.99
Inferior temporal sulcus			L	− 47	− 6	− 33	4.78
Rectal gyrus	535	0.002	L	− 9	35	− 29	4.97
Parahippocampal gyrus	321	0.024	R	24	− 33	− 14	4.90
Fusiform gyrus			R	32	− 41	− 9	4.13
Temporoparietal junction	662	0.000	L	− 45	− 66	21	4.90
Inferior temporal gyrus/sulcus	544	0.001	R	56	− 14	− 27	4.58
Middle temporal gyrus			R	51	− 9	− 20	4.53
(DG > AG) × (Controls > HFA)
Temporoparietal junction	868	0.000	L	− 44	− 65	20	5.21
Subcallosal gyrus	1209	0.000	L/R	0	15	− 18	4.88
Rectal gyrus/mOFC			L	− 2	48	− 21	4.58
Cuneus	598	0.001	R	12	− 60	21	4.85
Middle temporal gyrus	347	0.017	L	− 60	− 12	− 20	4.76
Precuneus	450	0.004	R	11	− 50	60	4.56
Middle temporal gyrus	270	0.021	R	57	− 14	− 18	4.48
Temporoparietal junction	549	0.001	R	48	− 68	14	4.32

Abbreviations: T = t-values of regions active in each contrast; L = left hemisphere; R = right hemisphere; MT/V5 = middle temporal area.

Second, we identified brain regions in each group of participants that responded more strongly to averted gaze compared to direct gaze (AG > DG; Fig. 3; Table 2). In the control group, this contrast did not yield any significant results. In the HFA group the same contrast yielded activations in the PCun and PCC, the left middle and superior frontal sulcus, as well as the mOFC. Other regions identified as differentially responsive were distributed bilaterally among the TPJ (localized in the posterior terminal ascending branch of the STS), the inferior temporal cortex, including the FG and the parahippocampal gyri.

The analysis of the group × condition interaction evaluating brain regions more responsive to direct than to averted gaze in the controls versus the HFA, revealed activations in the mOFC, the right Cun and PCun, left MTG, extending to the aSTS and bilaterally the TPJ (localized in the posterior terminal ascending branch of the STS; Fig. 3; Table 2). The interaction evaluating brain regions more responsive to direct than to averted gaze in HFA versus controls, did not reveal any significant differential neural response.

Further, we tested for the first-order parametric modulation of direct gaze in order to identify regions where the activation increased (or decreased) linearly with an increasing duration of direct gaze. The analysis showed that brain activity in the control group was modulated by gaze duration in the left TPJ (localized in the posterior terminal ascending branch of the STS) and dACC, whereas there was no significant modulation by DGd in any brain regions for the HFA group (see Fig. 4, Table 3). In the direct group comparison, the control participants showed significantly greater correlation of the DGd with the activity in the mOFC, left insula and dACC (see Fig. 4, Table 3). No brain region showed significantly greater activation for this contrast in the HFA compared to the control group. Decreasing gaze duration experience was associated with an engagement of the PCun only in the HFA group (see Fig. 4, Table 3).

Fig. 4

A. Neural activation associated with increasing gaze duration for the control group. B. Direct group comparison between the control and HFA group for the neural processing of increasing gaze duration. Plots illustrate corresponding contrast estimates obtained for the four stimulus categories for three different local maxima: left dACC (− 9, 33, 15), right mOFC (11, 38, − 17) and left insula (− 38, − 9, − 6). Error bars represent confidence intervals. C. Neural activation associated with decreasing gaze duration for the HFA group. The principally activated voxels are overlaid on the mean structural anatomic image of the 26 participants: p < .001, cluster-level corrected; DGd + = increasing direct gaze duration; DGd − = decreasing direct gaze duration; CON = control group; HFA = high-functioning autism group; dACC = dorsal anterior cingulated cortex; mOFC = medial orbitofrontal cortex.

Table 3

Effects of gaze duration.

Region	Cluster-level		Side	MNI coordinates			T
	Size	pFWE-corr		x	y	z
Gaze duration
Increasing controls
Dorsal anterior cingulate cortex	810	0.000	L	− 9	33	15	5.21
Dorsal anterior cingulate cortex			R	5	30	18	4.23
Temporoparietal junction	316	0.026	L	− 50	− 62	23	4.20
Increasing controls > HFA
Dorsal anterior cingulate cortex	595	0.001	L	− 9	33	15	5.40
Dorsal anterior cingulate cortex			R	5	27	18	4.03
Rectal gyrus/medial orbitofrontal gyrus	282	0.041	R	11	38	− 17	4.45
Insula	562	0.001	L	− 38	− 9	− 6	4.44
Decreasing HFA
Precuneus	551	0.001	L	− 6	− 80	38	3.95
Precuneus			R	3	− 72	38	3.48

Abbreviations: T = t-values of regions active in each contrast; L = left hemisphere; R = right hemisphere.

Eye tracking results

Results of the subsequent eye-tracking experiment showed that there was no significant effect of gaze duration on the amount of fixations to the eye region of the stimulus faces F(3,30) = 2.053, p = 0.128. Moreover, the main effect of group did not reach significance, F(1,10) = 0.208, p = 0.658, indicating that both groups attended to the eyes of the animated character to a similar extent. Finally, no significant interaction relationship was found, meaning that different gaze durations did not have any differential effect on the amount of fixations to this particular ROI for participants with ASD and control participants, F(3,30) = 0.947, p = 0.430. Similar results were found for all other ROIs.

Discussion

The present study focused on the influence of the two factors gaze direction and gaze duration on the neural processing of likeability of dynamic virtual human faces in HFA participants and a matched control group. Behavioral results revealed that increasing gaze duration increased likeability ratings linearly for the control but not for the HFA group. Neural results in the control group revealed two complementary cognitive processes related to the two different gaze parameters. On the one hand, the recruitment of regions of the SNN for direct gaze processing, including the pSTS, the insula, the PCun and the TPJ indicates salience detection. On the other hand, direct gaze duration processing revealed the involvement of regions of the mPFC (the dACC and the mOFC). These regions are typically associated with outcome monitoring, hence indicating higher-order social cognitive processes related to the evaluation of the ongoing communicational input conveyed by prolonged eye contact. In the HFA group solely the pSTS was engaged by direct compared to averted gaze, while several regions of the SNN, namely the PCun, the TPJ and the FG were activated by the opposite contrast. Moreover, in the HFA group, while processing increasing gaze duration did not elicit any differential activations, decreasing gaze duration was correlated with neural activity in the PCun. Thus, the present results also show that, participants with HFA may ascribe greater salience to averted rather than direct gaze.

Behavioral findings

In general, faces displaying direct gaze were perceived as significantly more likeable than those with averted gaze across both groups. This is in line with previous research findings, which have concluded that there is a general preference for facial cues to social interest over cues to disinterest (Clark and Mils, 1993; Jones et al., 2006). A main effect of gaze duration was found in the control group, indicating an overall positive effect of prolonged gaze on impression formation. Indeed, previous studies have robustly demonstrated that the longer a person looked into an observer's eyes, the more favorably this person was judged with regard to likeability, potency or self-esteem (Argyle et al., 1974; Bente et al., 2007a, 2007b; Brooks et al., 1986; Droney and Brooks, 1993; Knackstedt and Kleinke, 1991; Kuzmanovic et al., 2009). This is plausible, since in the context of social interaction, “prolonged gaze” is a cue of social interest and may convey signals of preference and/or approach (Argyle and Cook, 1976; Kampe et al., 2003; Mason et al., 2005). In the HFA group, the main effect of gaze duration did not reach significance (see Fig. 2). A characteristic observation in individuals with ASD is absent visual reciprocity and atypical gaze behavior (Buitelaar, 1995), which may suggest a general neglect of the eyes as a relevant social information source (Pelphrey et al., 2005a; Senju and Johnson, 2009a; Zürcher et al., 2013; for a review, see Senju and Johnson, 2009a). Interestingly however, our own subsequent eye-tracking experiment found no difference in the frequency or duration of fixations on various regions of the virtual faces, including the eyes, across conditions or groups (Fletcher-Watson et al., 2009; Rutherford and Towns, 2008). In other words, in the present paradigm, the eye region was well perceived but not integrated into the impression formation process of HFA subjects. This is in concordance with the finding that the degree to which nonverbal information contributes to complex subjective social decisions is significantly lower in HFA than in control participants (Kuzmanovic et al., 2011; Schwartz et al., 2010). A difference between groups failed to reach significance, however this may be due to the low sample size.

FMRI findings

Effects of gaze direction

The pSTS is recruited in direct gaze versus averted gaze in both groups

The finding of increased pSTS activation in both groups confirms our initial hypothesis and supports previous research that attests this region's involvement in processing direct gaze direction (Calder et al., 2002; Ethofer et al., 2011; Pelphrey et al., 2004; von dem Hagen et al., 2013; Wicker et al., 2003). However, the pSTS is also engaged during the processing of biological motion (Allison et al., 2000). The increased activation of the pSTS for direct compared to averted gaze, may be in part driven by additional biological motion in the direct gaze condition compared to averted gaze. Accordingly, it has been suggested that the pSTS is specifically involved in processing the social significance of motion cues and their contribution to social communication (Gao et al., 2012; Zilbovicius et al., 2006). In the context of gaze behavior, the pSTS might be involved in decoding intentions behind the eye movements, with respect to a communicative intention (Bristow et al., 2007; Hooker et al., 2003; Mosconi et al., 2005; Pelphrey et al., 2003, 2004). Taken together, in the current study we argue that the direct gaze condition was more suggestive of an intentional communicative intention compared to the averted one.

Several neuroimaging studies using dynamic facial stimuli failed to find pSTS modulation to gaze direction in autistic individuals (Pelphrey et al., 2003, 2005a; Pitskel et al., 2011; von dem Hagen et al., 2013). Behavioral studies have corroborated this finding by showing that autistic participants show no preferential response to eyes as a social cue (Ristic et al., 2005; Senju and Johnson, 2009a; Senju et al., 2003, 2005, 2008; Wallace et al., 2006). Thus, it has been suggested that there might be a difference in the way direct gaze is processed between autistic and control persons. The present results, however, show that direct compared to averted gaze does actually elicit a response in the pSTS in participants with HFA, but it tends to be weaker than in the control participants and restricted to the right hemisphere. However, these differences do not reach significance in the interaction effect (see Fig. 3C, Table 2). One speculation is that, although the gaze direction change is detected, direct gaze does not convey the same salience in participants with HFA. This hypothesis needs to be tested in future studies.

Strong activation for the direct gaze versus averted gaze was also observed in a region corresponding to the extrastriate area V5, which plays a central role in motion processing in general (MT +/V5) (Born and Bradley, 2005; Wilms et al., 2005). Indeed, eye motion has been found to elicit activation in this area (Puce et al., 1998; Watanabe et al., 2001, 2006). Considering the fact that in the present study the direct gaze conditions included more motion quantity due to the additional gaze shift, the enhanced MT/V5 activity is likely to indicate an automatic bottom-up analysis of eye motion as a salient moving physical stimulus. Interestingly however, the HFA group does not show activation of the MT/V5 complex, which is consistent with the finding of atypical motion perception in individuals with ASD (Freitag et al., 2008; Herrington et al., 2007).

Regions of the SNN are recruited by the perception of direct gaze versus averted gaze in the control group

Confirming the initial hypothesis, the neural correlates of the comparison between direct gaze and averted gaze in the control group are not solely restricted to the occipitotemporal areas. It additionally involves regions typically assigned to the SNN, namely the TPJ (localized in the supramarginal gyrus), the insula, and the PCun. Evidence from functional neuroimaging studies shows that the right TPJ is associated with mental state attribution (e.g. Saxe and Wexler, 2005). In the context of gaze processing, two studies have found the TPJ to be preferentially active for direct relative to averted gaze in typically developing individuals (Pitskel et al., 2011; von dem Hagen et al., 2013). Increased insula response has been previously found when subjects were exposed to eye motion (Pelphrey et al., 2005b), to direct gaze (Ethofer et al., 2011; Pitskel et al., 2011) or an increasing proportion thereof (Calder et al., 2002), as well as for inferences about the mental states of others on the basis of the eye region (Baron-Cohen et al., 1999). The PCun has also been engaged by gaze-based joint attention tasks (Bristow et al., 2007; Williams et al., 2005), reading Theory of Mind (ToM) stories (Fletcher et al., 1995; Young et al., 2010), and viewing ToM cartoons (Gallagher et al., 2000). Moreover, the PCun plays an important role in self-awareness and self versus non-self representation (Johnson et al., 2002; Legrand and Ruby, 2009; Lieberman and Pfeifer, 2005; Lou et al., 2004; Vogeley et al., 2001). Indeed, direct gaze displayed by another person signals social attention (Kampe et al., 2003; Kleinke, 1986; von Grünau and Anston, 1995) and is an indicator for self-relevance (Cristinzio et al., 2010; N'Diaye et al., 2009; Schilbach et al., 2006). Thus, self-referential processing might have increased in the direct gaze condition of the present study as a function of enhanced perceived interpersonal involvement. Together, these findings support the idea of direct gaze as an important social cue prompting mental state inference. Nevertheless, these regions are not active for the same contrast in individuals with HFA, supporting previous research that demonstrates differential neural processing of direct gaze in ASD (Grice et al., 2005; Pelphrey et al., 2005a; Pitskel et al., 2011; Senju et al., 2005; von dem Hagen et al., 2013).

Regions of the SNN are recruited by the perception of averted gaze versus direct gaze in HFA

In the HFA participants we found a set of regions to be preferentially activated by averted gaze versus direct gaze. Specifically, this group demonstrated greater recruitment of the PCun and PCC, the mOFC and left dlPFC, as well as bilaterally the TPJ (localized in the posterior terminal of the ascending STS branch) and the FG (extending to the parahippocampal gyrus). Interestingly, these are also regions, which are commonly associated with the SNN. This finding is in concordance with a recent study by von dem Hagen et al. (2013) who have shown that the SNN shows an atypical response in that it is not activated by direct compared to averted gaze, but by the reverse contrast. The authors suggest that in ASD averted gaze may be more salient or a preferred mode of social interaction and that this might explain why this type of gaze engaged the SNN network in a similar way to direct gaze in control participants.

The FG has been associated with the processing of faces and facial features (Kanwisher and Yovel, 2006). However, fMRI studies have previously found evidence of reduced or atypical activation in the FG in individuals with ASD when processing facial information (e.g. (Humphreys et al., 2008; Pierce et al., 2001; Schultz et al., 2000)). Given the fact that normal levels of FG activation in individuals with ASD can be elicited through experimental manipulations such as directing participants to fixate on the eye region (Hadjikhani et al., 2004, 2007) and considering that there is a correlation between FG activation and time spent fixating on the eye region (Dalton et al., 2005), the finding of increased FG activation could be explained by a longer time period that HFA participants look at the eyes in the averted gaze condition compared to the direct one. As our eye tracking data investigation did not reveal any difference in frequency of fixations to the eye region across gaze conditions, we don't consider differential visual attention reflecting the differences in FG activation as very likely. In contrast, it is possible that, averted gaze allowed HFA participants to integrate gaze processing with the facial context more easily to make a judgment on the perceived likeability of a virtual person. The additional finding of the engagement of the TPJ region corroborates this interpretation, considering that this particular brain region has been previously found to be maximally face sensitive (Kreifelts et al., 2009). Moreover, face-evoked activation in the mOFC has been previously found in fMRI studies, particularly during valence assessment of facial stimuli (Aharon et al., 2001; Kim et al., 2007; Kranz and Ishai, 2006; O'Doherty et al., 2003). Thus, it has been proposed that this region may encode information about valence and identity of faces (Kringelbach and Rolls, 2004). The mOFC is densely connected with the parahippocampal cortex (Carmichael and Price, 1995) and with posterior midline structures such as the PCC/PCun (Cavada et al., 2000), all of which are activated by this contrast. Previous studies point to a role of the parahippocampal regions in contextual (Rauchs et al., 2008) and autobiographical memory (Fink et al., 1996; Maguire et al., 2000). The medial parietal region (PCC/PCun) is engaged by tasks involving either a social or an outward-directed valuation component. Summarizing previous findings, Schiller et al. (2009) suggest that this region is involved in assigning value to social information guiding our first impressions of others. In sum, we suggest that the current pattern of activation in HFA participants is related to both cognitive control and specific social inferential processing. This reflects the fact that, for HFA participants, gaze information may be better integrated with contextual information to form a valence impression of a face in the averted compared to the direct gaze condition.

The current design has two limitations: i) the direct gaze conditions constituted 3/4 of all events, and ii) the direct gaze conditions included an additional gaze shift compared to the averted gaze condition. Both of these factors could have rendered the direct gaze stimuli more salient irrespective of the gaze per se. Thus, the activation of the SNN could be elicited by different factors in the two participant groups: by an effect of novelty for the averted gaze condition in HFA and by an effect of increased motion quantity in the control group.

Effects of gaze × group interaction

Our investigation of regions that demonstrated a group by gaze interaction identified several regions of the SNN, namely, the right PCun and TPJ (localized in the posterior terminal of the ascending STS branch), the left MTG, as well as the mOFC. Some regions, which we have previously discussed were sensitive to gaze direction in only one group; however there were also regions modulated by gaze direction in both groups.

In concordance with two recent studies (Pitskel et al., 2011; von dem Hagen et al., 2013) we have found a significant group by gaze direction interaction in the right TPJ, with control participants showing greater activity in this region to direct gaze versus averted gaze but the opposite pattern in participants with HFA. In particular, the right TPJ has been associated with mental state attribution (Lombardo et al., 2011; Saxe and Wexler, 2005; Vogeley et al., 2001). Moreover, in the present study the PCun was also active to direct gaze versus averted gaze in control participants, and recruited in response to averted compared to direct gaze in HFA participants. Indeed, this region has been previously engaged by gaze direction discrimination and joint attention tasks (Bristow et al., 2007; Carlin et al., 2011; Williams et al., 2005). Interestingly, both the TPJ and the PCun have also been involved in attentional reorienting (PCun, Cavanna and Trimble, 2006; TPJ Mitchell, 2008). Indeed, the TPJ region, as part of the ventral attention network (Corbetta et al., 2000) is particularly sensitive to stimuli that are considered task-relevant (Chang et al., 2013). Thus, the engagement of these regions may reflect covert attentional orienting responses to gaze (Carlin et al., 2011; Friesen and Kingstone, 2003). Differences in the gaze condition that suggests such a reorienting process might be caused by a “group-driven divergence in the type of gaze that holds the most social and attentional salience” (Pitskel et al., 2011, p 1691).

Neural correlates of gaze duration

Regions of the mPFC and the insula are engaged by processing increasing direct gaze duration by the control group

Confirming the initial hypothesis, we have found a positive correlation of signal increases with increasing gaze duration in a region of the mPFC, namely the dACC. This region has been involved in optimizing behavioral performance when confronted with continuously evolving environmental demands (Sheth et al., 2012). Therefore, it has been suggested that it also plays an important role in updating our social information from other people (Adolphs, 2009). In addition to the dACC, the direct group comparison also revealed an involvement of another region of the mPFC, namley the mOFC. This region may encode information about valence and identity of faces (Kringelbach and Rolls, 2004) and has been involved in monitoring the reward value of stimuli (Amodio and Frith, 2006; Kringelbach, 2005; Kringelbach and Rolls, 2004). Evidence for the reward potential of direct gaze manifests in early ontogeny as even very young infants preferentially attend to faces with direct compared to averted gaze (Farroni et al., 2002; Symons et al., 1998) and improve affect regulation and suckling behavior when experiencing direct gaze (Blass et al., 2007). Along the same line, eye contact has been found to serve as a reward in operant conditioning (Argyle and Cook, 1976). This result is consistent with our behavioral findings of increased likeability with increasing gaze duration. In addition, the involvement of the mOFC in direct gaze processing has been previously linked to enhanced emotional processing during direct gaze perception (Conty et al., 2007; Wicker et al., 2003). Finally, the mOFC has also been involved in contextual updating, i.e. as contexts change, the threshold at which prepotent tendencies are expressed is shifted (Hughes and Beer, 2012). Thus, the current activation pattern may reflect the updating of underlying strategies for likeability judgments. Therefore, the initial gaze direction detection may trigger an automatic response tendency, which needs to be updated with respect to the incoming information transmitted by varying durations of the eye contact: The longer the direct gaze duration, the more information with respect to a potential communicative exchange is conveyed.

In addition, the direct group comparison also demonstrates the involvement of the left insula for processing increasing direct gaze duration for the control versus the HFA group. A functional model on the insula has proposed that particularly its anterior portion could be associated with subjective experience and conscious awareness (Craig, 2009). Thus, it has been suggested that it is part of a “salience network” which integrates social and contextual information with internal states (e.g. arousal; Critchley et al., 2000) to provide a neural substrate of conscious experience that guides behavior (Craig, 2009; Seeley et al., 2007). In this line, a study by Ethofer et al. (2011) has found that particularly the anterior insula is selectively sensitive to the social significance of direct gaze (i.e. gaze shifts towards the observer). Both the ACC and the insula have been involved in indexing the sequential progression of the feeling of subjective awareness (for a review, see Craig, 2009), which leads us to suggest that the present insular activation might point to a subjective feeling of an enhanced emotional salience or arousal initiated by the perception of increasing direct gaze duration.

The PCun is engaged by processing decreasing direct gaze duration in the HFA group

Participants with HFA did not show any differential neural response to increasing gaze duration. This suggests that increasing direct gaze does not signal the same communicative intent to individuals with HFA as it does to the control participants. Interestingly, the same region engaged by averted compared to direct gaze, the PCun was also preferentially engaged by decreasing direct gaze perception in HFA participants. Considering that this region is involved in attentional orienting tasks (Cavanna and Trimble, 2006), activation in this region may reflect covert attentional orienting responses to a stimulus that is salient (Carlin et al., 2011; Friesen and Kingstone, 2003). In the case of HFA participants this seems to be the case for shorter rather than longer gaze durations.

Conclusion

The present study focused on the processing of gaze direction and gaze duration by making use of virtual characters as stimuli. While direct gaze and increasing direct gaze duration may signal social salience and a communicative intent to typically developing individuals, gaze duration did not lead to the same significant relationship in HFA. However, the present results also demonstrate, that in participants with HFA, gaze processing deficits are not based on gaze direction discrimination per se. Rather, they seem to result from ascribing salience to averted gaze rather than direct gaze and from being impaired in using subtle aspects of gaze, such as the duration of direct gaze, to understand others.