The neural correlates of social attention: automatic orienting to social and nonsocial cues.

Previous evidence suggests that directional social cues (e.g., eye gaze) cause automatic shifts in attention toward gaze direction. It has been proposed that automatic attentional orienting driven by social cues (social orienting) involves a different neural network from automatic orienting driven by nonsocial cues. However, previous neuroimaging studies on social orienting have only compared gaze cues to symbolic cues, which typically engage top-down mechanisms. Therefore, we directly compared the neural activity involved in social orienting to that involved in purely automatic nonsocial orienting. Twenty participants performed a spatial cueing task consisting of social (gaze) cues and automatic nonsocial (peripheral squares) cues presented at short and long stimulus (cue-to-target) onset asynchronies (SOA), while undergoing fMRI. Behaviorally, a facilitation effect was found for both cue types at the short SOA, while an inhibitory effect (inhibition of return: IOR) was found only for nonsocial cues at the long SOA. Imaging results demonstrated that social and nonsocial cues recruited a largely overlapping fronto-parietal network. In addition, social cueing evoked greater activity in occipito-temporal regions at both SOAs, while nonsocial cueing recruited greater subcortical activity, but only for the long SOA (when IOR was found). A control experiment, including central arrow cues, confirmed that the occipito-temporal activity was at least in part due to the social nature of the cue and not simply to the location of presentation (central vs. peripheral). These results suggest an evolutionary trajectory for automatic orienting, from predominantly subcortical mechanisms for nonsocial orienting to predominantly cortical mechanisms for social orienting.

Introduction

A salient stimulus in the environment will automatically attract one’s attention. Such an orienting reflex is quite adaptive, as the stimulus may signal a threat or a desired resource and require subsequent action. Likewise, viewing the direction of another’s eye gaze automatically directs attention toward the location of the gaze (Friesen & Kingstone, 1998; Friesen, Moore, & Kingstone, 2005). This orienting response is also evolutionarily adaptive, as the focus of another’s attention often indicates intentions to act upon the environment. Such intentions will likely influence our own actions, especially in the case of a threat or desired resource.

Experimentally, orienting attention in space has been studied extensively using a spatial cueing paradigm originally described by Posner (1980). Traditional models of spatial attention distinguish between automatic and controlled orienting, with distinct methodological specifications to measure each one (Jonides, 1981). When measuring automatic orienting, a peripheral cue (e.g., an asterisk appearing in a possible target location) typically precedes a target stimulus to which the participant must respond. Generally, peripheral cues are nonpredictive, indicating the location of the target 50% of the time. When measuring controlled orienting, a centrally presented, symbolic cue (e.g., an arrow pointing toward a possible target location) typically precedes the target. To further engage controlled processes, central cues are often predictive, indicating the location of the target more than 50% of the time. In spite of these differences, gaze cues that were centrally presented (though nonpredictive) demonstrated similar behavioral effects to traditional automatic (peripheral) cues (Driver et al., 1999; Friesen & Kingstone, 1998).

To examine the effects of social cues on orienting attention (henceforth social orienting), Friesen & Kingstone (1998) developed a variant of Posner’s spatial cueing paradigm using schematic eyes looking left or right as the cue preceding the target. Several investigators have since looked at the effects of gaze cues with spatial cueing paradigms, using both schematic faces (Friesen et al., 2005; Ristic, Friesen, & Kingstone, 2002) and pictures of real faces (Driver et al., 1999; Langton, 2000). These studies suggest that directed eye gaze influences attentional orienting in a seemingly automatic manner. Similar to classic automatic cues, participants respond faster to validly cued (gazed at) targets than invalidly cued targets, an effect termed facilitation, even when the gaze is not predictive of the target location. In addition, this effect has a relatively short time course, occurring when the time interval from the onset of the cue to the onset of the target (generally referred to as the stimulus onset asynchrony: SOA) is very short (105 ms). Further, facilitation holds even when the gaze cues are counter-predictive (Driver et al., 1999; Friesen, Ristic, & Kingstone, 2004). That is, when participants know that eyes looking left indicate that the target will most likely appear on the right, they are still faster to respond to a target in the gazed at location. Thus, there is something special about eye gaze that automatically directs attention, even when contradicting cognitive goals.

It has been proposed that social orienting depends on a distinct neural system from that supporting automatic orienting in response to nonsocial information (Kingstone, Friesen, & Gazzaniga, 2000; Ristic et al., 2002). One reason for this claim stems from a behavioral difference between social and peripheral cueing. When the SOA is longer than approximately 300 ms, participants are actually slower to respond to targets that are validly cued than invalidly cued when peripheral cues are used. This inhibitory effect, termed inhibition of return (IOR), is thought to reflect an evolutionary mechanism that aids in efficient visual search by preventing a return to previously searched locations (Klein, 2000). However, IOR is rarely found with gaze cues (Friesen & Kingstone, 1998; McKee, Christie, & Klein, 2007; but see Frischen & Tipper, 2004). Rather, as the SOA increases, facilitation decreases and eventually disappears without a reverse in the effect. Does this imply that social cues override the inhibitory mechanism present for peripheral cues? This is a tempting theory, as a stimulus worthy of another’s attention is most often worthy of our own. In addition, with the recent interest in identifying neural networks relevant to social behavior, it is important to investigate whether some attentional systems may be specialized for social information and subserving social interactions. From an evolutionary perspective, social interactions and the use of social cues have become increasingly sophisticated throughout evolution (Emery, 2000). Therefore, identifying neural networks that are relevant to orienting based on social cues may provide clues as to how the brain evolved to support this change.

To investigate the neural mechanisms underlying social orienting, Kingstone, Friesen, & Gazzaniga (2000) administered a lateralized version of the gaze cueing paradigm to two split-brain patients. Previous testing with these patients revealed that one had face processing lateralized to the right hemisphere, while the other had face processing lateralized to the left hemisphere. Interestingly, they found that social orienting effects only occurred in the hemisphere specialized for processing faces. This laterality effect was not found in previous work using nonsocial cues (Kingstone, Grabowecky, Mangun, Valsangkar-Smyth, & Gazzaniga, 1997). Thus, the authors argue that social orienting relies on asymmetric cortical mechanisms, whereas orienting to nonsocial cues relies on subcortical mechanisms that are shared between the hemispheres.

Studies investigating the specific cortical networks involved in social orienting are limited. Neuroimaging and single-unit recording studies examining gaze perception have found that the superior temporal sulcus (STS) plays a prominent role in processing gaze (Hoffman & Haxby, 2000; Hooker et al., 2003; Perrett et al., 1985). When Hoffman and Haxby compared blood-oxygenation-level-dependent (BOLD) activity when viewing gaze directed toward and away from the participant, they found stronger activation in the intraparietal sulcus (IPS) in the latter. The posterior parietal cortex (PPC), including the IPS, is part of a fronto-parietal network presumed to support spatial attention (Corbetta & Shulman, 2002; Mesulam, 1981, 1999). Parietal activity, along with STS activity, has also been found using fMRI in adults and children who viewed gaze shifts (Mosconi, Mack, McCarthy, & Pelphrey, 2005; Pelphrey, Singerman, Allison, & McCarthy, 2003). Thus, perceiving another’s averted gaze recruits part of the circuitry involved in attentional orienting.

To date, there are three neuroimaging studies that have compared brain activity involved in social orienting to nonsocial orienting. Kingstone, Tipper, Ristic and Ngan (2004) measured BOLD activity while participants responded to a target that was cued by an ambiguous stimulus, which could be interpreted as a social figure (eyes with a hat) or as a nonsocial figure (a car). They found the right STS to be selectively active for the social cue condition in contrast to the nonsocial cue condition. More recently, Tipper, Handy, Giesbrecht and Kingstone (2008) developed another ambiguous stimulus cue, which could be perceived as a profile of an eye or as an arrowhead. They found largely overlapping areas of BOLD activation for both percepts in fronto-parietal regions, with stronger activation for the social cue in select occipital and frontal regions. They also measured event-related potentials (ERPs) from the two possible percepts, finding evidence for a sensory gain effect when the cue was perceived as an eye profile. The authors argue that social and nonsocial orienting engage the same network, with stronger engagement of select regions for social orienting. In contrast to Kingstone et al. (2004), they did not find selective activation in the STS for the eye profile percept. Thus, it is possible that the ambiguous stimulus used by Tipper et al. did not resemble an actual eye well enough to engage social networks, and simply required more symbol interpretation. In the third fMRI study, Hietanen et al. (2006) compared schematic gaze and central arrow cues. They found that directional arrow cues selectively recruited fronto-parietal areas, while both arrow and gaze cues recruited occipito-temporal areas. In these overlapping regions, activation was stronger and the extent was larger for the arrow cues than the gaze cues, a finding that parallels comparisons of controlled and automatic orienting, respectively (Kim et al., 1999; Rosen et al., 1999). Thus, gaze cues may in fact rely on more automatic processes than arrow cues.

In the above imaging studies, gaze cues were compared to symbolic cues, which typically engage controlled orienting. Although it has been suggested that nonpredictive arrow cues can elicit automatic orienting (Tipples, 2002), they may not be purely automatic and likely involve controlled processes as well (Friesen et al., 2004). The imaging results of Hietanen et al. (2006) further suggest that gaze cues are more automatic than arrow cues. However, a direct comparison of the neural activity involved in gaze cueing and purely automatic nonsocial cueing has not been conducted, and that was the purpose of the present study.

The present experiment used fMRI to directly compare BOLD activity involved in social orienting to that involved in purely automatic nonsocial orienting. In order to maximally engage automatic processes, social and nonsocial cues were nonpredictive of the target location. Since the major behavioral difference between gaze cues and traditional automatic cues is the influence of the SOA on behavior (IOR found at long SOAs for nonsocial cues, but not for social cues), we implemented two SOAs to investigate the neural correlates of this effect. Subcortical regions, specifically the superior colliculus, have been shown to play a role in IOR (Dorris, Klein, Everling, & Munoz, 2002; Sapir, Soroker, Berger, & Henik, 1999). We hypothesized that the neuroimaging data will parallel the behavioral results, demonstrating subcortical activation in conditions that yield IOR. This hypothesis is consistent with the theory that social orienting relies on cortical mechanisms, while nonsocial orienting recruits subcortical mechanisms shared between the hemispheres.

Methods

Participants

Twenty right-handed normal volunteers (9 men and 11 women, ages 18–30) at the University of California, Los Angeles were paid to participate in the experiment. None reported having a psychiatric or neurological history, learning disability or attention deficit. Handedness was assessed with a modified version of the Edinburgh Handedness Inventory (Oldfield, 1971). Participants were fully informed of the nature of the study, potential risks, and their rights as subjects under UCLA Institutional Review Board (IRB) guidelines. The consent forms described these risks and discomforts clearly.

Behavioral task

The software program Presentation® (www.neurobs.com) was used to present stimuli and record reaction time (RT) data. Visual stimuli were presented through magnet-compatible goggles and responses were collected unimanually from a magnet-compatible button box (Current Designs, Inc.).

Figure 1 illustrates the experimental procedure. The fixation stimulus consisted of a black fixation cross (0.5°) on white background, surrounded by a face-like image (3.6° in diameter), and two boxes (2.3° horizontal, 2.4° vertical) to the left and right of the “face”, 6° from the fixation cross. Each trial began with the appearance of a warning cue, which consisted of pupils (each subtending 0.5° of visual angle) filling in the eyes of the “face” looking to the left or the right (gaze cue) or a solid black square (subtending 1°) presented within one of the two peripheral boxes (square cue). The cue was presented for 125 ms, after which the fixation display was presented for the remainder of the SOA, which was either 150 or 950 ms. Thus, the fixation display was presented for 25 ms following the cue in the short SOA condition and for 825 ms in the long SOA condition. After the SOA, a target ‘X’ appeared in one of the two peripheral boxes and remained on the screen until the participant responded. This was followed by the fixation stimulus for a duration calculated to maintain the length of each trial to 4,000 ms.

Fig. 1

Experimental procedure of the behavioral paradigm; example of a valid gaze cue and a valid square cue

The cues were valid or invalid in identifying the location of the upcoming target. In order to engage automatic processes by minimizing expectations, the cues were nonpredictive, such that they were valid in only 50% of the trials. Thus, the present experiment implemented a 2 (Cue type: gaze, square) × 2 (Validity: valid, invalid) × 2 (SOA: short, long) within-subjects design.

Participants performed a block of ten practice trials outside the scanner first in order to become acquainted with the task. It was emphasized in the instructions that the cues were not predictive of the target location, and that it was not to their advantage to use the cue to predict where the target will occur. It was also emphasized to remain fixated on the central cross at all times in order to measure attentional shifts independent of eye movement. Upon visual inspection by the experimenter, participants had no trouble maintaining fixation during the practice block. In the scanner, participants completed four functional runs of 72 trials each and were reminded of the instructions before each run. We implemented an event-related design, such that each run comprised equal numbers of each condition of the task. The conditions were intermixed in an order optimized to produce maximal signal discriminability and to ensure temporal jitter among the conditions using a genetic optimization algorithm (Wager & Nichols, 2003). Participants responded via a button box situated on their torso while in a supine position in the scanner. The participants’ task was to press the button corresponding to the location of the target (left or right) as quickly and accurately as possible. Responses were made unimanually, such that the middle finger of the left hand or the index finger of the right hand indicated a target on the left, and the index finger of the left hand or the middle finger of the right hand indicated a target on the right. Response hand alternated between blocks and the order was counterbalanced across participants. We adopted a choice response task, rather than the most typical simple detection task, for two reasons. First, choice response tasks have been shown to elicit the same effects as simple detection tasks in gaze cueing paradigms. Second, reaction times are longer in choice response tasks, which may amplify effects at the neural level.

Imaging

Images were acquired using a Siemens Allegra 3.0 T MRI scanner. Two sets of high-resolution anatomical images were acquired for registration purposes. We acquired an MP-RAGE structural volume (TR = 2,300, TE = 2.93, flip angle = 8°) with 160 sagittal slices, each 1 mm thick with 0.5 mm gap and 1.33 mm × 1.33 mm in-plane resolution. We also acquired a T2-weighted co-planar volume (TR = 5,000, TE = 33, flip angle = 90°) with 36 transverse slices covering the whole brain, each 3 mm thick with 1 mm gap, a 128 × 128 matrix and an in-plane resolution of 1.5 mm × 1.5 mm. Each functional run involved the acquisition of 150 EPI volumes (gradient-echo, TR = 2,000, TE = 25, flip angle = 90°), each with 36 transverse slices, 3 mm thick, 1 mm gap, and a 64 × 64 matrix yielding an in-plane resolution of 3 mm × 3 mm. A functional run lasted 5 min and 4 s.

Behavior analysis

Trials with reaction times faster than 150 ms and slower than 800 ms were considered attentional errors and removed from analysis. A repeated-measures ANOVA was conducted on reaction time data for correct trials with the factors Cue type (gaze, square), SOA (short, long), and Validity (valid, invalid). To investigate the specific effects of each cue, separate ANOVAs were conducted on each cue type with the factors SOA (short, long) and Validity (valid, invalid). Paired-samples t-tests were used to investigate the specific effects of facilitation and IOR.

Imaging analysis

Analysis was carried out using FEAT (FMRI Expert Analysis Tool), part of FSL (FMRIB’s Software Library, www.fmrib.ox.ac.uk/fsl). After motion correction, images were temporally high-pass filtered with a cutoff period of 50 s and smoothed using an 8 mm Gaussian FWHM algorithm in three dimensions. We modeled the BOLD response using a separate explanatory variable (EV) for each of the eight conditions of the task. The design was convolved with a gamma function to produce an expected BOLD response. The temporal derivative of this timecourse was also included in the model for each EV. Functional data were then fitted to the model using FSL’s implementation of the general linear model.

Each participant’s statistical data were then warped into a standard space based on the MNI-152 atlas. We used FLIRT to register the functional data to the atlas space in three stages (Jenkinson, Bannister, Brady, & Smith, 2002; Jenkinson & Smith, 2001). First, functional images were aligned with the high-resolution co-planar T2-weighted image using a 6 degrees of freedom rigid-body warping procedure. Next, the co-planar volume was registered to the T1-weighted MP-RAGE using a 6 degrees of freedom rigid-body warp. Finally, the MP-RAGE was registered to the standard MNI atlas with a 12 degrees of freedom affine transformation.

After analyzing the functional data for each subject, data were passed into a higher-level mixed-effects analysis. Higher-level analysis was carried out using FLAME (FMRIB’s Local Analysis of Mixed Effects) (Behrens, Woolrich, & Smith, 2003). Z (Gaussianised T/F) statistic images were thresholded using clusters determined by Z > 2.3 and a (corrected) cluster significant threshold of p = 0.05 (Worsley, Evans, Marrett, & Neelin, 1992).

Due to evidence suggesting involvement of the superior colliculus in IOR, we performed an ROI analysis on this region. A mask of the ROI was generated from the functional activity in the Square cue > Gaze cue contrast for the long SOA trials. We included all voxels within the superior colliculus with a Z value greater than 2.3 in the contrast of interest. We then calculated percent signal change from baseline within this superior colliculus ROI for each experimental condition.

Results

Behavioral results

Table 1a lists the mean reaction time for each condition and Fig. 2a displays the results. The 2 (Cue type) × 2 (Validity) × 2 (SOA) ANOVA revealed a significant main effect of SOA, such that reaction time was faster for the long SOA (M = 358.73, SD = 42.98) than the short SOA (M = 385.05, SD = 49.15), F(1,19) = 40.97, p < 0.001. There was also a significant interaction of Cue × Validity, F(1,19) = 6.62, p = 0.019, and of Validity × SOA, F(1,19) = 23.04, p < 0.001.

Table 1

Mean latency for each condition

Cue type	Short SOA		Long SOA
	Valid	Invalid	Valid	Invalid
(a)
Gaze	379.1(45.2)	393.4(52.9)	362.7(36.8)	363.5(55.1)
Square	379.0(51.9)	388.7(52.3)	362.2(43.2)	346.5(44.5)
(b)
Gaze	388.2(43.7)	410.1(50.4)	364.0(51.7)	359.0(55.1)
Square	394.8(46.5)	405.1(51.5)	371.7(46.8)	350.0(56.2)
Arrow	393.7(41.7)	413.6(50.8)	357.5(42.4)	352.1(48.8)

Values are listed as “mean (standard deviation)” in ms

(a) main experiment (n = 20); (b) control experiment (n = 10)

Fig. 2

Mean latency for (a) main experiment: Gaze and Square cues; and (b) control experiment: Gaze, Square, and Arrow cues

The 2 × 2 ANOVA conducted to specifically investigate the effects of gaze cues revealed a significant main effect of SOA, F(1,19) = 23.42, p < 0.001, and a significant SOA × Validity interaction, F(1,19) = 5.09, p = 0.036. The interaction revealed a significant facilitation effect at the short SOA, in which reaction times were faster for valid than invalid cueing, t(19) = 4.02, p = 0.001, and no effect at the long SOA (p > 0.05). The 2 × 2 ANOVA conducted to examine the effects of square cues revealed a significant main effect of SOA, F(1,19) = 47.83, p < 0.001, and a significant SOA × Validity interaction, F(1,19) = 18.26, p < 0.001. At the short SOA, there was a trend toward a significant facilitation, in which reaction times were faster for valid than invalid cueing, t(19) = 2.0, p = 0.06. At the long SOA, significant IOR was found, such that reaction times were slower for valid than invalid cueing, t(19) = 3.19, p = 0.005.

Imaging results

To assess the brain regions involved in each cue type separately, we contrasted trials with each cue type to baseline. Table 2 lists the peak activation coordinates. Gaze cues yielded activation bilaterally in occipito-temporal and fronto-parietal regions. The fronto-parietal peaks were located where the precentral gyrus meets the superior frontal gyrus, likely corresponding to the human frontal eye field or FEF (Paus, 1996), in the medial wall of the frontal lobe, likely corresponding to the supplementary eye field or SEF (Grosbras, Lobel, Van de Moortele, LeBihan, & Berthoz, 1999), and in the PPC, including the intraparietal sulcus, likely corresponding to the parietal eye field or PEF (Culham, Cavina-Pratesi, & Singhal, 2006). For the square cue, we found activation largely in similar areas. Figure 3 displays the statistical maps for each cue type. We subsequently masked the two activation maps with one another to identify regions that survived threshold for both cue conditions. This confirmed overlapping activity in the FEF, SEF, PEF, and occipito-temporal regions.

Table 2

Coordinates (MNI) and peak activation statistics for each cue condition versus resting condition

Anatomical region	Side	Gaze cue				Square cue
		x	y	z	Max Z-score	x	y	z	Max Z-score
PMd	L	–40	–8	66	5.18	–40	–8	66	5.02
	R	44	–12	66	5.67	44	–12	66	5.57
PMv	L	–54	2	32	4.70	–54	2	32	4.35
	R	–	–	–	–	54	4	8	4.08
SMA	L	–6	–6	50	4.75	–6	–6	50	4.69
	R	8	–4	52	4.21	6	–4	58	4.57
SPL	L	–44	–46	58	4.82	–44	–46	58	4.74
	R	42	–56	60	4.38	42	–44	62	4.35
SMGa	L	–50	–32	44	4.53	–50	–32	44	4.72
	R	48	–28	44	4.57	50	–30	46	4.47
MTG	L	–54	–54	6	5.35	–58	–60	10	4.58
	R	56	–52	–4	4.88	56	–52	–4	4.65
LOCi	L	–52	–72	2	5.48	–52	–72	6	5.18
	R	50	–66	–6	5.87	52	–66	–6	5.44
TOF	L	–38	–64	–20	5.92	–36	–64	–20	5.36
	R	42	–60	–20	6.37	44	–60	–20	4.99
LingG	L	–2	–74	–8	3.96	–2	–72	6	4.04
	R	2	–74	–8	4.23	2	–70	–4	4.19
Cerebellum	L	–32	–62	–28	5.76	–30	–60	–28	6.34
	R	46	–56	–28	6.28	26	–60	–22	6.13

LinG, lingual gyrus; LOCi, lateral occipital cortex, inferior division; MTG, middle temporal gyrus; MNI, Montreal Neurological Institute; PMd, dorsal premotor cortex; PMv, ventral premotor cortex; SMA, supplementary motor area; SMGa, supramarginal gyrus, anterior division; SPL, superior parietal lobule; TOF, temporal occipital fusiform gyrus

Fig. 3

Z statistic activation maps (corrected) of Gaze cue > Rest and Square cue > Rest. Anatomical left is image left. Transverse slices from z = 60 to z = 10 in anatomical atlas space. Color indicates Z statistic. FEF, frontal eye field; LOC, lateral occipital cortex; PEF, parietal eye field; SEF, supplementary eye field; SMG, supramarginal gyrus

We then contrasted the cue types directly. The Gaze cue > Square cue contrast revealed activation in extrastriate areas, including the lateral occipital gyrus, fusiform gyrus, and inferior temporal cortex. Figure 4 displays the statistical activation map and Table 3 lists peak activation coordinates. The reverse contrast, Square cue > Gaze cue, did not yield significant activation (p > 0.05).

Fig. 4

Z statistic activation map (corrected) of Gaze cue > Square cue. Anatomical left is image left. Coronal slice at y = –84 in anatomical atlas space; Transverse slice at z = –2. Color indicates Z statistic

Table 3

Coordinates (MNI) and peak activation statistics for Gaze cue > Square cue contrast

Anatomical region	Side	Gaze > Square
		x	y	z	Max Z-score
LOCi	L	–40	–80	–12	4.13
	R	36	–86	–10	5.58
OccipP	L	–28	–94	–2	4.92
	R	32	–92	–2	5.32
FusG	L	–42	–74	–20	3.22
	R	40	–62	–20	4.70
TOF	L	–42	–48	–24	3.64
	R	38	–44	–24	3.44

FusG, fusiform gyrus; LOCi, lateral occipital cortex, inferior division; OccipP, occipital pole; TOF, temporal occipital fusiform gyrus

To examine differential activity depending on SOA, the same contrasts were carried out separately at the short and long SOAs. For the short SOA, the results yielded similar activation patterns to the overall effects of cue (i.e., occipito-temporal activation for Gaze > Square, and no significant activation for Square > Gaze). For the long SOA, the pattern was different. The Gaze > Square contrast again showed occipito-temporal activation. However, the Square > Gaze contrast yielded activation in subcortical regions, including the cerebellum and brain stem. This is shown in Figure 5 and peak coordinates are listed in Table 4. These results are consistent with the behavioral difference between cue types, as subcortical areas were more active in the experimental condition that produced IOR.

Fig. 5

Z statistic activation map (corrected) of Square cue > Gaze cue for long SOA trials. Anatomical left is image left. Sagittal slice at x = 2 in anatomical atlas space, coronal slice at y = –24; Transverse slice at z = –24. Color indicates Z statistic

Table 4

Coordinates (MNI) and peak activation statistics for Square cue > Gaze cue contrast at the long SOA

Anatomical region	Side	Long SOA: Gaze > Square
		x	y	z	Max Z-score
QuP	R	24	–46	–38	3.60
QuA	R	12	–50	–24	3.10
Pve	L	–4	–54	–52	3.22
Ave	R	2	–80	–16	3.16
Medulla	R	2	–38	–42	3.37

Cerebellar regions: QuP, posterior quadrangular lobule; QuA, anterior quadrangular lobule; Pve, posterior vermis; Ave, anterior vermis

The ROI analysis of the superior colliculus (SC) revealed a 0.15% signal increase in the square cue condition and a 0.08% signal decrease in the gaze cue condition. A t test demonstrated that there was a trend toward a significant difference in percent signal change, t(19) = 1.93, p = 0.069. Thus, there was increased engagement of the SC in the behavioral conditions that produced IOR, consistent with previous research that demonstrated SC involvement in IOR (Dorris et al., 2002; Sapir et al., 1999).

Control experiment

It is possible that differences in brain activity for the social and nonsocial cues we used were due to differences in the physical location of the cue. We presented gaze cues in the center of the display, while square cues were presented in the periphery. In order to further examine this alternative explanation, we performed a control experiment in a smaller sample (ten participants) while they performed a version of the task that included central arrow cues. The paradigm was identical to that described in “Methods”, with the addition of central arrow cues that appeared at the location of the mouth within the fixation display (see Fig. 6). Like the gaze and square cues, the arrow cues were presented for 125 ms and were not predictive of the target location. Four functional runs were collected, with 78 trials each.

Fig. 6

Arrow cue condition

Behavioral results are presented in Table 1b and Fig. 2b. The SOA × Validity interaction was significant for gaze cue trials, F(1,9) = 6.6, p = 0.03, and for square cue trials, F(1,9) = 27.2, p = 0.001, and there was a trend toward significance for arrow cue trials F(1,9) = 4.5, p = 0.06. Examination of the interaction for gaze cue trials showed significant facilitation at the short SOA (p = 0.01) and no effect at the long SOA (p > 0.05). For square cue trials, there was a trend toward significant facilitation at the short SOA (p = 0.075) and significant IOR at the long SOA (p = 0.007). For arrow cue trials, there was significant facilitation at the short SOA (p = 0.02) and no effect at the long SOA (p > 0.05).

The imaging results of the control experiment confirmed greater extrastriate activation for gaze cueing than square cueing and for gaze cueing than arrow cueing. We first replicated the higher activity for Gaze cue > Square cue in occipito-temporal cortical areas (Fig. 7a) at fully corrected threshold. The same areas showed higher activity for Gaze cue > Arrow cue (Fig. 7b) although only at uncorrected threshold. However, considering that these areas were our regions of interest and that the control experiment was performed on a smaller group of participants, a correction for multiple comparisons seems unnecessarily conservative to us. We also conducted ROI analyses on these extrastriate regions to examine percent signal change for each cue type. ROIs were defined by the activation maps derived from the Gaze cue > Square cue contrast. Figure 7c shows that in the left extrastriate region, percent signal change for the gaze cue (M = 0.16, SD = 0.11) was significantly greater than that for the arrow cue (M = 0.11, SD = 0.10), p = 0.026, which was significantly greater than that for the square cue (M = 0.07, SD = 0.09), p = 0.047. Similarly in the right extrastriate, percent signal change for the gaze cue (M = 0.14, SD = 0.12) was nearly significantly greater than that for the arrow cue (M = 0.09, SD = 0.12), p = 0.08, which was significantly greater than that for the square cue (M = .04, SD = 0.11), p = 0.02. Thus, bilateral extrastriate regions were engaged the least during square cueing, more so during arrow cueing, and the most during gaze cueing. Therefore, our results were not simply driven by the visual location of the cue, but revealed differences between social and nonsocial cues.

Fig. 7

a Z statistic activation map (corrected) for Gaze cue > Square cue in the control study (n = 10), used for ROI analysis; b Z statistic activation map (uncorrected) for Gaze cue > Arrow cue in the control study; c Percent signal change from baseline in each ROI for social (gaze), central nonsocial (arrow), and peripheral nonsocial (square) cues

Discussion

The goal of the present study was to compare the neural circuitry involved in social orienting to that involved in automatic nonsocial orienting. We used event-related fMRI to measure BOLD activity while participants performed a spatial cueing task consisting of social (eye gaze) and automatic nonsocial (peripheral) cues. Our results revealed that the neural networks involved in automatic orienting driven by social and nonsocial cues are largely similar, involving oculomotor and attentional regions. On the other hand, social cueing recruited extrastriate regions involved in social perception, while nonsocial cueing recruited subcortical regions during conditions that produced IOR.

The task implemented in the present study allowed us to examine how automatic orienting influences subsequent actions. Participants had the specific goal of responding to a target stimulus. Thus, they maintained an intention to act solely upon the location of the target. However, their actions were influenced by automatic orienting processes driven by the cues (both social and nonsocial), resulting in variations in response speed. This reflects the close relationship between automatic orienting and intentional action. On a daily basis, we automatically orient toward important stimuli, but what is the purpose of this process? We are not simply passive observers of the world. Rather, we are engaged, active participants, and as such, automatic processes are useful to guide intentional actions.

Our finding that social cueing and automatic nonsocial cueing recruited largely overlapping cortical networks of activity is consistent with the results of Tipper et al. (2008). Although Tipper et al. used an ambiguous stimulus and we used two visually different stimuli, the social and nonsocial conditions in both studies resulted in overlapping activation in frontal premotor areas, the PPC, and occipito-temporal regions. By using an ambiguous stimulus, Tipper et al. were able to control for simple visual differences that may drive differences in neural activity. They reported more clusters of activation in lateral frontal areas than we did here. Lateral frontal activity has been attributed to the need for symbol interpretation in some attention tasks (Woldorff et al., 2004). Thus, it is possible that the recruitment of these regions was due to the symbol interpretation required to perceive their ambiguous stimulus as an eye profile. Such a cognitive requirement brings top-down processes into the task, possibly contaminating the measurement of automatic, bottom-up processes. It should be noted that their behavioral results were equivalent for both cue percepts. Yet, it is possible that the BOLD signal may have been more sensitive to additional top-down processing than reaction time performance. In our main experiment, symbol interpretation was not required, as the stimuli were automatically perceived as social or nonsocial. Thus, our nonsocial cue is purely automatic. Of course, we compared two visually different stimuli, but even so, social and nonsocial cueing demonstrated largely overlapping activity. Our results together with that of Tipper et al. suggest that social and automatic nonsocial orienting engage similar cortical mechanisms.

Overlapping activity for social and nonsocial cues further supports the idea that we automatically orient to where someone else is looking. Sudden changes in the periphery cause automatic attention shifts, allowing one to be vigilant to the environment and mobilize appropriate responses to the stimulus at hand. When another individual gazes toward a particular location, he/she may be mobilizing an appropriate response toward his/her focus of attention. It is advantageous for one to attend to the same location as others and possibly mobilize the same response. Thus, it makes sense that similar neural mechanisms were selected for responding to eye gaze cues and peripheral cues.

Specifically, overlapping activity was found in a fronto-parietal network including the FEF, SEF, and PEF. Several neuroimaging studies have found activation in this fronto-parietal network for both covert shifts in attention (without eye movement) and overt eye movements (Beauchamp, Petit, Ellmore, Ingeholm, & Haxby, 2001; Corbetta et al., 1998; de Haan, Morgan, & Rorden, 2008; Nobre, Gitelman, Dias, & Mesulam, 2000), supporting the premotor theory of attention (Rizzolatti, Riggio, Dascola, & Umilta, 1987). This theory proposes that covert orienting and saccadic eye movements are mediated by the same neural network, and hence, covert orienting reflects planning an eye movement without proceeding with its execution. In the present study, covert orienting in response to social and nonsocial cues engaged these oculomotor regions. Thus, our results are consistent with the premotor theory of attention and suggest that it extends to social cues. As we were unable to track eye movements, it is possible that the oculomotor system was engaged due to uncontrolled eye movements. However, several studies of covert orienting have shown that participants have little trouble maintaining fixation when instructed to do so (Corbetta et al., 1998; Corbetta, Miezin, Shulman, & Petersen, 1993). Given that such instructions were emphasized during the practice block and before each experimental run, it is unlikely that our participants moved their eyes freely.

When comparing the cue conditions, we found that the social cues yielded greater activity in extrastriate regions than the nonsocial cues. It has been proposed that these areas are specialized for processing social information and make up a network for face perception (Haxby, Hoffman, & Gobbini, 2000). Specifically, we found greater activation for the social cue condition in the fusiform gyrus, which includes the “fusiform face area” (FFA) (Kanwisher & Yovel, 2006), and in the inferior lateral occipital cortex, which has been termed the “occipital face area” (OFA) (Gauthier et al., 2000). Both the FFA and OFA have demonstrated selectivity in responding to faces. In our social cue condition, the pupils filled in empty eyes, enhancing the stimulus’ appearance as a face. Thus, it was not surprising to find greater activity for social cueing in these regions.

As previously mentioned, it is possible that the increased extrastriate activity found for social cues was a result of the physical location of the cue (central vs. peripheral), not its social nature. In order to examine this alternative explanation, we ran another ten participants on the same task with the inclusion of central arrow cues. Again, we found greater activation in the same extrastriate regions when comparing social and peripheral cues. Examination of the BOLD signal within these regions revealed a greater increase in activity for the social cue than for the central nonsocial cue (arrow), as well as a greater increase for the central nonsocial cue than for the peripheral nonsocial cue. Thus, increased extrastriate activation was not simply driven by the central location of the cue. We used an arrow with an arrowhead on both ends in an attempt to minimize the perception of the central nonsocial cue as social. Yet, it is possible that the arrow cue, appearing in the location of a mouth, resembled a social gesture such as a smirk in one direction or the other. Thus, the cue may have been perceived as more “social” than the square cue, but less obviously “social” than the gaze cue, fitting with the intermediate engagement of extrastriate regions. Interestingly, the behavioral results of our control experiment did not differentiate between gaze and arrow cues, suggesting again that in some cases BOLD changes may have higher sensitivity than response times in differentiating mechanisms for attentional orienting.

It is worth noting that we did not find greater STS activity for the gaze cue condition compared to the nonsocial cue condition. In fact, we did not find greater STS activity for either of the cue conditions compared to rest. This may be due to the constant view of the empty face image during the resting baseline condition. It is unlikely that participants did not view the gaze cue as eyes, since very simple stimuli, like those used in our task, are easily perceived as eyes, even in nonprimate animals (for a review, see Emery, 2000). In fact, it is quite difficult not to perceive the eyes. Thus, the continuous display of empty eyes may have subtracted out STS activity involved in gaze perception.

Since the time interval between the cue and the target (SOA) influences performance on a behavioral level, we included a short (150 ms) and a long (950 ms) SOA in order to examine the neural correlates of such behavioral differences. At the short SOA, both social and nonsocial cues elicited facilitation effects (faster reaction times to validly cued targets than invalidly cued targets). Yet, at the long SOA, behavior diverged and nonsocial cues yielded the reverse effect, IOR, while social cues showed no effect, consistent with previous findings (Friesen & Kingstone, 1998). Interestingly, our imaging data showed that brain activity diverged along with behavior. At the short SOA, nonsocial cueing did not elicit greater neural activity than social cueing. At the long SOA, however, there was increased activity in subcortical regions for the nonsocial cue. Specifically, several cerebellar regions were active, suggesting a role for the cerebellum in IOR. It is unlikely that this activation was driven by the visual differences between cue types because if that were the case, increased activity would be found regardless of SOA.

The cerebellum has a well know role in oculomotor control (Robinson & Fuchs, 2001). The posterior vermis has been called the “oculomotor vermis,” since Purkinje cell activity in this region increases during saccades, and stimulation of this region can induce saccades (Noda & Fujikado, 1987) or smooth-pursuit eye movements (Krauzlis & Miles, 1998). Evidence suggests a necessary role for the oculomotor vermis and its deep cerebellar nuclei (i.e., caudal fastigial nuclei) in both accurate saccades and smooth-pursuit eye movement (Robinson & Fuchs, 2001). A question that has arisen in the literature is the role of the cerebellum in attention. Patients with cerebellar damage have demonstrated impaired performance on neuropsychological tests of attention, and several neuroimaging studies implementing attention paradigms have found activation in the cerebellum (for a review, see Haarmeier & Thier, 2007). However, upon closer examination of the specific tests used in these studies, Haarmeier and Thier concluded that the deficits observed in patients and the activations observed with neuroimaging were a result of oculomotor demands. Thus, the cerebellar activation we observed during peripheral cueing at the long SOA is consistent with an oculomotor theory of IOR, which proposes that IOR results from oculomotor planning (Rafal, Calabresi, Brennan, & Sciolto, 1989). This inhibitory mechanism is important because of the tight link between automatic activation of motor plans and intention to act upon the world. In order to facilitate efficient and adaptive actions within the environment, it is just as necessary to inhibit oculomotor responses to repeated stimuli as it is to activate oculomotor responses to novel stimuli. Our results indicate that the cerebellum may mediate such inhibition, biasing oculomotor control away from repeated stimuli. Other types of cues, such as eye gaze, can override this mechanism on both a neural and behavioral level.

The superior colliculus (SC) is another subcortical region that has been implicated in IOR. Evidence has shown that a patient with damage to the right SC demonstrated IOR in the hemifields projecting to the left (intact) SC, but not in the hemifields projecting to the damaged SC (Sapir et al., 1999). Single-unit recording data from SC neurons in the rhesus monkey demonstrated suppressed SC activity when a target occurred at a cued location (Dorris et al., 2002). However, neurons were more excitable during the time window between the cue and the target. Thus, the authors suggest that higher cortical areas may be responsible for inhibiting SC activity, and this inhibition in turn leads to IOR. The PPC has been proposed as a likely candidate of this inhibition due to its role in attentional orienting and its neuroanatomical connections with the SC (Klein, 2000). Neuroimaging research has supported the involvement of the PPC and other oculomotor areas, including the SEF and FEF, in IOR (Mayer, Dorflinger, Rao, & Seidenberg, 2004; Muller & Kleinschmidt, 2007). In the present study, we did not find increased activity in the fronto-parietal network specifically for the conditions in which IOR was found (peripheral cue, long SOA). Rather, this network was engaged during facilitation and IOR, and for both social and nonsocial cueing.

Due to the evidence supporting a role for the SC in IOR, we expected to find SC activity during the conditions that yield IOR. However, activation did not survive cluster thresholding likely because of the tiny size of the SC and because our imaging protocol was not optimized to measure collicular activity. Therefore, we performed an ROI analysis of the region, which revealed a trend toward increased engagement of the SC for peripheral cue, long SOA trials. Thus, both the superior colliculus and the cerebellum demonstrated greater activity during conditions that generate IOR, suggesting that IOR indeed involves subcortical mechanisms. Therefore, it is possible that IOR is not found with social cues because social cues engage cortical structures that override the subcortical mechanisms. The activity in occipito-temporal cortical areas differentiating between social and nonsocial cues may—at least in part—reflect a modulatory role of these areas over subcortical structures.

The results of this study carry interesting evolutionary implications. The importance of eye gaze in social interactions has increased throughout evolution. We suggest that the social orienting system may have developed at a later evolutionary stage than nonsocial automatic orienting, consistent with the evolutionary shift from subcortical to cortical structures. Social orienting may have stemmed from the evolutionarily older system for automatic orienting, shifting away from subcortical mechanisms but utilizing some of the same cortical mechanisms, as indexed by overlapping cortical networks of activity. Additionally, as the relevance of social information became more prominent throughout evolution, cortical mechanisms may have evolved to override the subcortical ones that are involved in inhibitory behavior. Given the rich information conveyed by gaze direction, it may not be adaptive to inhibit an oculomotor response toward repeatedly gazed at locations. If a stimulus is worth looking at by another individual, it is important enough to override previous intentions and influence future actions.