Feedback of visual object information to foveal retinotopic cortex.

The mammalian visual system contains an extensive web of feedback connections projecting from higher cortical areas to lower areas, including primary visual cortex. Although multiple theories have been proposed, the role of these connections in perceptual processing is not understood. We found that the pattern of functional magnetic resonance imaging response in human foveal retinotopic cortex contained information about objects presented in the periphery, far away from the fovea, which has not been predicted by prior theories of feedback. This information was position invariant, correlated with perceptual discrimination accuracy and was found only in foveal, but not peripheral, retinotopic cortex. Our data cannot be explained by differential eye movements, activation from the fixation cross, or spillover activation from peripheral retinotopic cortex or from lateral occipital complex. Instead, our findings indicate that position-invariant object information from higher cortical areas is fed back to foveal retinotopic cortex, enhancing task performance.

Virtually all theories of visual feedback share the central idea that feedback exerts its effect on perception by modulating or anticipating representations generated by feed-forward responses to stimuli. In predictive coding models 1, feedback from high-level visual areas serves to “explain away” and hence reduce activity in lower areas 2, 3, simplifying the description of the stimulus. In figure-ground segregation, feedback putatively increases V1 responses to image regions corresponding to figure relative to those corresponding to ground 4–7. Attentional models posit that feedback to primary visual cortex modulates, tunes or anticipates8 the feedforward response to visual stimuli. Other theories argue that feedback sharpens perceptual representations 9 or enhances consciously available representations10. In contrast to all of these theories in which feedback modifies or anticipates a feedforward representation, we now demonstrate a case in which feedback apparently constructs a totally new representation in a different cortical region from the feedforward representation. Because the feedback and feedforward representations arise in nonoverlapping parts of retinotopic cortex, the two representations can be clearly distinguished, and studied separately.

The novel form of feedback reported here was discovered unexpectedly in a paradigm designed to investigate position-invariant object representations in higher-level cortex. Subjects fixated centrally while viewing three categories of novel objects (Figure 1A) in the periphery11. In each trial, two objects were presented simultaneously in diagonally opposite peripheral retinal locations (Figure 1B or 1C). These two objects were always from the same object category, and subjects were asked whether the two objects were identical (Figure 1B) or subtly different (Figure 1C) exemplars of that category. We then used multivariate pattern analysis methods 12, 13 to ask whether information about object category was present in the pattern of response across voxels in each of several cortical regions of interest (ROIs).

Figure 1

A) A single exemplar from each of the three stimulus categories; for information on the slight variations across exemplars in each category see 11. B) Example of the presentation display with identical ‘smoothies’ present in the top left and bottom right quadrants (left diagonal). C) Example of the presentation display with different ‘smoothies’ present in the top right and bottom left quadrants (right diagonal). Objects had a mean width of 1. 8° visual angle and were presented 7° from fixation.

Our experiments show that the pattern of fMRI response in foveal retinotopic cortex contains information about the category of objects presented more than five degrees outside the fovea, in the visual periphery. This object information has two properties unexpected for retinotopic cortex: i) it is in a part of the retinotopic map (the fovea) that does not correspond to the stimulus location (the periphery), and ii) it is position-invariant, that is, the pattern in foveal cortex is similar across different peripheral stimulus locations. Further, we show that the object information in foveal retinotopic cortex is behaviorally relevant: i) it is present during a discrimination task on objects presented in the periphery, but not during a color discrimination task performed on the same stimuli, and ii) stronger object information in foveal cortex is correlated with higher performance on the object discrimination tasks. These findings sugget a novel phenomenon not predicted by prior theories of feedback, in which position-invariant object information is fed back from high-level object areas to foveal retinotopic cortex, enhancing task performance.

Results

We applied correlation analyses to investigate the information contained in the spatial patterns of fMRI response as described previously 12–14. Specifically, the data were split in half (even versus odd runs), separately for each subject, and the spatial pattern of the fMRI response to each object category was extracted from each half of the data independently15. The presence of category information in a given ROI is indicated by a greater similarity or correlation between two activation patterns when they are generated by the same stimulus category than when they are generated by two different categories.

Specifically, in each ROI, we computed the correlation between independent pairs of activation patterns from the same object category (e.g., “smoothies” correlated with “smoothies”, see Figure 1A) versus from different object categories (e.g., “smoothies” correlated with “spikies”). This allowed us to look at which brain regions contain information that can discriminate one category from another (e.g., smoothies versus spikies). A higher correlation for same object category than different object category indicates the presence of information in that ROI that can discriminate between those object categories. To test whether the category information is tolerant to changes in object location, we compared correlations across pairs of activation patterns in which the objects were presented at the same retinotopic locations (e.g., both data sets from the left diagonal locations as shown in Figure 1B) with data in which objects were presented in different locations (e.g., correlation of the activation pattern from left diagonal locations as in Fig 1B with the activation pattern from right diagonal locations as in Figure 1C). If category information is specific to object location, correlations should be significantly higher for same versus different category only when the two activation patterns come from the same locations, not when the patterns come from different locations.

These analyses were computed separately on each of several ROIs, separately for each subject. One ROI was in the lateral occipital complex (LOC ROI), a region with a well-established role in shape representation 16, and the other three were in retinotopic cortex: the “Object Location ROI” was the part of retinotopic cortex that responded to the stimuli, the “Between Objects ROI” was a retinotopic region representing the space between the peripheral objects along the vertical or horizontal meridian, and the “Foveal ROI” was the foveal region of retinotopic cortex (see Figure 2; Supplemtary Figure 1).

Figure 2

ROIs in one example subject. A) Native functional slices 4 to 15 (see Methods), with this subject’s functionally-defined ROIs in color: lateral occipital complex (LOC; Objects>Scrambled from localiser experiment) in yellow, foveal retinotopic cortex (All>Rest from the Localiser runs, including active voxels only at the occipital poles) in red, and peripheral retinotopic ROIs (smoothies, spikies, cubies > rest from half the experimental runs) in blue; in each analysis, the union of two peripheral (blue) regions constitute the Same Location ROI (the cortical region corresponding to the location where the stimuli occur, e.g. in the upper right and lower left visual field) and the union of the two other peripheral (blue) regions constitute the Different Location ROI (the cortical region corresponding to the location where stimuli occur in the other stimulus location condition, e.g. upper left and lower right). B) Inflated cortical surface from the same subject showing the location of these retinotopic ROIs on the cortical surface (inflation was performed using Freesurfer 34).

In Experiment 1, we scanned six subjects during the object discrimination task. In the analysis, we examined the object category information (Same vs Different Category) present in each ROI under conditions where the objects were in the same versus different locations (Same vs Different Location). An omnibus ANOVA across subjects on the correlations found a significant three-way interaction of ROI * Same/Different Category * Same/Different Location (F(1,5) = 5.52, p=.009), indicating that the pattern of correlations differed significantly across ROIs. We therefore analyzed each ROI separately. First, within the LOC, same-category correlations were higher than different-category correlations (F(1,5) = 13.32, p=.015), indicating the presence of object category information in this region, consistent with previous results 11, 15. Further, we found no significant interaction of Same/Different Location * Same/Different Category; F(1,5) = 2.53, p=.173; see Figure 3A;), indicating that the information about object category in LOC is largely position-invariant, also consistent with prior results 17.

Figure 3

Mean correlations (± 1 SE) for same (black bars) versus different (grey bars) categories and for same versus different locations for the four ROIs in Experiment 1. A) Lateral Occipital Complex (LOC; objects>scrambled objects); B) Object Location ROI is the region of the peripheral retinotopic cortex corresponding to the location where the stimuli occur; C) Between Locations ROI is a peripheral retinotopic region corresponding to the gap between stimulus locations; D) The Foveal Region in retinotopic cortex (Foveal; central object presentation > fixation).

In the Object Location ROI, which corresponded to the parts of retinotopic cortex that are activated directly by the stimuli, object categories could be distinguished based on their activation patterns. However, as we would expect given the nature of retinotopic cortex, this was only true for the Same Location condition (i.e., when the objects were presented on the same diagonal in both data sets, t(5) = 3.61, p=.015), not when they were presented in Different Locations (t(5) = 1.27, p=.261); this difference between same versus different locations was significant (interaction of Same/Different Location * Same/Different Category (F(1,5) = 10.34, p=.024; see Figure 3B). The Between-Objects ROI showed no significant object information for either the Same Location or Different Location analyses. There was no main effect of Same/Different Category (F(1,5)< 1, n.s.), and no interaction of Same/Different Category by Same/Different Location (F(1,5) = 1.17, p=.330; Figure 3C). These results are consistent with expected location-specificity of retinotopic cortex.

Our key result, however, was the astonishing finding that the foveal retinotopic cortex ROI contained object category information (See Figure 3D). Correlations in the foveal region of retinotopic cortex were higher for same-category than different-category pairs in the Same Location condition, (t(5) = 3.609, p=.015), demonstrating that activity in this region of cortex can distinguish among these three object categories, even though no stimuli were presented in the foveal retinotopic location. More strikingly yet, this object information was invariant to changes in stimulus location; That is, correlations were significantly higher for same-category than different-category also in the Different Location conditions, (t(5) = 2.660, p=.045), and object category information was not significantly stronger in the Same Location than Different Location conditions (see Figure 3D). Indeed, the invariance to stimulus location is as strong in the foveal ROI as in LOC (interactions were not significant for f ROI * Same/Different Location * Same/Different Category, F(1,5) = .79, p=.414, or for Same/Different Location * Same/Different Category, F(1,5)=2.49, p=.175). Importantly, mean percent signal change in the foveal ROI was very low and did not differ significantly across object categories, showing that it is the spatial pattern of response across voxels, not the mean response, that carries the object information (F(1,5) = .37, p=.70; See Supplementary Figure 2).

The presence of object information in foveal retinotopic cortex is surprising because the stimuli were presented in the periphery only. This object information is therefore present in a cortical region that is not involved in the feedforward processing of the stimuli. In addition, the information is position-invariant, a phenomenon that has not to our knowledge been reported previously in retinotopic cortex. The most parsimonious account of these findings is that object information from higher cortical areas (possibly including LOC) is fed back to foveal retinotopic cortex. However, before this hypothesis can be accepted, several alternatives must be considered.

Testing Alternatives to the Feedback Hypothesis

First, might the Foveal ROI show response properties that should have been attributed to the Object Location ROI, but that somehow got spuriously attributed to the Foveal ROI?

This could result either if some voxels that should have been assigned to the Object Location ROI were wrongly assigned to the Foveal ROI, or if functional signals from the Object Location ROI were displaced to the foveal region by veins 18 or ghosting artifacts. This hypothesis cannot account for the fact that the Foveal ROI shows position-invariant category information whereas the peripheral ROIs do not.

Second, might category information in the foveal ROI result from participants actually foveating the stimuli? This is unlikely because it would place the other stimulus so far in the periphery that task performance would be nearly impossible. More subtly, might subjects have made consistently different eye movements for each object class? This hypothesis seems unlikely because such category-specific eye movement patterns would have to be the same across stimulus location to produce the observed position-invariant information. Nonetheless we tested both possibilities by analyzing the eye movement data from Experiment 1. There was no significant difference in eye movements or pupil diameter between the object categories (Horizontal: F(1,4)=.62, p=.473; Vertical: F(1,4)=.26, p=.632; Pupil: F(1,4)=.66, p=.540 see Supplementary Figure 3).

Finally, even if eye movements are too small to reach significance in our eye tracker data, might they nonetheless produce systematically different patterns of activation in the foveal ROI because of their effect on the cortical response to the fixation cross itself? To test this hypothesis, in Experiment 2 we re-ran the experiment on well-trained subjects without the central fixation cross. Instead, four crosses were placed permanently in the location of the objects and subjects were asked to fixate at the implied central intersection of the crosses. The significant object information in the foveal ROI remained under these conditions (F(1,15)=32,83, p = .029; see Experiment 2, Figure 4), thus ruling out an account of the phenomenon in terms of a response to the foveal fixation cross.

Figure 4

Mean correlations (± 1 SE) within- and between-categories (always between locations) for two ROIs in Experiment 2. Three subjects participated in this experiment, which was identical to Experiment 1 except that no central fixation was presented. A) Lateral Occipital Complex (LOC; objects>scrambled objects); B) The foveal representation in retinotopic cortex (Foveal; central object presentation>fixation).

Taken together, these considerations provide compelling evidence that position-invariant object information from higher cortical areas (possibly including LOC) is fed back to foveal retinotopic cortex. Next we consider further the spatial specificity of the effect and its relevance to perception.

Is the object information restricted to the foveal region?

Might the effect observed here reflect feature-based attention 19, 20? Although feature-based attention effects can spread beyond the attended object 21, and one recent paper found feature-based attention effects across the entire visual field 20, no prior data or theory would predict that feature-based attention effects should be restricted to foveal retinotopic cortex. Given this apparently critical difference between our result and feature-based attention, we revisited this question in Experiment 3, which more precisely tested the spatial specificity of our effects by including an eccentricity mapping localizer scan. Specifically, our localizer included three disc or ring-shaped stimuli composed of a flashing checkerboard texture: A foveal disc 2 degrees in radius, a middle eccentricity annulus extending from 2 to 4 degrees radius, and an outer annulus extending from 4 to 6.7 degrees radius (excluding the four stimulus locations). We measured position-invariant object information in each of these ROIs in an experiment otherwise identical to Experiment 1; the results are shown in Figure 5. An omnibus ANOVA across subjects on the correlations found a significant two-way interaction of ROI (Foveal/Outer Ring) * Same/Different Category (F(1,4) = 8.941, p=.040), indicating that the pattern of correlations differed significantly across ROIs. We therefore analyzed each ROI separately, and found significantly higher correlations for same than different categories only in the Foveal ROI (t = 3.840, p = .018), not in the Outer Ring ROI (t = 1.611, p=.183) or the Middle Ring ROI (t = .975, p = .385). The same pattern of results was observed when linear SVMs were used to analyze the data instead of correlations; see Supplementary Figure 4. These results confirm the spatial specificity of object information to foveal retinotopic cortex, and render our findings difficult to explain in terms of feature-based attention.

Figure 5

ROIs in one example subject and the corresponding mean correlations. A) Inflated cortical surface with this subject’s functionally-defined ROIs in color: Outer Ring (Object Location; Outer Ring>Middle Ring) in blue, Middle Ring (Middle Ring > Outer Ring + Foveal Ring) in green and; Foveal retinotopic cortex (Foveal Ring>Mddle Ring) in red, on the cortical surface (inflation was performed using Freesurfer 34). B) Mean correlations (± 1 SE) within- and between-categories (always between locations) for three ROIs in Experiment 3. Five subjects participated in this experiment, which was identical to Experiment 1 except that a checkerboard eccentricity mapping and meridian mapping were conducted.

According to a second alternative hypothesis, perhaps the object information is not restricted to the fovea per se, but rather is found at retinotopic positions corresponding to the spatial midpoint between the two simultaneously-presented stimuli, which happened to coincide with the fovea in our previous experiments. In a fourth experiment we tested this hypothesis by presenting the two stimuli either in the two upper positions, or the two lower positions, rather than along diagonals. Here again we found position-invariant object information in the foveal ROI, i.e. a significantly higher correlation within object category than between object category (t(4)= 6.13, p < .005) for the between-location analysis, even though the fovea was not between the two simultaneously-presented stimuli in this experiment. Further, we found no significant object information in ROIs positioned at the midpoint between the two stimuli (t(4) = 1.523, p = .202, upper and lower ROIs separately calculated and averaged), providing no evidence for an additional effect of the midpoint position. This experiment generalizes our result to a new stimulus configuration and shows that the foveal specificity of our effect is not an artifact of stimulus configurations that straddle the foveal region.

Behavioral Relevance of Category Information

Is the object information we find in foveal retinotopic cortex epiphenomenal (i.e., unrelated to task performance), as found in several prior studies of pattern information in retinotopic cortex 15, 20, 22, 23, or does it reflect a behaviourally relevant process that improves task performance? In Experiment 5, we addressed this question by asking whether foveal cortex contains category information whenever objects are presented, or only when the participant performs an object discrimination task.

Object information in Experiment 5 was found in the foveal retinotopic ROI only when subjects performed an object comparison task (t(5) = 5.93, p=.004), not when they performed an equally-difficult color comparison task on the same stimuli (t(5) = 2.8, p=.070). This task dependence was supported by a significant two-way interaction of Task * Same/Different Category, (F(1,5) = 58.38, p=.001; Figure 6). Further, task modulated object information more strongly in the foveal ROI than in LOC (significant interaction of Task * ROI * Same/Different Category (F(1,5) = 8.12, p=.036; Figure 6), where there was no significant difference between the strength of object information in the colour and shape tasks

Figure 6

Correlations within and between categories (always between locations) for LOC and the foveal retinotopic cortex in Experiment 4, showing that task modulates object information more strongly in the foveal ROI than in LOC. A) Results from the Object Discrimination task showing object information in both LOC and foveal retinotopic cortex; B) Results from the Color Task demonstrating object category information in LOC but not in foveal retinotopic cortex.

To further test the link between the object information in foveal retinotopic cortex and behavioural performance, we binned the data from Experiments 1 and 6 into four consecutive 4-second time bins within each block of a particular stimulus category. Interestingly, both behavioral accuracy (Figure 7A, bottom), and object information in foveal retinotopic cortex (Figure 7A, top), build up gradually over the course of each block. This increase in pattern information over the course of each block was specific to the foveal ROI (significant interaction of ROI * Time Bin, F(1,11) = 3.32, p =.032; Fig 7A). Finally, the magnitude of behavioral improvement over the block was correlated across subjects with the magnitude of the increase in category information for foveal retinotopic cortex (r2 = .64, p=.027), but not for LOC (r2 = .24, p=.446) or for the Object Location ROI (r2 = .10, p=.761) (see Figure 7). Thus, position-invariant object information accrues in foveal retinotopic cortex over successive trials of the same stimulus type, in lockstep with improved behavioral performance.

Figure 7

Time course and behavioral relevance of position-invariant information in foveal retinotopic ROI in Experiments 1 and 4

A) Top graph: Time course of appearance of object category information (Between Locations) across each block, for foveal ROI, LOC, and Object Location ROI; each time bin represents two trials and four seconds. Bottom graph: Corresponding behavioral data collected from the same subjects during scanning, showing that performance increases across trials within a block, mirroring the increase in category information in the foveal ROI. B) Scatterplots showing the correlation across subjects between the amount of increase in performance from the beginning to the end of blocks, and the increase in the amount of object information in each ROI. Note that only for foveal retinotopic cortex is category information correlated with behavioral performance15.

In sum, the object information in foveal cortex is behaviorally relevant both in the sense that it is found only when the task requires object form information, and in the sense that stronger information in foveal cortex is correlated with higher task performance.

Discussion

We have now replicated in six independent experiments the finding of significant position-invariant information in foveal retinotopic cortex about peripherally-presented objects (for the sixth Experiment, see Supplemental Figure 5). Our data cannot be explained by differential eye movements across object categories, or activation from the fixation cross itself. The results cannot be explained by spillover activation from peripheral to foveal retinotopic cortex, because the latter is position-invariant and the former is not (see Figure 3) or from LOC, because the object information in foveal retinotopic cortex is task-dependent whereas object information in LOC is not (see below and Figure 6,7). Although fMRI cannot directly distinguish feedback from feedforward responses, it is difficult to account for our results without invoking feedback, because the object information is only present in foveal retinotopic cortex when the subject performs an object discrimination task (as shown in Experiment 4). Passive propagation of information across retinotopic cortex, in the absence of feedback from higher areas, would not explain why the information would be propagated to the fovea, but not to other locations in retinotopic cortex, or how the object information can be position invariant in foveal retinotopic cortex when it is not position invariant in the Object Location ROI. Furthermore, the lateral spread of monosynaoptic horizontal connections is less than three degrees of visual angle 24, much less than the eccentricity of our peripheral stimuli from the fovea. Instead, our data strongly suggest that position-invariant object information from higher cortical areas (possibly including LOC) is fed back to foveal retinotopic cortex.

In contrast to the predictions of virtually all prior theories of feedback in perceptual processing, the feedback representation described here occurs in a completely different cortical location from the corresponding feedforward representation. Although one recent study reported information in retinotopic cortex adjacent to the region directly activated by the stimulus 25, and another reported the presence of feature-based attention signals across the entire visual field 20, our findings show a case in which the feedback representation occurs far from the stimulus location, in foveal retinotopic cortex, and not in other regions of retinotopic cortex (see Figures 3, 5). Finally, unlike all prior reports, the object information we describe in retinotopic cortex is invariant to stimulus location.

Most importantly, the object information in foveal cortex reported here is related to behavior. The foveal information is present only when subjects perform an object discrimination task, not when they perform a color task on the same stimuli, and the strength of the object information in foveal retinotopic cortex is correlated with subjects’ accuracy discriminating the peripherally-presented objects.

How exactly might feedback of object information to retinotopic cortex improve task performance? Automatic predictions of imminent saccades to peripheral stimuli might lead to predictive signals in foveal retinotopic cortex (where peripherally-attended stimuli frequently land next)26. However, this hypothesis cannot easily account for our finding that information in foveal cortex i) is weak or nonexistent at the beginning of each block, and ii) builds up over successive trials with the same object category (in which subjects never successfully foveate the object). Instead, the gradual improvement in behavioral performance over the course of each block suggests that as the block proceeds subjects develop a more precise mental representation of the particular object category (and/or its diagnostic features). Further, this improvement in performance is correlated with the appearance of category information in foveal retinotopic cortex. Thus one possibility is that foveal retinotopic cortex may serve as a kind of “scratchpad” to store or compute task-relevant visual information 27. This proposal is in line with the ideas of Mumford 28, and others 29 according to which “the best description of V1 is not the first stage in a feedforward pipeline (or the last in a fully top-down conception of brain function) but rather the unique high-resolution buffer in the visual system for geometric calculations” 28–30. Note that our results do not constrain the neuroanatomical pathways through which the feedback modulates the information contained in foveal retinotopic cortex. For example, feedback might directly modulate the activity of foveal neurons, or, alternatively, it might modulate the influence of horizontal connections 29.

Another open question is whether the fMRI patterns in foveal retinotopic cortex reflect a sub-threshold synaptic signal rather than the spiking output of neurons. The fMRI BOLD signal is strongly related to synaptic activity as measured by local field potentials 31, 32, and some cognitive signals have been shown to be stronger in local field potentials than in spiking output 33. Thus, it is possible that the fMRI patterns we observed in our experiments are not associated with differences in spiking output. To test this prediction, invasive extracellular recordings are needed, which will require the implementation of our paradigm in monkeys.

Whatever the ultimate understanding of its precise function, our data demonstrate for the first time the existence of position-invariant information in foveal retinotopic cortex, and strongly suggest that this information arises by feedback from higher cortical areas.

These findings open the door to a broad new landscape of investigation. Exactly what kind of object information is fed back to foveal retinotopic cortex, and over what range of information processing tasks does feedback occur? Why is position-invariant information fed back to the foveal region of retinotopic cortex in particular? By what mechanism does feedback improve task performance? The answers to these questions are bound to have fundamental implications for our understanding of visual information processing.

Methods

Subjects

Eight subjects were recruited for Experiment 1. One had to be excluded due to excessive head movement (>8mm across experiment), and one further participant was removed due to a technical issue with the eye-tracking system.

Six subjects were recruited for the Colour experiment (Experiment 2) and all were used during analysis.

Eight subjects were recruited for the eccentricity mapping experiment (Experiment 3). One was excluded as s/he withdraw from the experiment and two subjects were excluded due to non-significant localiser results.

Eight subjects were recruited for Experiment 4. One was excluded due to chance-level behavioral performance and two were excluded due to excessive eye movements.

Four subjects were recruited for the No Central Fixation experiment (Experiment 5). One had to be excluded due to excessive head movement (>8mm across experiment).

Seven subjects were recruited for Experiment 6. One had to be excluded due to excessive head movement (>8mm across experiment).

Experiment 1

Subjects viewed three categories of novel objects (“spikies”, “smoothies”, and “cubies”). Two objects of the same category were presented in diagonally opposite quadrants of the visual field (upper left/lower right or upper right/lower left) and participants indicated if the objects were same or different (i.e., within-category discrimination) via a 2-button response box. Objects (mean width 1.8° of visual angle) were presented 7° from fixation to center of objects for 100ms, followed by a blank inter-stimulus interval (ISI) for 1900ms. Each block contained 8 trials (i.e., 16 sec per block) with 4 blocks of each object category in each run. The order of object category was counterbalanced across runs and subjects. The accuracy of the participant’s response (across all 3 object categories) was 68% (± 9 SE) over the scan. Fixation was monitored using an I-Scan (ISCAN inc., Burlington, MA) eye-tracking system.

We identified the critical regions of interest in four independent localizer runs. Each run consisted of four 16-second blocks of faces, scenes, common objects and scrambled objects (totalling 16 blocks per category). These stimuli were 8° of visual angle and presented at the fovea to ensure no overlap with the objects in the experimental run. We defined the lateral occipital complex (LOC; Objects>Scrambled), the foveal representation in retinotopic cortex (Objects and Scrambled>Rest taking active voxels only on the occipital poles), and the retinotopic representation of the experimental stimuli (smoothies, spikies, cubies > rest, using half experimental runs restricted to voxels close to the calcarine sulcus). To confirm no overlap between the ROIs, we conducted a conjunction analysis using fROI (http://froi.sourceforge.net/).

To ensure that our foveal” ROI did not accidentally contain a few voxels that actually included the cortical region representing the peripheral stimuli, we checked that there was no overlap between the ‘foveal ROI’ and an ROI containing all active voxels in retinotopic cortex during presentation of the objects (see Figure 2). The “foveal” ROI includes all early retinotopic visual areas, as there is no way of separating these regions at the occipital pole in humans with fMRI 35.

To ensure the effects could not be explained by differences in the number of voxels in each ROI we adjusted the threshold (minimum threshold was 10−6) to approximately equate the number of active voxels (Foveal ROI: Mean = 839; SE = 61; Object Location ROI: Mean = 776; SE = 94; LOC ROI: Mean = 849; SE = 93; F(1,5) = .954, p = .374)

Experiment 2 (No Fixation cross)

The methods were similar to those of Experiment 1 except that there was no central fixation. Instead, four crosses were placed permanently in the location of the objects and subjects were asked to fixate at the intersection of these fixations.

Experiment 3 (Eccentricity mapping)

The methods were similar to those of Experiment 1 except that for the independent localiser task. The localizer included three disc or ring-shaped stimuli composed of a flashing checkerboard texture: A foveal disc 2 degrees in radius, a middle eccentricity annulus extending from 2 to 4 degrees radius, and an outer annulus extending from 4 to 6.7 degrees radius (excluding the four stimulus locations). Standard meridian mapping was also conducted to ascertain the V1/V2 borders using with flashing horizontal and vertical checkerboards.

Experiment 4(Stimuli both above or both below fixation, not on diagonal =straddling fixation)

The methods were identical to those of Experiment 1 except that the two stimuli were presented either in the two upper positions, or the two lower positions, rather than along diagonals. Eye position was monitored. The foveal ROI and the between-location ROI (midway between the two stimuli) were localized by independent localizer scans in which square-shaped stimuli with a flashing checkerboard pattern (3.3 degrees) were presented in the corresponding location.

Experiment 5 (Color and Shape)

The methods were similar to those of Experiment 1 except that the objects were colored either red, blue, or green. In half the runs, subjects were asked to perform the original object discrimination task and in the other half they were asked to perform a difficult colour discrimination task (same/different colour judgement). Six runs of each task were performed in addition to the four independent localizer runs.

Experiment 6

The methods were identical to those of Experiment 1 except that no eye tracker was used; This experiment was conducted before the others and is included here simply to show an additional replication of the basic effect (see Supplementary Figure 3).

FMRI Scanning

Scanning was done on a 3T Siemens Trio scanner at the Athinoula A. Martinos Imaging Centre at the McGovern Institute for Brain Research at MIT. FMRI data analysis was conducted using FreeSurfer Functional Analysis STream (FS-FAST; http://surfer.nmr.mgh.harvard.edu/) and region of interest analysis was conducted using FS-Fast region of interest toolbox (FROI; http://froi.sourceforge.net/). Functional MRI runs were acquired using the standard 12-channel head matrix coil and a gradient-echo echo-planar sequence (TR=2s, TE=40ms, 1.4 * 1.4 * 2.0 mm + 20% spacing**maybe included matrix size and field of view**). The processing steps for both the localiser and experimental runs included motion correction and spatial smoothing with a 3mm kernel and linear trend removal. Note that only the localiser task was smoothed (correlation analysis was performed on unsmoothed data).

Data Analysis

Spatial response patterns were extracted separately for each subject, and each ROI. Thus for each subject and ROI we obtained a different spatial pattern for each combination of stimulus configuration (upper left & lower right versus lower left and upper right) and stimulus type (smoothie, spikie, and cubie). The colour experiment contained 3 runs in each half; all other experiments contained 4 runs in each half) separately for each combination of ROI and stimulus category. Each run contained six blocks of each condition (eight trials in each block) and seven fixation blocks. A gamma function with delta 2.25 and tau =1.25 was used to estimate the hemodynamic response for each voxel for the different category and location conditions. The mean response in each voxel across all conditions was then subtracted from the response to each individual condition in each half of the data before calculating the correlations. Within each ROI, we then computed the correlation between the spatial pattern of fMRI response resulting from the same stimulus category versus different stimulus categories. For example, we compared the pattern of response in the LOC for spikies with other spikies versus spikies with smoothies. If the region carries information about the category (spikie, smoothie, cubie), there will be a higher correlation between the same-category spatial patterns than different-category spatial patterns. 11, 15