Early involvement of prefrontal cortex in visual bottom-up attention.

Visual attention is guided to stimuli either on the basis of their intrinsic saliency against their background (bottom-up factors) or through willful search of known targets (top-down factors). Posterior parietal cortex (PPC) is thought to be important for the guidance of visual bottom-up attention, whereas dorsolateral prefrontal cortex is thought to represent top-down factors. Contrary to this established view, we found that, when monkeys were tested in a task requiring detection of a salient stimulus defined purely by bottom-up factors and whose identity was unknown before the presentation of a visual display, prefrontal neurons represented the salient stimulus no later than those in the PPC. This was true even though visual response latency was shorter in parietal than in prefrontal cortex. These results suggest an early involvement of the prefrontal cortex in the bottom-up guidance of visual attention.

The dorsolateral prefrontal (dlPFC) and posterior parietal cortex (PPC) are two brain areas activated during the processing of visuospatial information and orienting of attention, as evidenced by imaging and neurophysiological experiments[1,2,3,4,5]. Anatomical studies suggest that the posterior parietal cortex comprises the end stage of the dorsal visual pathway, receiving information from visual areas and transmitting it to the dorsolateral prefrontal cortex[6,7]. This hierarchical and serial nature of organization is captured in current models of visual processing and attention[8]. Recent physiological studies consistent with the serial view also suggest that bottom-up information about the location of salient stimuli is first represented in the posterior parietal cortex (the lateral intraparietal area) and only later in the prefrontal cortex (the frontal eye fields and dlPFC); top-down information follows the reverse course, being represented first in prefrontal and then in parietal cortex[9].

Determining the time course of saliency representation presents challenges however, stemming from the inherent presence of top-down and bottom-up factors in a behavioral task and the effect of planning of eye movements, which are intrinsically connected with visual attention circuits[10,11]. To obtain a physiological estimate of bottom-up representation in the prefrontal and parietal cortex while minimizing the influence of these factors, we recorded neuronal activity in a behavioral task that required subjects to detect a salient stimulus defined solely by bottom-up factors, and to respond with a lever release (not directed to the stimuli) while maintaining fixation. If the posterior parietal cortex plays a general role in the guidance of visual bottom-up attention then, we reasoned, an earlier representation of salient stimulus information would be expected in the activity of parietal neurons, under these circumstances as well. Contrary to this prediction, the results revealed an early involvement of the dorsolateral prefrontal cortex in the representation of salient stimuli.

RESULTS

Two monkeys were trained to perform a task that required them to detect a salient stimulus in a visual display (Fig. 1a) while recordings were performed from the dorsolateral prefrontal and posterior parietal cortex (Fig. 1b, S1). Neither the identity nor the location of the stimulus was known prior to the display appearance; the monkeys were required to recognize the salient (green or red) stimulus by virtue of its color difference from distractor stimuli of equal luminance. The animals indicated correct detection by releasing a lever when a subsequent stimulus appeared at the same location (Fig. 1a). To ensure that the stimuli used in this paradigm “popped out” (and attracted attention automatically), after recordings in this task were completed we retrained the monkeys in a reaction-time version of the task (Fig. 1c) requiring immediate release of the lever once a salient target was detected (and no lever release if it was absent). We found no significant effect of stimulus set size on behavioral response time (regression analysis, slopes of 1.4 and −2.2 ms/item for the two monkeys respectively, p>0.5 in both cases), indicating that stimuli popped out and did not require a serial search of items in the display (Fig. 2).

Figure 1

Schematic diagram of the behavioral tasks and the monkey brain. a. Delayed match-to-sample task. The monkey was required to locate the salient stimulus in the cue frame and release a lever when a matching stimulus appeared. b. Schematic diagram of the monkey brain and areas where recordings were performed. Abbreviations: AS, arcuate sulcus; IPS, intraparietal sulcus; PS, principal sulcus; STS, superior temporal sulcus. c. Reaction-time version of the task. The monkey was required to release the lever immediately if a target was present and continue to hold if it was absent.

Figure 2

Behavioral response time as a function of stimulus set size. The reaction time version of the task was used. a. Average of 30 sessions is plotted for one monkey. b. Average of 13 sessions is plotted for the second monkey using the dense stimulus array. Error bars represent standard deviations across individual trials.

Time of target discrimination in mean firing rate

A total of 1233 and 479 neurons were recorded during execution of the task of Fig. 1a from the two animals, respectively. We used the same selection criteria in all areas to identify neurons that were responsive to single visual stimuli and displayed significant selectivity for their spatial location (2-way ANOVA, p<0.05). To ensure that any difference in the time of target discrimination between areas was not due purely to a difference in proportion of neurons that were color selective, we initially excluded neurons with significant color selectivity from analysis. We then compared responses to arrays with the salient stimulus appearing in and out of their receptive fields, as defined by the responses to single color stimuli presented at trials randomly interleaved with the array presentations (although this comparison was not a criterion for selection). A total of 278 neurons were analyzed in this fashion in dlPFC, 187 neurons in the lateral intraparietal area (LIP), and 71 neurons in area 7a. In agreement with prior studies[9,12,13], many of the neurons responding to single stimuli were selective for salient stimuli of either color (Fig. S2). The time course of neuronal activity that represented the salient stimulus was investigated by comparing the discharge rate elicited by arrays with the target of either color appearing in the receptive field (RF), vs. arrays with distractors in the receptive field. We computed the time at which averaged population discharge rate for the salient stimulus became significantly higher than that of distractors (time of target discrimination, black arrows in Fig. 3), and used bootstrap methods to determine the variance of this estimate and the statistical significance of differences between areas. Times of target discriminations were 125 ms (s.d.=7 ms) for dlPFC, 125 ms (s.d.=15 ms) for LIP, and 152 ms (s.d.=16 ms) for area 7a neurons. Using a permutation test with equal sample sizes we found no significant difference between dlPFC and LIP times of target discrimination (p>0.5) but faster target discrimination in dlPFC than area 7a (p<0.05). Alternative measures of target estimation time also confirmed this early involvement of dlPFC, and in the two monkeys separately (Fig. S3a,b). Including neurons with color selectivity in this analysis (Fig. S4) only decreased the relative time of target discrimination in dlPFC (113 ms) compared to LIP (125 ms). Despite this early target discrimination observed in the dlPFC, visual response latencies to stimuli were shorter in the parietal than the prefrontal areas. Population latency values, defined as the times of the earliest visual responses in the population PSTH to arrays with the target in the receptive field (gray arrows in Fig. 3) were 52 ms (s.d.=5 ms) in dlPFC, 42 ms (s.d.=10 ms) in LIP, and 49 ms (s.d.=8 ms) in area 7a.

Figure 3

Population firing rate. a–c. Average discharge rates from dlPFC (N=278), LIP (N=187), and area 7a (N=71) respectively. Peri stimulus time histograms are plotted with mean discharge rate elicited during presentation of target in the receptive field (RF) and distractor in the receptive field conditions over fixation (F), cue (C), and delay (D) period. Shaded area represents standard error of the mean discharge rate. The dotted vertical lines illustrate the time of cue onset and offset. Gray arrows in each panel indicate the population visual response latency; black arrows, the population time of neuronal target discrimination. Insets are schematic illustrations of stimulus condition in or out of the receptive field; each neuron’s receptive field was located at a different position.

We sought to verify these findings on a neuron-by-neuron basis as well (Fig. S5), identifying latency and time of target discrimination separately for each neuron with the same criteria described above (although for some neurons insufficient number of spikes were available to perform this analysis). The neuron-by-neuron analysis showed that average neuronal visual response latency was significantly longer (t-test, p<0.05) in dlPFC (mean=107, s.d.=82 ms) than in LIP (mean=93, s.d.=70 ms), whereas the average time of target discrimination (mean=182, s.d.=95 ms in dlPFC vs. mean=192, and s.d.=86 ms in LIP) was not significantly different between areas (t-test, p>0.3). The results of the neuron-by-neuron basis analysis confirmed that the target discrimination in prefrontal cortex occurred no later than the parietal cortex, suggesting that the location of the salient stimulus was computed independently within the prefrontal cortex, rather than being transmitted in a serial fashion from the parietal cortex. The result also implied that the available processing time between visual response latency and time of target discrimination is shorter in dlPFC than in LIP. Indeed, when we examined the processing time on a neuron-by-neuron basis, we found it to be significantly shorter (t-test, p<0.05) in dlPFC (75 ms) than LIP (99 ms).

To ensure that this early representation of a salient stimulus by the prefrontal cortex was not somehow the result of random variation of perceptual time across sessions in which the prefrontal and posterior parietal data were collected, we also analyzed recordings that were performed simultaneously from the prefrontal and the parietal cortex. In these sessions, 71 dlPFC and 61 LIP neurons were recorded simultaneously, as were 48 dlPFC neurons and 23 area 7a neurons. The times of target discrimination in this sample were again not significantly different between dlPFC and LIP (permutation test, p>0.05) and if anything, slightly earlier in dlPFC (121 vs. 128 ms in dlPFC and LIP, respectively); visual response latencies were 45 ms for dlPFC and 44 ms for LIP. The time of target discrimination was significantly earlier for dlPFC than area 7a in this sample, as well (permutation test, p<0.05).

Anatomical and functional evidence suggests that the prefrontal cortex itself may be organized in a hierarchical fashion across the rostro-caudal axis[14,15]. We therefore wished to determine whether an early time of target discrimination was present only in the posterior part of the prefrontal cortex. For this analysis, we divided recording sites in an anterior and a posterior half for each monkey, and repeated the estimate of visual response latency and time of target discrimination (Fig. S6). The population visual response latency for the posterior prefrontal cortex was indeed very similar to that of LIP (41 vs. 42 ms), though considerably longer for the anterior aspect of the dlPFC (55 ms). However, the time of target discrimination in the anterior dlPFC (129 ms) was still not significantly different (permutation test, p>0.3) than that of LIP (125 ms). In fact, the difference between visual response latency and time of target discrimination was the shortest in the anterior dlPFC (74 ms) compared to the posterior dlPFC (87 ms), and LIP (99 ms). The results argue against a serial transmission of the saliency signal from LIP first to the posterior dlPFC and then to the anterior dlPFC.

Target discriminability

The finding of an early prefrontal activation in response to the salient stimulus also translated into an early target discrimination, from a signal detection standpoint. A time-resolved receiver operating characteristic (ROC) analysis comparing responses to a salient stimulus vs. a distractor indicated that the area under the ROC curve, constructed based on average neuronal responses, rose in dlPFC as early as in LIP, and faster than in area 7a (Fig. 4a). To ensure that this average ROC value did not obscure a population of LIP neurons that may be able to detect the salient stimulus faster, we also performed a neuron-by-neuron ROC analysis (Fig. 4b). A bootstrap test was used to estimate the time point when the area under the ROC curve became significantly different than chance. A total of 156 dlPFC neurons, 116 LIP neurons, and 27 area 7a neurons reached significance based on this criterion. No significant difference existed between the dlPFC and LIP distributions (Kolmogorov-Smirnov test, p>0.1); if anything, the LIP distribution trailed dlPFC at early time points between 100–150 ms after stimulus onset. Similar patterns of ROC values were observed in both monkeys (Fig. S3c,d).

Figure 4

Receiver operating characteristic (ROC) analysis and mutual information analysis. a. Area under the ROC curve is plotted as a function of time. Solid vertical line represents the time of cue onset. Dotted line indicates the average mid-point between the peak and baseline of recorded neurons irrespective of area. Red, cyan, and blue curves represent dlPFC (N=278), LIP (N=187), and area 7a (N=71) respectively. b. Cumulative distribution of neurons with significant area under the ROC curve. Each vertical arrow indicates the time point when 50% of neurons of an area reached significance (dlPFC: N=170, LIP: N=114, 7a: N=31). c. Proportion of neurons with significant mutual information as a function of time. Total sample size, dlPFC: N=793, LIP: N=406, 7a: N=485. d. Cumulative distribution of neurons that represented significant mutual information (dlPFC: N=137, LIP: N=83, 7a: N=20). Vertical arrow indicates the time point when 50% of neurons in an area reached statistical significance.

Target discrimination based on neuronal responses is essentially a signal detection problem and stronger signals will on average be detected earlier than weaker signals. For this reason, early detection of the target in the prefrontal cortex may merely be an effect of stronger discrimination of the stimuli. We therefore examined the relationship between discriminability (determined as the area under the ROC) and time of target discrimination (Fig. 5). As predicted, the two variables were negatively correlated (PFC: r= −0.28, p<10−4; LIP: r= 0.15, p=0.08). We then performed an analysis of covariance (ANCOVA) comparing the times of target discrimination between areas, treating the area under the ROC curve as a covariate. When discounting for the effect of strength of discriminability in this way, dlPFC discriminated the target faster than LIP by 6 ms on average, a non-significant difference (intercept comparison, p>0.05).We also analyzed the group of neurons at the top quartile of discrimination values, pooled across both areas. Target discrimination of dlPFC neurons in this sample was 27 ms faster than that of LIP neurons (which had essentially identical mean ROC values of 0.900 and 0.898), a difference at the margin of statistical significance (t-test, p=0.048).

Figure 5

Relationship between discriminability and time of target discrimination. Area under the ROC curve over the entire stimulus presentation period (discriminability value) is plotted as a function of the time of target discrimination for each neuron for a, dlPFC b, LIP. Solid lines represent regression of area under the ROC curve on time of target discrimination. Arrows indicate the top quartile of ROC values obtained after pooling the values of both areas together; vertical dotted lines represent the mean time of target discrimination in each area based on neurons with ROC values above the arrow.

The analysis presented so far was based on neurons with significant visual responses and spatial selectivity for single stimuli. It has been argued, however, that information about salient stimuli may be more subtle, and that an informational theoretical approach examining data from all neurons may be more appropriate[9]. We therefore performed a mutual information analysis on all recorded neurons, with no regard for spatial selectivity – or even overt responses to visual stimuli. Our entire sample of 793 dlPFC neurons, 406 LIP neurons, and 485 area 7a neurons were tested in this way. The mutual information statistic reflects how well one can separate the salient stimulus location from other locations based on the firing rate of a neuron. The number of neurons with significant information in each time bin started rising in dlPFC and LIP with similar time courses (Fig. 4c). This was also illustrated in the cumulative distribution of neurons that reflected significant information at each time point (Fig. 4d) which was not significantly different between areas LIP and dlPFC (Kolmogorov-Smirnov test, p>0.7). Similar patterns were observed for the two monkeys (Fig. S3e,f). The results confirm that dlPFC neurons reflect information about the target location no later than posterior parietal ones.

Reaction-time task

It could be argued that the time of target discrimination reflected in the neuronal activity in our task was not critical for the selection of salient stimuli since the subjects were only tested after the presentation of the stimulus array and an intervening delay period (Fig. 1a). For this reason, we also collected recordings from 561 neurons in a reaction-time version of the task (Fig. 1c). In this task, the monkey was required to release the lever as fast as it detected the salient stimulus (and to hold if a salient stimulus was not present). Analysis of neuronal recordings revealed that the neuronal target detection in this task were overall faster than in the match-to-sample task (Fig. 6a,b). The average times of target detection were 107 ms for dlPFC, 105 ms for LIP, and 120 ms for 7a, an average of 23 ms difference over the match-to-sample task. The time of target detection remained not-significantly different between dlPFC and LIP (permutation test, p>0.4), although the average response latency was again considerably shorter for LIP than dlPFC (47 vs. 60 ms, respectively). No earlier target detection was observed for LIP, when we aligned responses to the lever release, and comparing neurons recorded simultaneously (Fig. 6c,d). ROC and mutual information analysis confirmed that target discriminability and information did not appear earlier in LIP than dlPFC (Kolmogorov-Smirnov test, p>0.6 for ROC comparison in Fig. S7b, p>0.3 for mutual information comparison in Fig. S7d).

Figure 6

Population responses during the reaction-time task. a,b. Mean discharge rate of target in the receptive field (RF) and distractor in the receptive field during fixation (F), cue (C), and delay (D) periods are plotted (dlPFC: N=42, LIP: N=36 neurons from one monkey). The arrows indicate the visual response latency (gray arrow) and the time of neuronal target discrimination (black arrow). c,d. Average discharge rate synchronized to lever release; only neurons recorded simultaneously are included here (dlPFC: N=17, LIP: N=9). Vertical red line and gray zone represents mean and 1 standard deviation around the time of stimulus presentation, relative to the response time.

To gain insight on the nature of salient stimulus representation, we also examined error trials in which the target was present but the monkey missed it and those in which the target was absent, yet the monkey falsely reported its presence (Fig. S8). Comparing firing rates of neurons for which both misses and false alarms were available, we observed a similar pattern between areas. Average firing rate in the time interval of 0–300 ms after the stimulus onset was lower in misses than in hits for both dlPFC (26.0 vs. 29.4 spikes/s) and LIP (18.5 vs. 20.3 spikes/s). Firing rate was greater in false alarm than in correct rejections (22.3 vs. 21.8 spikes/s in dlPFC, and 17.9 vs. 15.6 spikes/s in LIP). The time course of the difference in firing rate in correct and error trials had little predictive power over the time course of target discrimination, as the two were not significantly correlated in either area (r=0.2 in dlPFC, r=0.02 in LIP, p>0.2 in both cases). This result indicates that firing rate differences in correct and error trials were not tightly tied with the time of target discrimination in either area and the time of target discrimination could not be accounted for by behavioral choice. We did note however that the difference in response between target and distractors (Fig. S8c) diminished greatly in error trials for LIP (−87%), and less so for dlPFC (−51%).

Difficulty and display size

It is conceivable that the lack of a temporal advantage by the posterior parietal cortex only applies to unambiguous, highly salient stimuli; the parietal cortex may still be critical in orienting attention to less salient stimuli. We examined this possibility by varying the distractor color so as to render the target stimulus more difficult to detect. In an initial set of experiments we identified four levels of chromatic contrast over which performance declined in an approximately linear fashion (Fig. S9a). We then analyzed neuronal activity recorded from 58 dlPFC, 55 LIP and 25 area 7a neurons during presentation of these displays in the context of the match-to-sample task. In agreement to previous studies[16], responses to arrays with target stimuli among more similar distractors were generally weaker and the time of target discrimination occurred later (Fig. 7a). However, we did not observe a significant advantage of the posterior parietal cortex over dlPFC (Fig. 7b), for any level of difficulty (permutation test, p>0.1 for all comparisons), or based on ROC and mutual information analysis (Fig. 8). This result was consistent with an early involvement of dlPFC in the representation of target stimuli even when they were more difficult to discriminate.

Figure 7

Population responses in the difficult-discrimination task. a. Responses of target in the receptive field (red lines) and distractor in the receptive field (blue lines) are plotted for the three brain areas at four levels of difficulty (dlPFC: N=58, LIP: N=55, area 7a: N=25). The arrows indicate the response onset latency (gray) and the time of neuronal target discrimination (black arrow). Data from salient stimuli of both colors have been pooled together (only one color is shown in the insets). b. Average difference in responses between a salient stimulus and a distractor in the difficult-discrimination task. Vertical lines represent time of target discrimination in dlPFC (red line) and LIP (cyan line).

Figure 8

ROC and mutual information analysis for responses in the difficult-discrimination task. a–c. Average ROC values obtained from three levels of difficulty, respectively. Vertical lines illustrate the time of cue onset. d–f. Cumulative distribution of neurons with significant area under the ROC curve. g–i. Cumulative distribution of neurons that represented significant mutual information. Each vertical arrow indicates the time point when 50% of neurons in an area reached significance.

It is also possible that systematic differences in receptive field size between the prefrontal and posterior parietal neurons we sampled influence the processing time for displays of different size. To determine whether the early activation of the prefrontal cortex generalizes across display size, we trained the monkeys to perform the reaction-time version of the task, using a denser array of 3° stimuli, spaced 7° apart from each other (instead of 15°, for the previous experiments). A total of 140 and 399 neurons were recorded from the two monkeys respectively, 242 from dlPFC and 297 from LIP. Difference in firing rate for the best and worst location in this dense array was lower overall (Fig. S10), however the timing of mutual information for stimulus location indicated no significant difference between the distributions of dlPFC and LIP neurons for this stimulus set, either (Kolmogorov-Smirnov test, p>0.9).

DISCUSSION

Early psychophysical experiments showed that visual attention is guided automatically to stimuli that stand from their background by virtue of their inherent saliency, a process referred to as “bottom-up” attention[17,18]. Attention may also be allocated willfully, e.g. when actively scanning the visual field for a known target, known as “top-down” attention[19]. Neurophysiological correlates of selective representation of stimuli that differ from their local background are evident as early as area V1, and with very short latencies[20]. However, responses in early visual areas are not adequate for guiding attention to the overall most salient stimulus in the field of view, as neurons have access to information over a small receptive field area[21,22]. For bottom-up attention to be deployed effectively, information from all feature streams is thought to be represented in a global saliency or priority map[23,24]. It is now clear that saliency maps are simultaneously present in multiple brain areas[25], including areas LIP and 7a in the parietal lobe[12,26], the frontal eye fields (FEF)[13] in the frontal lobe, and the superior colliculus[27]. Our present results demonstrate early involvement of the dorsolateral prefrontal cortex in visual attention guided by a target stimulus that is defined purely by bottom-up factors and so add dlPFC in this list. Our results also demonstrate that detection of the salient stimulus in these areas proceeds in a parallel, rather than a serial manner, unlike the pattern of visual response latency.

Our finding of early involvement of the prefrontal cortex in bottom-up attention is consistent with some neurophysiological studies that have suggested similar patterns of activation in parietal and frontal areas (LIP and FEF), though these studies did not involve recordings from both areas in the same animals[28,29]. Recent imaging studies also report robust prefrontal activation in bottom-up visual search[30]. Finally, anatomical studies show that the pattern of projections between the parietal and prefrontal cortex is parallel (originating from and terminating to the same layers in both directions) rather than strictly serial[7,31], as would have been predicted by a hierarchical relationship between the parietal and prefrontal cortex[25].

On the other hand, our conclusions stand in contrast with a previous neurophysiological study finding faster LIP recruitment during detection of pop-out stimuli[9]. The study reported comparable mean time of target discrimination in LIP to the 105 ms we report here for the reaction time task (based on firing rate), but much slower for FEF (>150 ms) and dlPFC. There are some critical differences between the studies, including the absence of a saccade requirement in our task, which preferentially activates LIP neurons[32], and the use of stimuli appearing in peripheral vision in our task (where most ecologically important stimuli are likely to appear first) as opposed to perifoveal stimuli within 4° of the center of vision. The time of stimulus discrimination also appears to be considerably sensitive on stimulus parameters. Mean target discrimination times as early as 50–70 ms have been reported recently in the literature for area LIP[33,34]. Experiments relying on much larger stimuli in a match-to-sample task also resulted in faster discrimination times in area 7a[12] than those we report here for the same area.

Although we have emphasized that the target was defined by bottom-up factors in our task, top-down factors undoubtedly were present as well; the monkey was required to interpret the color of the stimuli in the display and conditionally release a lever. The earlier involvement of the prefrontal cortex in our study seems unlikely however to be driven purely by top-down factors since equivalent factors were present in the prior study[9] (in fact, it is arguable whether that task relied on bottom-up factors, since the animals were explicitly cued about the sought out target in each trial). We did see that the absolute time of target discrimination in neuronal activity decreased considerably when the same animals were required to perform a reaction time task, compared to the match-to-sample task. A corresponding change in the time of target discrimination has previously been observed in animals trained to search for a specific stimulus feature[35] (faster discrimination) or not required to perform a task at all[36] (slower discrimination). There is little doubt, therefore, that task demands can influence the time of target representation in neuronal activity, and comparison of the absolute time of target discrimination across experiments may be of little value. We should also point out that a similar time course of target representation does not necessarily imply that the two brain regions play identical roles. Our current results do provide evidence for early involvement of the prefrontal cortex in the guidance of visual attention by bottom up factors that cannot be accounted for by the posterior parietal cortex.

METHODS

Two male, rhesus monkeys (Macaca mulatta) weighing 5–8 kg were used in this study. All surgical and animal use procedures were conducted with approval from Institutional Animal Care and Use Committee of the Wake Forest University according to National Institutes of Health guidelines.

Behavioral tasks

The monkeys sat in front of a computer monitor positioned 60 cm away with their head fixed in a dark room. While the monkeys were maintaining their gaze on a white target of 0.2° in size located at the center of the screen, visual stimuli were presented. Eye position was sampled at 240 Hz, digitized, and recorded using an infrared eye position tracking system (model RK-716; ISCAN, Burlington, MA). If the animals broke fixation exceeding a 2° window, the trial was immediately terminated. The monkeys were rewarded with fruit juice upon correct completion of a trial. The visual stimulus presentation, online monitoring of eye position, and synchronization of stimuli with neurophysiological data were controlled by in-house software[37], implemented in MATLAB (Mathworks, Natick, MA), using the Psychophysics Toolbox[38].

Monkeys were tested with the delayed match-to-sample task which required them to locate a salient stimulus among distractors and to release a lever when a subsequent stimulus appeared at the location of the salient stimulus (Fig. 1a). The trial started with the animals pulling a behavioral lever and fixating at the center of the monitor. The cue was displayed at one of nine locations, with or without distractor stimuli. The cue was rendered salient due to its difference in color from 8 distractors arranged along a 3×3 grid of 15° separation between adjacent stimuli (diagonal elements appeared at an eccentricity of 21°). Stimuli consisted of green or red squares of 1.5° in size. In some sessions the salient stimulus appeared at any of 4 possible locations (either the cardinal or diagonal positions in the array). The location of the stimulus was randomized from trial to trial, and stimulus arrays with cue and distractors of either green or red color were randomly interleaved with equal probability. The monkeys therefore were not able to predict either the location or the identity of the salient stimulus. The cue was displayed for 0.5 s followed by a delay period of 1.0 s. Then, a pseudorandom sequence of 0–2 non-match stimuli was presented, each lasting 0.5 s and separated by delay periods of 0.5 s. The sequence ended with a match stimulus appearing at the same location as the cue. The animals were trained to hold the lever until a match presentation (0.5 s) and to release the lever within 0.5 s after the match stimulus disappeared in order to receive a liquid reward. Release of the lever at any other time during the trial was considered as an error, and the trial was immediately aborted.

A variation of the basic task (referred to as the “difficult-discrimination” task in the main text) involved the same target stimuli (red/green) appearing among distractors of varying similarity. Four levels of difficulty were used involving a) same distractor stimuli (green/red) as those used in the standard task, b) distractor stimuli identical to the target, rendering the presentation a “catch trial” rewarded randomly, and c–d) two intermediate levels of chromatic difference. Psychophysical performance decreased monotonically for targets of either color (Fig. S9).

To assess the animal’s speed in detecting the salient stimulus, a reaction-time version of the task was also used (Fig. 1c). In this task, the monkey was required to release the lever as quickly as possible when a target was present in the display and keep holding the lever if there was no target. If the monkey continued to hold the key after 0.8 s and a salient stimulus was present, then the trial was aborted; if the salient stimulus was not present then the monkey was rewarded. In this task, the duration of the fixation period also varied randomly so as to make it impossible to time the lever release. A denser stimulus array with 3° size stimuli, spaced 7° apart of each other was also used in the reaction-time task. Displays with variable number of distractor stimuli (0, 2, 4, or 8) were also used in some sessions, to determine the effect of set size on reaction time.

Surgery and neurophysiology

After the animals were trained, they were prepared for neurophysiological recordings. Two 20 mm diameter recording cylinders were implanted over the prefrontal and parietal cortex of the same hemisphere followed by a craniotomy. Extracellular recordings were performed using arrays of 2–8 microelectrodes in each cylinder. We used either glass-coated, tungsten electrodes of 250 μm diameter, with an impedance of 1 MΩ at 1 kHz (Alpha-Omega Engineering, Nazareth, Israel) or epoxylite-coated tungsten electrodes with a diameter of 125 μm and an impedance of 4 MΩ measured at 1 KHz (FHC Bowdoin, ME). Electrodes were positioned through a grid system and advanced into the cortex with a microdrive system (EPS drive, Alpha-Omega Engineering, Nazareth, Israel). The electrical signal obtained from each electrode was amplified, band-pass filtered between 500 and 8 kHz, and recorded with a modular data acquisition system (APM system, FHC, Bowdoin, ME). Waveforms that reached a user-defined threshold were sampled at 25 μs resolution, digitized, and stored for off-line analysis.

The anatomical location of electrode penetration was determined based on MR imaging of the brain after implantation of the recording cylinders. Prefrontal data were collected from areas 8a and 46 including both banks of the principal sulcus, the area between the principal and the arcuate sulcus, and the superior convexity of the lateral prefrontal cortex. Posterior parietal data were collected from area 7a at the crown of the gyrus posterior to the intraparietal sulcus and the lateral intraparietal area (LIP) in the lateral bank of the intraparietal sulcus, at depths >3 mm from the surface of the cortex. These parietal areas are directly interconnected with the prefrontal cortex[39].

Data analysis

Spike waveforms were sorted into separate units using an automated cluster analysis method based on the KlustaKwik algorithm[40], which applied principal component analysis of the waveforms. We then identified units with significant elevation of firing rate during the 0.5 s of visual stimulus presentation compared to 0.5 s interval of fixation (paired t-test, p<0.05). Only the data during the cue period from correct trials were used for the analysis. The spatial tuning of visually responsive neurons was assessed by comparing the discharge rates during the presentation of single stimuli of either color at the nine grid locations. Neurons with significant main effect of stimulus location (2-way ANOVA; p<0.05) but no significant main effect of color (2-way ANOVA; p>0.05) were included in analysis. An additional analysis using all neurons exhibiting significant location selectivity with or without color selectivity was also performed (Fig. S4). For the reaction-time task with a denser stimulus array, all neurons with significant visual response were used (Fig. S10).

Population peri-stimulus time histograms (PSTHs) were constructed averaging responses of all neurons from each area, pooling data from salient stimuli of both colors. Average population visual response latency in each area was determined based on the population responses, as the first of 10 consecutive 10 ms windows stepped by 1 ms that were significantly higher (paired t-test, p<0.05) than the baseline firing rate[41]. We relied on an analogous procedure to determine the time of target discrimination: we identified the time point of the first of 10 consecutive 10 ms bin windows stepped by 1 ms, for which population responses to a salient stimulus inside the receptive field were significantly higher than responses to distractors (paired t-test, p<0.05). Differences between areas were tested by using a bootstrap test[42] in which population PSTHs were computed by randomly sampling neurons regardless of area, calculating response latencies or target discrimination times, and repeating the procedure 1000 times. The significance value of the observed differences was determined based on the empirical distribution of latencies of the randomized tests. An alternative method of latency estimation (Fig. S3a,b) was based on determining the inflection point of the cumulative sum of neuronal responses[43].

We also analyzed trials that resulted in errors in the reaction time version of the task, excluding error trials due to breaks in fixation and blinks. Error trials were categorized into two types: misses in which the target was present but the monkey did not release the lever, and false alarms in which the target was absent but the monkey released the lever. Firing rate of each error type was computed and compared with firing rates of two types of correct trials (Fig. S8). For correct trials, the difference in firing rate between target-present and target-absent trials was computed across time. For error trials, the absolute difference in firing rate between miss and hit trials, as well as between false-alarm and correct rejection trials was calculated separately, and then averaged together. A correlation coefficient was computed between the averaged absolute values of firing rate difference obtained with error trials and the firing rate difference obtained with correct trials in the interval of 0–300 ms after cue onset. To investigate if target discriminability changed in error trials, averaged firing rates over 0–300 ms after cue onset were also calculated for each of the four conditions.

A receiver operating characteristic (ROC) analysis was performed by comparing the distributions of firing rates of a neuron to stimulus arrays with the salient stimulus appearing at the location that elicited the best responses in the receptive field and at its diametric location[12]. The area under the ROC curve represents the probability that an ideal observer can discriminate between a salient stimulus and a distractor based on their firing rate in each trial[44]. The analysis was performed in a time-resolved fashion, comparing responses in a 25 ms long moving window, computed in 10 ms steps[29]. A bootstrap test was also performed to evaluate the significance of the area under the ROC curve. For each neuron, ROC values were obtained based on samples of responses obtained with no regard of whether they were recorded during the presentation of the salient stimulus in the receptive field or opposite to it. Significance was evaluated for the observed ROC value based on the distribution of the randomized ROC values (bootstrap test with 1000 repetitions). For each bin, the observed ROC value was presumed to be significant if the value exceeded 95% of the randomized distribution (p<0.05). The time point when a neuron reached significance was defined as the first of two consecutive bins with significant ROC values.

For the comparison of the relationship between discriminability and time of target discrimination (Fig. 5), the area under the ROC curve was computed over the entire stimulus presentation period (500 ms) for each neuron. The top quartile of ROC values was determined after pooling the values of both areas together and used that as a common criterion for both areas. Neurons that met the criterion were used to compare the time of target discrimination between areas. Analysis of covariance (ANCOVA) was conducted by applying a linear model on the data from two areas given with the time of target discrimination as the dependent variable and target discriminability (the area under the ROC) as a covariate.

A mutual information statistic was calculated to determine how well the salient stimulus location can be discriminated from others, based on the firing rate of a neuron[45]. Mutual information was calculated in non-overlapping 25 ms time windows and values were averaged across neurons in each area. Statistical significance was determined by means of a bootstrap test: data were randomly sampled with no regard of the actual salient stimulus location. The process was repeated 1000 times and the observed mutual information value was compared to this randomized distribution. The time point when each neuron’s activity began to significantly represent the target location was defined as the time point when mutual information values became significant for two successive bins (one for the dense stimulus array).

Supplementary Figure 1. Locations of recordings in the delayed match-to-sample task. Electrode penetration sites are shown for two areas of each monkey: a,c. Posterior parietal cortex of the two monkeys. b,d. Prefrontal cortex of the two monkeys. Blue open circles and cyan filled circles indicate area 7a and LIP, respectively. Red circles indicate dlPFC. Abbreviations: AS, arcuate sulcus; IPS, intraparietal sulcus; LS, lunate sulcus; PS, principal sulcus; STS, superior temporal sulcus.

Supplementary Figure 2. Sample rasters and histograms of a PFC neuron for each stimulus presentation condition. Fixation, cue (gray shaded area), and delay periods are presented. The receptive field (RF) is indicated in orange. For the single stimulus presentation, the neuron displayed high activity when a stimulus of either color appeared in the receptive field (a,b), but almost no activity when it appeared out of the receptive field (e,f). Neuronal discharges were also greater during a presentation of salient stimulus of either color in the receptive field surrounded by distractors (c,d). Although there were brief responses when distractors appeared in the receptive field, those responses diminished quickly (g,h). The results indicated that this neuron selectively signaled a salient stimulus in the receptive field.

Supplementary Figure 3. Target discrimination time, ROC, and mutual information analysis for two monkeys separately. a. The difference between the discharge rate elicited by the salient stimulus vs. distractors is represented as a cumulative sum. Time of target discrimination is indicated as the inflection point of the cumulative sum curve. Results from one monkey (dlPFC: N=250 neurons, LIP: N=162 neurons, area 7a: N=49 neurons). b. Results from the second monkey (dlPFC: N=28 neurons, LIP: N=25 neurons, area 7a: N=22 neurons). c,d. Cumulative distribution of neurons that exhibited a significant area under the ROC curve in the two monkeys. e,f. Cumulative distribution of neurons with significant mutual information in the two monkeys. Each vertical arrow indicates the time point when 50 % of neurons in an area reached significance.

Supplementary Figure 4. Population PSTHs of location-selective neurons, including those with color selectivity. a–c. Average firing rates of dlPFC (N=303), LIP (N=192), and area 7a (N=74), respectively. Mean firing rate elicited during presentation of target in the receptive field (RF) and distractor in the receptive field conditions over the fixation (F), cue (C), and delay (D) period are plotted. Shaded area represents standard error of mean neuronal firing rate. The dotted vertical lines illustrate the time of cue onset and offset. Gray arrows in each panel indicate the population visual response latency; black arrows, the population time of neuronal target discrimination. Insets are schematic illustrations of stimulus condition in or out of the receptive field; each neuron’s receptive field was located at a different position.

Supplementary Figure 5. Neuron-by-neuron analysis of visual response latency and time of target discrimination. a,c,e. Distribution of visual response latencies for each area (dlPFC: N=240, LIP: N=168, area 7a: N=57). b,d,f. Distribution of times of target discrimination for each area (dlPFC: N=200, LIP: N=147, area 7a: N=43). The dotted lines represent means of each distribution.

Supplementary Figure 6. Population PSTHs of anterior vs. posterior dlPFC. Mean firing rates of target in the receptive field and distractor in the receptive field were computed for the anterior and posterior half of dlPFC recordings in each monkey. a. The anterior half (N=122). b. The posterior half (N=156). PSTHs are plotted over fixation (F), cue (C), and delay (D) period. Shaded area represents standard error of mean discharge rate. The dotted vertical lines illustrate the time of cue onset and offset. Gray arrows indicate the visual response latency; black arrows, the time of neuronal target discrimination. Insets are schematic illustrations of stimulus condition in or out of the receptive field; each neuron’s receptive field was at a different location.

Supplementary Figure 7. ROC and mutual information analysis in the reaction-time task. a. Average ROC values are plotted as a function of time (dlPFC: N=42, LIP: N=36, 7a: N=8). Vertical line illustrates the time of cue onset. Horizontal dotted line indicates the mid-point between the average peak and baseline ROC values of all neurons irrespective of area. b. Cumulative distribution of neurons with significant area under the ROC (dlPFC: N=36, LIP: N=32, 7a: N=6). c. Number of neurons with significant mutual information as a function of time (total number of neurons used for analysis, dlPFC: N=217, LIP: N=239, 7a: N=105). d. Cumulative distribution of neurons that exhibited significant mutual information (dlPFC: N=42, LIP: N=40, 7a: N=6). Each vertical arrow indicates the time point when 50 % of neurons in an area reached significance.

Supplementary Figure 8. Error trial analysis in the reaction-time task. a,b. PSTHs are plotted for correct (solid lines) and error trials (dotted lines), over fixation (F), cue (C), and delay (D) periods. Average firing rates of hits and correct rejections are represented by the red and blue solid lines, respectively. Average firing rates of all available miss trials (involving holding of the lever when there was a target) and false-alarms (involving release of the lever when there was no target) are represented by red and blue dotted lines, respectively. a. dlPFC (N=40). b. LIP (N=24). c. The absolute difference in firing rate obtained from 0–300 ms after cue onset between hit and correct rejection (cr) trials, miss and false-alarm (fa) trials, hit and miss trials, and false-alarm and correct rejection trials are plotted. Error bars represent standard errors of mean firing rates across individual neurons.

Supplementary Figure 9. Behavioral performance and reaction time in the difficult-discrimination task. a. Percentage of correct trials is plotted as a function of the degree of difference in color between salient stimulus and distractors (represented as 4 difficulty levels). Black and gray data points represent averages from trials with the red and green target, respectively. The delayed match-to-sample task with 4 difficulty levels including catch trials was used. Average of 36 sessions is plotted. Error bars represent standard errors of individual sessions. b. Behavioral response time as a function of stimulus set size. The reaction-time task with 3 difficulty levels (no catch trials) and 4 different set sizes was used. Averages of 12 sessions are shown. Blue, cyan, and red data points represent conditions of increasing difficulty. Error bars represent standard deviations across individual trials.

Supplementary Figure 10. PSTHs and mutual information analysis in the reaction-time task with a denser stimulus array. a,b. Average firing rates were computed from the dense stimulus arrays for neurons with significant visual responses (but not necessarily with location selectivity). a. dlPFC (N=108). b. LIP (N=190). Shaded area represents standard error of mean neuronal firing rate. Gray and black arrows indicate the visual response latency and the time of neuronal target discrimination, respectively. Insets are schematic illustrations of stimulus condition in or out of the receptive field. c–e. Mutual information analysis was performed on data from two monkeys together and each monkey separately. Vertical arrows indicate the time points when 50 % of neurons of an area reached significance. c. Cumulative distribution of neurons that represented significant mutual information for two monkeys together (dlPFC: N=242, LIP: N=297). d. Cumulative distribution for the first monkey (dlPFC: N=47, LIP: N=52). e. Cumulative distribution for the second monkey (dlPFC: N=183, LIP: N=216).