Differential effects of exogenous and endogenous cueing in multi-stream RSVP: implications for theories of attentional blink.

The attentional blink (AB) refers to the finding that performance on the second of two targets (T1 and T2) in a rapid serial visual presentation (RSVP) stream is impaired when the targets are presented within 200-500 ms. To explore the possible interaction between spatial attentional orienting and temporary attentional deficits, this study used central (endogenous) and peripheral (exogenous) cues in a multi-stream RSVP task and compared the endogenous and exogenous cueing effects inside and outside of the AB period. While the endogenous cueing effect was constant in magnitude over time, the exogenous cueing effect was significantly larger inside than outside of the AB period. Theoretical implications of these findings for the interaction between attention mechanisms in spatial and temporal domains are discussed.

Introduction

A crucial function of human goal-directed behavior is to identify targets among distractors. These distractors may appear simultaneously with a target but at different locations or appear at the same location as the target but in different time windows. As the amount of attentional resources is limited, the resources have to be allocated optimally along either the spatial or the temporal dimension or both dimensions, selecting information relevant to our goals or intentions. The mechanisms of attentional selection over spatial or temporal positions have typically been investigated in paradigms that do not have direct links with each other.

The attentional blink (AB) is a paradigm for examining attentional allocation in the temporal dimension. When observers search for two targets in a rapid serial visual presentation (RSVP) stream, they usually have no difficulty in reporting the first target (T1). But if the second target (T2) appears within the period of 200–500 ms after T1, the report accuracy drops dramatically relative to a condition in which T2 appears later than 500 ms (Broadbent and Broadbent 1987; Raymond et al. 1992). This deficit in T2 report is called attentional blink (AB), which is commonly assumed to reflect the difficulty in reallocating attentional resources from T1 to T2 within a relatively short period.

On the other hand, allocation of attentional resources over spatial locations is usually explored with the cue-target paradigm (Posner 1980), in which a cue preceding the target informs the observer about the potential location of the upcoming target.

Two types of cues have been differentiated: the peripheral cue and the central cue. The peripheral cue, occurring at the target location (for valid cueing) or a nontarget location (for invalid cueing), is usually an abrupt-onset stimulus and is noninformative with regard to the location of the subsequent target. It captures attention exogenously to the peripheral location, and its function is therefore usually called exogenous cueing. The central cue usually occurs at the central fixation position and indicates the likely target location in the periphery through an intentional interpretation of the meaning of the cue (e.g., an arrow pointing to the left is interpreted as indicating that the target is likely to be there). The central cue usually predicts the location of the subsequent target above chance level, orients attention to the cued location endogenously, and involves more complicated voluntary processes. This kind of cueing function is usually called endogenous cueing. As summarized by Klein (2004), exogenous and endogenous cueing processes differ in many aspects, including the time course of the cueing effect (Jonides 1981; Müller and Rabbitt 1989), the sensitivity to additional memory load and to cue validity (Jonides 1981).

Exogenous and endogenous cueing are subserved by different attention neural networks. The dorsal frontoparietal network, which includes the intraparietal sulcus and the frontal eye field, is involved in endogenous attentional control. The ventral attentional network, which includes the right temporal–parietal junction (rTPJ) and the right inferior frontal gyrus, is involved in exogenous attentional control (Corbetta et al. 2008; Corbetta and Shulman 2002; Shulman et al. 2009). For example, patients with lesions in TPJ have deficits in exogenous cueing, although they are able to use probability information to facilitate attention shifting (Friedrich et al. 1998). The rTPJ is known to act as a circuit-breaker for the dorsal frontoparietal network when it is activated by salient or relevant stimuli. It functions to reorient attentional resources to these stimuli (Corbetta et al. 2008; Corbetta and Shulman 2002; Shulman et al. 2009).

At the same time, the rTPJ is also involved in target detection and plays an important role in the AB (Hommel et al. 2006; Husain et al. 1997; Shapiro et al. 2002). For example, Shapiro et al. (2002) found that patients with TPJ lesions show stronger AB effect compared to healthy controls, whereas patients with a superior parietal lobe lesion do not differ in AB performance from controls. Thus, rTPJ might play a crucial role in both exogenous cueing and the AB. This overlap might predict an interaction between the exogenous cueing and the AB.

The main purpose of the present study is to provide behavioral evidence for this hypothesis. In other words, we would like to determine whether and how the facilitatory effects of spatial exogenous and endogenous cueing affect the AB deficit in the temporal domain. Such a finding would deepen our understanding of the attentional selection process in the temporal domain.

Earlier studies have provided clues to the answers to this question. On the one hand, some studies (Nieuwenstein 2006; Nieuwenstein et al. 2005) used a task-irrelevant feature on a preceding distractor to cue the upcoming T2 in the RSVP and observed a facilitatory cueing effect inside but not outside the AB period. Other studies (Experiment 4 in Nieuwenstein et al. 2005; Zhang et al. 2007a) combined the exogenous cueing technique with the attentional dwell-time paradigm (Duncan et al. 1994). In Zhang et al. (2007a), two masked targets were presented at different locations. In addition, a peripheral cue with 100% (informative) or 50% (noninformative) validity was presented after T1 to cue the location of the upcoming T2. Relative to the no-cue baseline, T2 performance was enhanced by a valid cue, whether informative or noninformative, but only when the T1–T2 interval was within the usual AB period. Zhang et al. (2007b) presented a colored T2 in one of the three simultaneously presented RSVP streams of letters (cf. Peterson and Juola 2000). In half of the trials, a distractor between T1 and T2 had the same color as T2 or had a different color. This cue, which was always in the same stream as T2, thus predicted the T2 location with 100% validity. Both types of cues (with matching or nonmatching color) facilitated the report of T2, and these facilitatory effects were larger inside than outside the AB period.

On the other hand, a study employing an endogenous cueing technique with the attentional dwell-time paradigm found that the cueing effects appeared, in equal magnitudes, both inside and outside the AB (Zhang et al. 2008). Three experiments adopted three levels of cue validity (50, 80, and 100%) respectively. Relative to the no-cue condition, facilitatory as well as inhibitory cueing effects increased with increasing cue validity and with increasing the onset asynchrony between cue and target (CTOA). Importantly, however, these effects did not interact with the onset asynchrony between T1 and T2 (TOA).

Thus, these studies showed differential patterns of central and peripheral cueing in attentional blink, with the peripheral cueing showing a larger facilitatory effect inside than outside the AB, whereas the central cueing having comparable facilitatory effects inside and outside the AB. These results suggest that different types of spatial cueing may compensate for the AB deficit via different mechanisms. Specifically, following the boost and bounce model (Olivers and Meeter 2008), we assume that the dorsal frontoparietal network signals inhibitory control over sensory or perceptual processing during the AB. A salient stimulus, say, an exogenous cue, activates rTPJ, which breaks the inhibitory circuit via a transient signal and reorients attentional resources to input processing. If the subsequent target appears within a relatively brief period, it would benefit from such rTPJ activation. When the AB effect decays, the top–down inhibitory signal no longer plays a dominant role, and consequently T2 would benefit less from this rTPJ activation. A central cue, in contrast, would activate the dorsal network to signal top–down control and to facilitate target processing at the cued location. This positive signal would compensate for the inhibitory signal within the dorsal network. Presumably, target processing could benefit from this relatively constant compensation both inside and outside the AB, in contrast to the transient circuit-breaker effect induced by a peripheral cue.

As argued by Ghorashi et al. (2009), however, temporal integration between a target and its mask due to visual persistence (Di Lollo et al. 1994) might be a problem for exogenous cueing. Temporal integration is the phenomenon that a target and its mask are perceived as a single compound stimulus, because the blank interval between the target and the mask (i.e., inter-stimuli interval, ISI) is not long enough. In Experiment 4 of Nieuwenstein et al. (2005), the negligible exogenous cueing effect outside the AB might be due to a ceiling effect; that is, the T2 performance is degraded by the temporal integration and is limited to a certain level. To minimize the possible impact of temporal integration, Ghorashi et al. (2009), following Experiment 4 of Nieuwenstein et al. (2005), presented targets for a brief duration (13.3 ms). Between T2 and its mask, a critical ISI was inserted, with the duration of this ISI being varied dynamically and individually for each participant by parameter estimation sequential testing (PEST; Taylor and Creelman 1967), resulting in a level of 70% T2 accuracy. Using the dynamic ISI as the index of T2 performance, the authors found that exogenous cueing shortened the duration of the ISI, but this cueing effect did not vary with T1–T2 lag. The authors concluded that spatial cueing and the AB have independent mechanisms.

Nevertheless, the PEST procedure, which measures target performance in terms of ISI, is not commonly used for studying the AB and has little comparability with previous studies measuring target report accuracy. To collect convergent evidence for the dissociation between exogenous and endogenous cueing in the AB, the present study directly compares both types of cueing in more conventional RSVP paradigm, using essentially the same task, stimuli, and procedure. In two experiments, exogenous and endogenous cues were used respectively, and two colored targets were embedded in triple, simultaneous RSVP streams of letters (Fig. 1; cf. Peterson and Juola 2000).

Fig. 1

Schematic representation of RSVP for Experiments 1 (a) and 2 (b), respectively. In both experiments, a trial started with a display consisting of a central fixation and three frames indicating the positions of the upcoming RSVP streams. Subsequently, these frames were replaced by three RSVP streams while the central fixation remained present. The RSVP streams consisted of uppercase letters, with two colored letters serving as targets. Cues were inserted between the first and second targets. A square frame encircling a distractor letter served as the peripheral cue in Experiment 1 (see panel a) and an arrow replacing the central fixation served as the central cue in Experiment 2 (see panel b)

After T1 appeared in one of the streams, an exogenous cue (the sudden appearance of a frame at the location of one of the remaining streams; Experiment 1) or an endogenous cue (an arrow presented at the central fixation position; Experiment 2) was presented, in half of the trials, to indicate the location where T2 would be presented. The cue was 100% valid with regard to the target location. The SOA between T1 and T2 (TOA) could be short (i.e., 376 ms; T2 inside the AB period) or long (752 ms; T2 outside the AB period), and the SOA between the cue and T2 (CTOA) could also be short (188 ms) or long (282 ms). To avoid the problems of ceiling effect and temporal integration, a varying inter-stimulus interval (consisting of a blank screen) was inserted between consecutive frames in the RSVP display. While the SOA between frames was constant, the actual duration of each frame was determined for each participant by a pre-test, in which only one target was presented among a stream of distractors. The proper duration of each item was determined using a criterion of ~70% target accuracy (cf. Shulman et al. 2003).

Method

Participants

Forty right-handed students from Peking University were recruited in return for monetary compensation. Their ages ranged between 19 and 26 years, with a mean of 21 ± 2.4 years. All the participants reported having normal color vision and normal or corrected-to-normal eyesight. They were randomly assigned to either Experiment 1 or Experiment 2.

Design

Experiments 1 and 2 adopted essentially the same within-subject design. The onset asynchrony between targets (TOA) had 2 levels, 376 and 752 ms (i.e., T2 presented at lag 4 or 8 following T1). A cue was present in 50% of the trials. In these cued trials, the onset asynchrony between the cue and T2 (CTOA) was either short (188 ms) or long (282 ms); that is, the cue was presented in the second or the third frame before T2.

Apparatus and stimuli

The experiments were run in a sound attenuated, dimly illuminated room, with stimuli presented on a 17″ CRT monitor running at a resolution of 1,024 × 768 pixels and at a refresh rate of 85 Hz. Stimulus presentation and recording of the participants’ responses were controlled by Presentation software (http://nbs.neuro-bs.com/). Targets and distractors were all uppercase letters drawn from the alphabet and were presented in three simultaneous RSVP streams placed in an up equilateral triangular arrangement along an imaginary circle with a 1.4° diameter (cf. Peterson and Juola 2000, see Fig. 1). The background was black (with the RGB value of [0, 0, 0]). Each stream consisted of 21 items, with the restriction that no letters were repeated within a stream or appeared in more than one stream at the same frame. For half of the participants, the first target (T1) was in red ([130, 0, 0]), and the second target (T2) was in green ([0, 100, 0]), and this assignment was reversed for the other half of the participants. The letter acting as either T1 or T2 occurred only once in a trial. T1 appeared as the 7th–11th item in any stream with equal probability. T2 always appeared with equal probability in the other two streams. The distractor letters were in gray ([125, 125, 125]). The onset asynchrony between consecutive RSVP frames was 94 ms for all participants. The actual duration of a letter in each frame was determined for each participant in a pretest (cf. Shulman et al. 2003). In this pretest, the RSVP streams were similar to the formal experiment with the exception that only T1 was presented in each trial. In addition, T1’s duration was randomly varied to be either 24, 35, 47, 59, or 71 ms. The duration that led to a target accuracy of approximately 70% was used in the formal experiment. A group mean of 36-ms target duration was found (SD = 14) and was used in the formal experiment.

In each experiment, half of the trials had a cue inserted between T1 and T2. For Experiment 1, a gray ([125, 125, 125]) square frame, subtending 0.5° × 0.5°, encircled a distractor letter in the stream where T2 was to be presented. For Experiment 2, an arrow cue ([125, 125, 125]) with the length of about 0.4° substituted the central fixation. Both types of cues lasted for 94 ms and predicted the location of the upcoming T2 with 100% validity.

Procedures

By pressing the space bar, participants initiated each trial. A fixation display consisted of a central fixation cross and three gray outline frames. The frames marked the locations where the RSVP streams would occur. Whereas the frames were presented for 1,000 ms (see Fig. 1), the central fixation sign remained on the screen until the end of the trial (except when a central cue was presented). Subsequently, the three streams of letters were presented simultaneously at the three locations. Participants were instructed to keep fixating on the central sign and to avoid eye movements during a trial. They were asked to monitor the RSVP streams and identify the colored letters. At the end of each trial, participants reported the targets by typing them, in the order of appearance, on a computer keyboard. They were encouraged to guess whether they were unsure what the targets were.

In each experiment, there were 45 cued trials for each combination of CTOA and TOA and 90 uncued trials for each TOA, leading to a total of 360 trials. In each experiment, trials from different experimental conditions were equally distributed over five testing blocks, with 1- to 2-min breaks between blocks. In addition, participants received 40 practice trials.

Results

The accuracy of T1 report in both experiments was about 70%, with no difference between experimental conditions. Trials with T1 being incorrectly reported were excluded from further analyses. Data analyses were then conducted on the accuracy of T2 report (Fig. 2), conditionalized upon T1 being correctly identified (T2|T1).

Fig. 2

Results of T2|T1 in Experiments 1 (a) and 2 (b). The peripheral cueing effect was larger with a short CTOA than with a long CTOA, and was larger within than outside the AB period. The central cueing effect increased with increasing CTOAs but was constant across TOAs

Experiment 1: peripheral cueing

Mean accuracy scores for T2|T1 (Fig. 2a) were entered into a 2 (TOA: short vs. long) × 3 (cue condition: no-cue and cues with short or long CTOA) ANOVA. The main effect of TOA was significant, F(1, 19) = 254.98, p < .001, η2 = .931, with worse T2 performance for the short (57.3%) than for the long TOA (65.1%), showing a typical AB effect. The main effect of cue condition was also significant, F(2, 38) = 67.70, p < .001, η2 = .781, with the lowest T2 performance for the no-cue condition (56.4%), the highest for the cue condition with a short CTOA (65.4%), and intermediate for the cue condition with a long CTOA (61.7%). Importantly, the interaction between TOA and CTOA was significant, F(2, 38) = 35.83, p < .001, η2 = .653. Figure 2a shows that the cueing effects were evidently larger inside than outside the AB period. Indeed, relative to the no-cue condition, the cueing effect for the short CTOA was significantly larger inside (14.9%) than outside of the AB period (3.2%), t(19) = 8.27, p < .001. A similar but smaller cueing effect was found for the long CTOA (9.6% inside vs. 1.1% outside), t(19) = 6.38, p < .001.

Experiment 2: central cueing

T2 performance in Experiment 2 was analyzed in the same manner. The main effect of TOA was significant, F(1, 19) = 151.40, p < .001, η2 = .888, with T2 performance being lower for the short (53.8%) than for the long TOA condition (63.9%), showing a typical AB effect. The main effect of cue condition was significant, F(2, 38) = 84.80, p < .001, η2 = .817, with the best T2 performance for the cue condition with a long CTOA (63%), the lowest for the no-cue condition (55.9%), and intermediate for the cue condition with a short CTOA (57.8%). Pairwise comparisons showed that, relative to the no-cue condition, the facilitatory effects for the short- and long CTOA conditions were all significant, p < .01. Importantly, the interaction between TOA and CTOA was far from being significant, F(2, 38) < 1, η2 = .017, indicating that the central cue affected T2 report to the same extent regardless of T2 being inside or outside the AB period.

Discussion

By directly comparing exogenous and endogenous cueing in the same RSVP paradigm, this study demonstrated the dissociation between exogenous and endogenous cueing effects in the AB. Experiment 1 replicated previous studies (Nieuwenstein et al. 2005) by showing that the facilitatory exogenous cueing effect is larger inside than outside the AB period. This experiment also ruled out the possibility that a smaller cueing effect outside the AB is due to a ceiling effect of high T2 performance. Experiment 2 replicated our previous finding that the control of endogenous spatial attention is intact either inside or outside the AB period (Zhang et al. 2008), and generalized the noninteraction between endogenous cueing and the AB effect to the RSVP paradigm. Therefore, this study, convergent with previous studies, reveals differential spatial cueing effects for exogenous and endogenous cueing in RSVP.

These differential cueing effects are consistent with the general finding of differential time courses of peripheral and central cueing in the spatial domain. That is, exogenous cueing reaches its maximal facilitation within 150–200 ms and then decreases with increasing SOAs between the cue and the target. In contrast, the facilitatory effect of endogenous cueing builds up monotonously over time (Müller and Findlay 1988). It is clear from Fig. 2 that the exogenous cueing effect by the peripheral cue was larger when the SOA between the cue and T2 (i.e., CTOA) was short (188 ms) than when the CTOA was long (282 ms), while the endogenous cueing effect by the central cue showed equal facilitatory effects inside and outside the AB.

The consistent findings of a significant interaction between exogenous cueing and the AB effect and the absence of an interaction between endogenous cueing and the AB effect have important implications for our understanding of the mechanisms of the AB. Since the discovery of the AB phenomenon (Broadbent and Broadbent 1987; Raymond et al. 1992), a number of theoretical models have been introduced, trying to account for its underlying mechanism(s). In classical models (Chun 1997; Chun and Potter 1995; Duncan et al. 1994; Isaak et al. 1999; Jolicoeur 1998; Shapiro et al. 1994), the AB is attributed to the limited capacity of working-memory processing. That is, the processing and consolidation of T2 suffers from the limited resources depleted by T1 or from the bottleneck of post-perceptual processing of T1. However, the notion of a bottleneck or depletion of resources is incongruent with the present results. According to Jonides (1981), endogenous orienting elicited by a central cue is resource- and time-consuming; an increase in memory load or a decrease in the processing time of the cue would reduce the size of the cueing effect. If during the AB period there are only limited resources available for processing the central cue, we would expect the cueing effect to be smaller than when the cue is presented outside the AB period. This challenge to the traditional notion of resource depletion is consistent with the recent trend in which the cause of an AB is shifted from attentional resources to attentional control (e.g., Di Lollo et al. 2005; Martens and Wyble 2010; Olivers and Meeter 2008; Olivers et al. 2007; Taatgen et al. 2009). More recent computational models of the AB (e.g., Olivers and Meeter 2008; Taatgen et al. 2009) bypass the notion of limited capacity. However, these models make no explicit statements concerning the dissociable roles of exogenous and endogenous cueing in the AB.

To account for their finding of the absence of an interaction between spatial cueing and the AB, Ghorashi et al. (2009) proposed a dual pathway account in terms of a ventral and dorsal visual pathway. Specifically, they suggested that the spatial selection is carried out mainly along the dorsal pathway, while the target identification occurs mainly within the ventral pathway. This account, however, lacks detail. We go a step further by suggesting an account, mentioned in the Introduction, in terms of the dorsal and ventral attentional networks (Corbetta et al. 2008; Corbetta and Shulman 2002; Shulman et al. 2009).

In line with the boost and bounce model (Olivers and Meeter 2008), we assume that the dorsal network, which is also involved in endogenous attentional control, signals inhibitory control over input processing during the AB. Both exogenous and endogenous cueing could compensate for the inhibitory signal and facilitate T2 performance, but in different patterns and through different mechanisms.

In the case of exogenous cueing, it functions by activating the rTPJ, which serves as a circuit-breaker and transiently disrupts the top–down inhibitory signal from the dorsal network. However, the facilitatory effect is relatively small outside the AB, when the inhibitory top–down signal is no longer dominant.

In the case of endogenous cueing, a reallocating attention signal is built up within the dorsal network, directing attention toward the cued location, thereby facilitating T2 processing. As at least the spatial aspect of the top–down control of attention presumably remains unaltered during the AB period, endogenous cueing effects also remain intact both inside and outside the AB. Further studies, particularly neuroimaging studies, are needed to test these speculations.

The delayed attentional engagement model (Nieuwenstein 2006; Nieuwenstein et al. 2005) is also a good account for the interaction between the exogenous cueing and the AB. According to this model, the AB is due to delayed attentional engagement. That is, the attentional engagement on T2 is somehow delayed when the onset of T2 is temporally close to T1. During the AB, a pre-cue matching the attentional set toward T2 could initiate the attentional engagement, thereby attenuating the AB effect. When the onset of T2 is outside the AB period, the delay of attentional engagement is negligible. Thus, the effect of a pre-cue that minimizes any delay is also small. The endogenous cueing effect that remains equal inside and outside the AB, however, is hard to explain for this model. Alternatively, the boost and bounce model (Olivers and Meeter 2008), which attributes the AB to inhibitory top–down control, is clear in explaining how the AB occurs but does not seem to provide a ready explanation for the pattern of cueing effects described here, either. Compatible with the notion that the AB is due to the inhibitory top–down control in the boost and bounce model, our neural hypothesis further enriches and supplements the exploration of the neuropsychological mechanism(s) underlying the AB.

However, our finding of a larger exogenous cueing effect inside than outside the AB seems to be discrepant with the study of Olivers (2004), in which an RSVP stream was followed by a masked or unmasked cue display containing spatial cues indicating the possible location of the target in a subsequent visual search task. The author found that compared to the no-cue condition, cues accelerated search performance; but this improvement was smaller inside than outside the AB. The author thus concluded that the capability of using cues inside the AB was reduced. We do not have a ready explanation for this discrepancy, except that this may be attributable to differences in procedure, including the use of much longer TOAs (all outside the AB period) and multiple masked or unmasked cues in Olivers (2004). Importantly, when a single unmasked cue is presented, such as in Experiment 5 in Olivers (2004), comparable cueing effects are found inside and outside the AB, in line with the current study. If this is true, then the larger exogenous cueing effect inside the AB in this study can be interpreted with the notion that the cueing counteracts the inhibitory signals during the AB, as discussed above.

Since the processing of a central arrow cue has not been directly investigated in an AB context, the intact central cueing effect in the AB in this study might be accounted for in an alternative way. When a central symbol cue is inside the AB, its processing is more difficult than when it is outside the AB. But when successful, it may lead to a greater cueing benefit inside than outside the AB, for the reason that the endogenous orienting counteracts the inhibitory top–down signals during an AB. On average, the net benefit may be the same for both TOAs. In fact, it has been reported that when the processing of an endogenous cue is not deteriorated by the AB, e.g., a cue indicating TOA presented before each trial, endogenous attentional modulation of T2 performance is larger inside than outside the AB (Martens and Johnson 2005). To test the validity of this alterative account, more research is needed in which the stages of endogenous cueing are investigated separately.

In summary, although the dynamics of attentional selection in the spatial and the temporal domains have been extensively investigated along separate research lines, little is known about how spatial and temporal attentional selection might interact. The present study suggests that spatial exogenous and endogenous cueing may function via different mechanisms in interacting with attentional selection processes in the temporal domain.