Iconic memory requires attention.

Two experiments investigated whether attention plays a role in iconic memory, employing either a change detection paradigm (Experiment 1) or a partial-report paradigm (Experiment 2). In each experiment, attention was taxed during initial display presentation, focusing the manipulation on consolidation of information into iconic memory, prior to transfer into working memory. Observers were able to maintain high levels of performance (accuracy of change detection or categorization) even when concurrently performing an easy visual search task (low load). However, when the concurrent search was made difficult (high load), observers' performance dropped to almost chance levels, while search accuracy held at single-task levels. The effects of attentional load remained the same across paradigms. The results suggest that, without attention, participants consolidate in iconic memory only gross representations of the visual scene, information too impoverished for successful detection of perceptual change or categorization of features.

Introduction

The standard model of iconic memory considers it a pre-attentive store of visual information (Sperling, 1960). Here, we evaluated this model using two distinct paradigms. In the traditional paradigm, iconic memory of letters or digits is measured using partial-report (Sperling, 1960; Averbach and Coriell, 1961); more recent tests have expanded the stimulus set to include colors (Houtkamp and Braun, 2010), orientations (Houtkamp and Braun, 2010; Sergent et al., 2011) and shapes (Ruff et al., 2007). A new paradigm, the cued change detection task (Becker et al., 2000; Landman et al., 2003; Sligte et al., 2008, 2010), has proven especially effective in capturing the initial coding of information in iconic memory. The goal of the current study was to use both partial-report and cued change detection to address the question of whether attention mediates the formation of early iconic representations.

People frequently fail to notice change between two visual images, even when the change is relatively large. One explanation for such instances of “change blindness” (Rensink, 2002; Simons and Rensink, 2005) links the phenomenon to capacity limitations in visual scene perception (Rensink et al., 1997; O'Regan and Noë, 2001). On this account, limited attentional resources are available to encode the initial display in its entirety, leaving insufficient information to detect the image change. Consistent with this idea, the number of items people are able to monitor for change (i.e., approximately four; Luck and Vogel, 1997; Pashler, 1988; Rensink, 2000) closely matches estimates of capacity, in either attention (Pylyshyn and Storm, 1988; Rensink, 2000; Scholl, 2000) or working memory (Cowan, 2001). Further support is found in multi-unit recordings of monkey area V1 during change detection, which reveal a close correspondence between the neural activity elicited by the initial display and successful task performance (Landman et al., 2004). Likewise, in humans, the magnitude of event related potentials (ERPs) elicited by the initial display under different attentional manipulations predicts accuracy of change detection (Koivisto and Revonsuo, 2005).

A competing explanation is rooted in interpretations of the now classic experiments of George Sperling (Sperling, 1960) using a partial-report procedure, which revealed that, when post-cued, observers could accurately report any item from a brief multi-item visual display. Sperling concluded that the brief display was stored in an iconic memory, a pre-attentive, large capacity repository of detailed visual information, which required attention (post-cue) to transfer a subset of the information into a more durable form for inspection in working memory. Change blindness can thus be understood as the disruption of iconic memory (Becker et al., 2000; Landman et al., 2003). On this view, the intervening interval between images masks or overwrites the iconic memory of the initial image, leaving too sparse a representation to detect the image change.

Participants performing partial-report tasks often comment spontaneously that they seem to see all of the items in the display, but are unable to report them all due to forgetting. Such reflections imply that we experience more than we can report, and we report only what is attended. Several popular theories of consciousness have emerged from this view (Block, 1990; Lamme, 2003). For example, Lamme (Lamme, 2003) proposed that attention operates at a stage conceptually and operationally distinct from that required for conscious perception: we are conscious of many different visual inputs, however, attention allows only behavioral reporting, playing no role in whether visual information reaches a conscious state. Indeed, recent investigations using ERPs and magnetoenchephalography (MEGs) suggest that the earliest neural correlates of visual awareness, including the visual awareness negativity component (VAN; Koivisto and Revonsuo, 2003), function independently of spatial attention (Koivisto and Revonsuo, 2007; Wyart et al., 2011). In one study of change blindness that provided evidence consistent with this proposal, Landman et al. (Landman et al., 2003) cued participants during the intervening interval on the item changed between images. They found that the cue dramatically improved change detection for as long as 1.5 s after the initial display disappeared (see also Becker et al., 2000; Hollingworth, 2003). Reminiscent of Sperling's (Sperling, 1960) seminal results (see also Coltheart et al., 1974; Coltheart, 1980), the authors concluded that participants had awareness of roughly the entire visual scene, with attention acting to select from this rich, fleeting representation items to be stored in working memory.

Block (Block, 1990, 2005) advanced a similar proposal, drawing a distinction between “phenomenal” consciousness—detailed and possibly unlimited in capacity—and “access” consciousness—limited to the “consumer” information residing in the brain's systems of memory, reasoning, planning, and rational control of action. Possible neural correlates link phenomenal consciousness to posterior brain regions (e.g., visual cortex) and access consciousness to anterior regions (e.g., prefrontal cortex, see Lamme, 2006; Goodale, 2007). From this perspective, phenomenal consciousness simply “overflows” access consciousness. Although several extant experiments are consistent with Block's idea (Becker et al., 2000; Landman et al., 2003), in none was attention manipulated during presentation of the initial image. And although strong evidence exists for the role of attention in transferring information to working memory, it is unclear whether attention is required to create the initial iconic representation.

The purpose of the current study was to carry out this crucial manipulation in two separate experiments. In the first experiment we followed recent examples (e.g., Landman et al., 2003) in using a cued change detection task to measure participants' baseline ability to form an iconic memory of an initial visual display. This is an attractive paradigm because the comparison task enhances sensitivity to information encoded in the first display. In a second experiment we employed the traditional partial-report procedure (Sperling, 1960). In each case, we asked whether taxing attentional resources during the display affects iconic memory formation, thus representing an advance over previous studies using these paradigms. To manipulate attention we introduced a concurrent visual search task, which required participants to divide attention with the iconic memory task. Visual search was either easy, drawing relatively little on attentional resources, or hard, drawing heavily on resources. If the availability of attention is irrelevant in the formation of iconic memory representations, then visual search should leave change blindness or partial-report unaffected. However, if attention and consciousness are interrelated and perhaps even synonymous, as several authors have claimed (Posner, 1994; Prinz, 2000; O'Regan and Noë, 2001), then the formation of iconic memories (and, by implication, phenomenal consciousness) requires attention, which in the current study would cause instances of change blindness or report failure to multiply as the demands of the visual search task increase. Such results would demonstrate that, in the absence of attention, participants had impoverished iconic memory representations, which were insufficient for the successful transfer of target items to working memory.

Materials and methods

Experiment 1

Participants

Twenty-four undergraduate students (12 female, ages 18–32, M = 19.3) from the City College of the City University of New York participated in the experiment for course credit. All students reported normal or corrected-to-normal vision. None reported a history of neurologic illness, head trauma, or psychiatric illness. All participants were given written informed consent according to institutional guidelines prior to testing. The City College Human Research Review Committee approved the consent form, which contained detailed information regarding the purpose, risks, and benefits of participating.

Apparatus and stimuli

The stimuli appeared on a 16″ CRT monitor (Sony Model G220) with refresh rate of 100 Hz.

Stimuli for the visual search were a set of eight small white circles, each subtending 0.65°, presented within a gray disk, subtending 3.4°, superimposed on textured background (Figure 1). The positions of the circles were selected randomly on each trial from a set of 10 predetermined positions. In easy visual search, participants were asked to find a pegged circle (i.e., circle with attached white bar subtending 0.16° × 0.33°) among smooth distractor circles (i.e., no white bar), (Figure 1). Conversely, in hard visual search, participants were asked to find a smooth target circle among pegged distractor circles (attached white bar subtending 0.16° × 0.16°). In each condition, targets appeared with probability 0.5.

Figure 1

An example of a trial sequence in Experiment 1. Here, visual search is easy and the change detection task includes a change. Rectangles in vertical or horizontal orientation were composed of texture identical to background (inset).

Participants were asked to detect a change between two displays in the orientation (vertical to horizontal or vice versa) of one of eight rectangles. The rectangles, each measuring 2.3° × 1.3° and offset from the background in textural orientation (Figure 1, inset), appeared in an imaginary circle (diameter = 5.1°) at equally spaced positions. To prevent grouping, the positions were randomly jittered ±0.8° in radial direction.

Procedure

Participants sat in a dimly lit, sound attenuated testing chamber at a viewing distance of 57 cm. Participants performed five different tasks over the course of the experimental session, three single-tasks (200 trials each) and two dual tasks (400 trials each). The single-tasks were: (1) change detection alone, (2) easy visual search alone and (3) hard visual search alone. The dual tasks were (4) change detection with easy visual search and, (5) change detection with hard visual search. Each of the five different tasks included the display of circles (used in visual search) and the display of rectangles (used in change detection). This tack enabled us to equate the amount and type of visual stimulation across task conditions. Tasks were completed separately and presented in blocks of 50 trials separated by short breaks. All participants first completed a set of the three single-tasks, followed by the two dual tasks. Order of tasks within each set and response assignments were counterbalanced.

At trial onset a fixation cross subtending 0.2° appeared for 1000 ms, followed immediately by a display of eight white circles, which remained displayed for 250 ms. 50 ms after circle onset, eight randomly oriented rectangles appeared for 200 ms (see Figure 1). In visual search alone, a blank screen was then shown with the question “Did you see the target?” until participants made non-speeded “yes” or “no” responses to the target circles, ignoring the rectangles.

In change detection alone, a cue—a yellow line subtending 1.9° × 0.13° at an eccentricity of 0.9°—appeared for 100 ms at one of the eight positions previously occupied by a rectangle. Cue location was determined randomly on each trial. The screen was then made blank for 900 ms, followed by a second display of eight rectangles, which appeared for 250 ms, with the orientation of the rectangle at the cued location changing orientation on half of the trials. All other rectangles were identical across the two displays. A subsequent response prompt “Change?” signaled participants to decide whether the rectangle in the cued location had changed orientation. Trials were response terminated.

In the dual tasks, participants were required to attend to both circles and rectangles, with the relevant task (search or change detection) indicated on each trial only after the onset of the first display (see Figure 1). Participants were instructed to maintain high performance on visual search, while simultaneously attending to orientation change. The search prompt appeared with probability 0.6 to ensure that performance on the search task was maintained.

Experiment 2

The results of Experiment 1 were extremely robust: each of the participants showed better performance in change detection during low load. Consequently, we elected to test relatively fewer participants in Experiment 2. Six undergraduate students (3 female, ages 20–33, M = 24.7) from the City College of the City University of New York participated in the experiment. All students were neurologically normal individuals reporting normal or corrected-to-normal vision. Participants read and signed a consent form approved by the City College IRB. None participated in Experiment 1.

Apparatus, stimuli, and procedure

The apparatus matched Experiment 1. In both single and dual tasks, a checkerboard pattern mask presented centrally and exactly overlapping the search array, appeared for 50 ms immediately after the display of circles and rectangles. In visual search alone, the response prompt appeared immediately after the pattern mask. The change detection task from Experiment 1 was replaced with a partial-report task. In partial-report alone, the pattern mask was followed by a yellow cue that appeared for 200 ms at one of the eight positions previously occupied by a rectangle. A subsequent response cue “V or H?” prompted participants to decide whether the rectangle was oriented vertically or horizontally. In the dual tasks, participants were again required to attend to both circles and rectangles, with the relevant task (search or partial-report) indicated on each trial immediately after the offset of the pattern mask. All other aspects mimicked Experiment 1.

Results

To examine whether attention is required for iconic memory, in our first experiment we paired a change detection task with a visual search task. Participants were asked to detect change in the orientation (vertical or horizontal) of a visual object (rectangle). Attentional demands from the dual task were either made easy—simultaneously searching for a readily seen target—or made hard—searching for a less visible target. We were careful to assess each participant's baseline ability to detect change or search targets by measuring performance in each task alone before asking participants to perform dual tasks. In this way, we could gauge any decrements in change detection performance from adding an easy dual task (easy search), and then any further decrements that accompany a difficult dual task (hard search). Change detection should be unaffected by the attentional load manipulation if attention is uninvolved in the formation of iconic memories.

We first performed a series of manipulation checks. A comparison of the visual search tasks performed alone indicated that we were successful in manipulating attentional load: accuracy during easy visual search (M = 98.88%, SD = 2.59%) was significantly higher than during hard visual search (M = 69.40%, SD = 5.54%), t(23) = 21.42, p < 0.001. We also found that participants were able to maintain high accuracy when performing change detection alone (M = 87.85%, SD = 5.90%), a result consistent with previous studies indicating improved performance when the change location is cued (Becker et al., 2000; Landman et al., 2003). Finally, we found that participants were able to maintain search accuracy under dual task conditions: as shown in Figure 2A, there was no difference in visual search performance between single and dual conditions in either easy, t(23) = 0.19, p = 0.851, or hard, t(23) = 1.35, p = 0.190, search tasks.

Figure 2

(A) Accuracy for visual search for single and dual tasks as a function of search type in Experiment 1. (B) Change detection accuracy in dual tasks as a function of attentional load in Experiment 1.

The primary analyses involved evaluating the effects of attentional load on change detection under dual task conditions. Figure 2B depicts performance in change detection as a function of load. We found that the accuracy of detecting change fell from single-task performance (M = 87.85%, SD = 5.90%) when paired with easy visual search (M = 76.35%, SD = 7.16%), t(23) = 8.83, p < 0.001. Yet the more appropriate comparison is between the two dual task conditions (low vs. high load), as any dual task condition is more demanding than a single-task condition (e.g., in requiring that two sets of instructions be held in memory). Here, change detection dropped to near-chance levels when paired with hard visual search (M = 59.83%, SD = 5.43%). The difference in change detection when paired with easy or hard search was highly significant, t(23) = 12.81, p < 0.001. Importantly, each of the individual participants revealed an identical pattern of performance. These results are in line with the view that the formation of an iconic memory of object orientation is impaired when attentional resources are reduced.

Although the cued change detection task (Experiment 1) is now a common technique to probe iconic memory formation (Becker et al., 2000; Landman et al., 2003; Sligte et al., 2008, 2010) it is unlike the partial-report task Sperling (1960) used in his seminal studies of iconic memory because the former requires a comparison between two images presented sequentially. This key procedural difference invites another interpretation of our results: perhaps attentional load undermined the process of comparing the first and second displays, leaving untouched the iconic image of the first display (see Simons et al., 2002; Mitroff et al., 2004). To be sure, a previous study (Landman et al., 2003), which used our identical paradigm, has already ruled this explanation out; it also is highly unlikely that attentional resources could interfere with comparison processes 1.5 s later. Nevertheless, we felt compelled in Experiment 2 to confront this alternative directly. We also considered here the possibility that the cue used to query change was relatively ineffective under high load, even though the cue's very appearance, which masked the search array, made continued search irrelevant. Either of these alternatives is consistent with the findings of the Experiment 1.

The purpose of Experiment 2 was to address these alternative explanations using a reporting procedure closer in spirit to that employed by Sperling (1960). Here, we eliminated the second display entirely, instead asking participants merely to report the orientation of the cued rectangle in the first (now, the only) display. To ensure that participants were able to use the cue effectively we doubled its duration from 100 ms to 200 ms. Finally, to ensure that the iconic image formed to the rectangles for partial-report did not also include the search items, a pattern mask spatially overlapping only the search items appeared immediately after the display. Thus, participants could not continue searching the array once the reporting cue appeared. If attentional load disrupts partial-report in this modified Sperling paradigm, it will have provided especially strong and convergent evidence that iconic memory formation requires attention.

Presentation of the pattern mask had no discernible effect on visual search performance. As in the previous experiment, participants were significantly more accurate when search was easy (M = 99.42%, SD = 0.49%) than when it was hard (M = 68.33%, SD = 5.95%), t(5) = 13.17, p < 0.001. Once again we found that, search performance was virtually identical between single and dual tasks (Figure 3A) whether search was easy, t(5) = 0.94, p = 0.393, or hard, t(5) = −0.51, p = 0.635. We also again found that the accuracy of performance (here, categorization using partial-report) fell from single-task levels (M = 81.92%, SD = 6.78%) when coupled with easy visual search (M = 70.08%, SD = 10.88%), t(5) = 2.77, p = 0.039, but was indistinguishable from chance responding when coupled with hard visual search (M = 52.70%, SD = 4.41%), a difference that was highly significant t(5) = 5.26, p = 0.003 (Figure 3B). Moreover, as in Experiment 1, each of the individual participants evinced the same pattern of performance. In replicating the results of our previous experiment within a traditional partial-report paradigm, Experiment 2 effectively rules out the two chief alternative explanations of attentional load on change detection: disruption to comparison processes and ineffectiveness of reporting cue. The current results instead support the view that attention is required in the formation of iconic memories, an explanation that captures parsimoniously the principal findings of both experiments.

Figure 3

(A) Accuracy for visual search for single and dual tasks as a function of search type in Experiment 2. (B) Partial-report accuracy in dual tasks as a function of attentional load in Experiment 2.

Discussion

Two experiments investigated the role of attention in forming iconic representations. In each, attention was taxed during initial display presentation, focusing the manipulation on the consolidation of information into iconic memory, prior to transfer into working memory. Observers were able to maintain high levels of accuracy even when concurrently performing an easy visual search task (low attentional load). However, when the concurrent search was made difficult (high attentional load), observers' performance dropped to almost chance levels, whereas search accuracy nevertheless remained at single-task levels. Moreover, the effects of attentional load were essentially the same whether observers were asked to detect change (Experiment 1) or to categorize features using traditional partial-report (Experiment 2). These results suggest that, without attention, participants consolidate in iconic memory only gross representations of the visual scene.

Theoretical implications

Several recent theories of consciousness embrace a conceptual division between access consciousness and phenomenal consciousness (Block, 1990, 2005; Lamme, 2003). Access consciousness involves reportable experience, which is directly accessible, requires attention, and is severely limited in capacity. By contrast, phenomenal consciousness is thought to be pre-attentive, not directly accessible, and large, possibly unlimited, in capacity. Lamme (Lamme, 2003), for example, claims that attention is necessary only for experiential reporting (i.e., access consciousness), and so is dispensable for phenomenal consciousness. The classic experiments investigating iconic memory (Sperling, 1960), as well as more recent behavioral (Coltheart et al., 1974; Coltheart, 1980; Becker et al., 2000; Landman et al., 2003) and electrophysiological (Koivisto and Revonsuo, 2007; Wyart et al., 2011) demonstrations, undergird this theoretical perspective in suggesting a separation between what humans experience and what they are able to report. Yet the results of the present study are consistent with the opposing view that phenomenal consciousness, like access consciousness, depends heavily on attention.

Recent evidence from neuroimaging bears on these views. Houtkamp and Brown (Houtkamp and Braun, 2010) measured the magnitude of the BOLD response in primary visual cortex (V1) and extrastriate cortex (V2) to a peripheral task (color or orientation discrimination) under dual task conditions. They concluded that attention acts to amplify already-formed iconic traces. However, the investigators did not manipulate attention to the central task, leaving unanswered the question of whether attention modulates iconic trace formation. Sergent and colleagues (Sergent et al., 2011) found that the magnitude of neural activation in visual areas V1 and V2 to the initial stimulus display predicted the accuracy of partial-report. These results are consistent with a possible role for attention in forming the display's iconic trace, as are the recent results of Koivisto and colleagues showing modulation by attention of early ERP correlates of consciousness in the occipital region (Koivisto et al., 2009). As in our study, both of these studies employed strong attentional manipulations. We believe that the inconsistency in demonstrating the role of attention on neural correlates of consciousness is due in part to the varying effectiveness of the attentional manipulations used across studies.

We recognize that our conclusions concerning the role of attention in phenomenal consciousness rests on the assumption that iconic memory is a form of phenomenal consciousness. Still, to the best of our knowledge, ours is the first study to evaluate rigorously the role of attention in phenomenal consciousness by manipulating attentional load during the time of the iconic image formation (cf. Braun and Julesz, 1998; Li et al., 2002). We found in both experiments that iconic image formation was severely disrupted by an increase in attentional load, indexed by a significant drop in change detection or categorization. In fitting the view that phenomenal consciousness relies heavily upon attention, our results suggest an intimate relationship between consciousness and attention (see also Posner, 1994; Merikle and Joordens, 1997; Prinz, 2000; O'Regan and Noë, 2001). Indeed, our results essentially undo the conceptual distinction between phenomenal and access consciousness (cf. Kouider et al., 2010). As an alternative, we propose that consciousness is best conceived as a binary phenomenon, with the contents of consciousness varying along a continuous scale. Here, consciousness always requires attention, though we leave open the possibility that attention and consciousness are fully distinct phenomena (Koch and Tsuchiya, 2007).

Alternative explanations

One might advance other interpretations of our findings. Perhaps attention is unnecessary to actually form iconic memories, but the diversion of attention accelerates subsequent memory decay or forgetting. In this case, hard visual search might be said to have a more pronounced effect than easy visual search on the rate of decay, leading to relatively poorer change detection under high attentional load. Of course, our post-cue was immediate, leaving little room for iconic decay. Moreover, investigators who have tackled the question of iconic decay directly (e.g., Landman et al., 2003) have shown that the rate of decay in iconic memory is unaffected by attentional focus, enabling us to rule this explanation out.

In the current study, we varied across experiments the delay after the initial display in prompting a response, from 900 ms in Experiment 1 to 200 ms in Experiment 2. This manipulation offers an additional means to test the hypothesis of decay in iconic memory, which implies that the information available from the iconic store falls off with delay. We found, however, that the pattern of performance was the same whether the response prompt followed the display by 200 ms or 900 ms, demonstrating that attentional load influenced the intercept (i.e., memory formation) rather than the slope (i.e., memory decay).

Another possibility is that attentional load differentially interfered with the process of transfer after the formation of the iconic image. Yet previous studies (Landman et al., 2003), as well as our own pilot experiments, have demonstrated that iconic memory representations in this paradigm persist for at least 500–1000 ms after the disappearance of display. We, therefore, think it highly unlikely that the attentional load manipulation affected the transfer of information.

Perhaps low attentional load afforded participants an opportunity to foveate peripheral stimuli sequentially, yielding relatively good task performance (change detection or partial-report) in this condition. However, two points argue against differential eye movements as an explanation of good performance during easy search. First, our design employed presentation times too brief to permit the sequential foveation of stimuli. Second, even if possible, sequential foveation here would actually have produced worse performance than constant fixation. The reason is that in both experiments the target rectangle was not known in advance, but instead required participants to wait until the cue appeared after the rectangle display terminated (and, in Experiment 1, for a second display to appear). Here, the optimal strategy is to fixate at the center of the display, equidistant from all rectangles, to form an iconic image of the set of rectangles. By contrast, looking directly at any single rectangle would, on average, result in relatively poor performance. Indeed, a recent ERP study found that during successful change detection attention is spread across the display, whereas during change blindness attention tends to be focused on specific locations (Koivisto and Revonsuo, 2005). Thus, our finding of good performance during easy visual search rules out differential eye movements as a potential confound in this study.

The current study considered only the effects of spatial attention on performance in iconic memory tasks. Yet experiments investigating the effects of masking (Koivisto et al., 2005) have demonstrated dissociation between consciousness and attention for objects occupying a central spatial position. Thus, it is possible that the effects of object attention on the formation of iconic memories are distinct from those of spatial attention. Future experiments could investigate this possibility by examining the effects of attentional load when the objects of search and change or partial-report are spatially superimposed.

Conclusion

We have demonstrated in two experiments that the formation of iconic memory, which traditionally is considered pre-attentive, is disrupted when attention is diverted. We showed this in experiments that required either detection of perceptual change or categorization of features. Our data thus provide evidence that phenomenal consciousness, although conceptually and theoretically distinct from attention, still requires attention. The role of attention, therefore, goes beyond the transfer from iconic to working memory stores, suggesting that phenomenal consciousness is indistinct from access consciousness.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.