Dissociable roles of different types of working memory load in visual detection.

We contrasted the effects of different types of working memory (WM) load on detection. Considering the sensory-recruitment hypothesis of visual short-term memory (VSTM) within load theory (e.g., Lavie, 2010) led us to predict that VSTM load would reduce visual-representation capacity, thus leading to reduced detection sensitivity during maintenance, whereas load on WM cognitive control processes would reduce priority-based control, thus leading to enhanced detection sensitivity for a low-priority stimulus. During the retention interval of a WM task, participants performed a visual-search task while also asked to detect a masked stimulus in the periphery. Loading WM cognitive control processes (with the demand to maintain a random digit order [vs. fixed in conditions of low load]) led to enhanced detection sensitivity. In contrast, loading VSTM (with the demand to maintain the color and positions of six squares [vs. one in conditions of low load]) reduced detection sensitivity, an effect comparable with that found for manipulating perceptual load in the search task. The results confirmed our predictions and established a new functional dissociation between the roles of different types of WM load in the fundamental visual perception process of detection. 2013 APA, all rights reserved

Centuries of magic acts and more recent experimental phenomena of “inattentional blindness” demonstrate the limits of visual perception. Unattended stimuli go unnoticed even when people wish to see them (e.g., when intending to “catch” the magician’s trick). Such perceptual failures are more likely when the attended task involves high perceptual load (e.g., Cartwright-Finch & Lavie, 2007; Lavie, 2005, 2010, for reviews). In contrast, when cognitive control functions such as working memory (WM) are loaded during task performance, irrelevant stimuli often intrude, despite the attempt to ignore them (e.g., De Fockert, Rees, Frith, & Lavie, 2001; Lavie & De Fockert, 2005). These seemingly paradoxical results, whereby different types of load have opposite effects on perception, are explained within load theory of attention and cognitive control (e.g., Lavie, Hirst, De Fockert, & Viding, 2004). When task processing involves high perceptual load (e.g., many different stimuli or complex perceptual discriminations, e.g., Lavie, 1995) that exhausts the available capacity, little capacity is left to perceive task-irrelevant stimuli. High perceptual load thus leads both to perceptual failures and more efficient rejection of irrelevant distractors. In contrast, high WM load that taxes cognitive control over task performance leads to the opposite effect; reduced capacity for control in accordance with the current task priorities leads to enhanced processing of low-priority and potentially distracting stimuli.

WM, however, is not unitary, and a major distinction has been drawn between short-term maintenance and cognitive-control WM processes (e.g., Baddeley, 1992; Smith & Jonides, 1999) and their differential recruitment of posterior (maintenance) versus anterior (control) prefrontal cortex regions (Cohen et al., 1997; Fiez et al., 1996; Smith, Jonides, Marshuetz, & Koeppe, 1998). Moreover, recent research indicates that visual maintenance involves the same sensory visual cortices as those involved in perception (Harrison & Tong, 2009; Malecki, Stallforth, Heipertz, Lavie, & Duzel, 2009; Munneke, Heslenfeld, & Theeuwes, 2010; Pasternak & Greenlee, 2005, for recent demonstrations). But the consequences for perception when these different WM functions are loaded have not, as yet, been drawn. Considering this research within the load theory framework led us to predict a novel dissociation between the effects of loading different WM processes on perception. Specifically, we predicted that during task performance, loading WM cognitive control processes would lead to enhanced perception for lower priority stimuli (due to reduced priority-based control). In contrast, loading visual short-term memory (VSTM) maintenance will increase the demand on sensory representation capacity and thus result in reduced perception of low-priority stimuli—an effect similar to that of perceptual load. We tested these predictions for the fundamental perceptual process of visual detection, using signal detection analysis to address perceptual sensitivity per se, independently from any effects on response bias. Although perceptual load is known to affect detection sensitivity (e.g., Carmel, Thorne, Rees & Lavie, 2011; Macdonald & Lavie, 2008), the effects of the different types of WM load on detection sensitivity are yet to be established.

Participants performed a visual search task during the delay period of a short-term memory task and were requested to also detect an additional meaningless shape in the periphery. The memory and search tasks were set as the participants’ primary tasks, and performance accuracy in the memory and search task (but not the detection assignment) was emphasized. Detection was set as the lowest priority task.

The level and type of load involved in the tasks was varied. In Experiment 1, participants performed a VSTM task requiring recognition of color and location for a colored-squares array. Perceptual load or VSTM load were varied through set size of the search or memory sample, respectively. Both manipulations were expected to draw on visual representation capacity and thus lead to reduced detection sensitivity.

In Experiment 2, participants performed the “successor naming” task (requiring recall of digit order) and load was manipulated by presenting the memory sample digits in either fixed (low load) or random order (high load). Recall of random-ordered digit sequence is known to recruit on cognitive control (e.g., Baddeley, Emslie, Kolodny, & Duncan, 1998; D’Esposito, Postle, Ballard, & Lease, 1999; Marshuetz, 2005),1 and loading cognitive control in this way is known to result in increased distraction (e.g., De Fockert et al., 2001; Lavie et al., 2004), but the effects in detection are unknown. We predicted that reduced control over task-processing priorities under high cognitive control load would lead to enhanced detection sensitivity for the low priority detection stimulus.

Note that although cognitive control processes may also be involved in the VSTM task employed in Experiment 1, our manipulation of VSTM load through the visual sample set size increases demands on visual maintenance, whereas the demand on cognitive control remains constant across the conditions of different visual set size. In contrast, the demand to maintain a random (vs. fixed) ordered digit sequence in Experiment 2 increases demands on cognitive control while leaving demands on visual maintenance constant between the conditions of low and high cognitive control load.

Experiment 1

Method

Participants

Participants in all experiments reported were recruited from the University College London subject pool. All procedures were approved by the ethics committee of University College London. All participants gave informed written consent and had normal or corrected-to-normal vision and no color blindness. Eight participants (aged 19 to 35 years) took part in Experiment 1a and 16 participants (aged 18 to 32 years) participated in Experiment 1b.

Stimuli and procedure

A viewing distance of 60 cm was maintained with a chin rest. Figure 1 shows the stimuli and trial sequence. Load conditions were blocked. Following a 48-trial practice block, participants completed four blocks of 72 trials each in an alternating low/high or high/low (counterbalanced across participants) order. In Experiment 1a, VSTM load was always low, whereas perceptual load was varied. In Experiment 1b, perceptual load was always low, whereas VSTM load was varied (see Figure 1).2

Figure 1

An example trial in Experiment 1b high VSTM load condition with a detection stimulus present. The correct memory probe response here is “same.” For the memory set, colored squares (0.38° × 0.38°) were chosen from black, blue, cyan, green, magenta, pink, red, white, and yellow (represented with texture here) and placed in random on a 3 × 3 grid (1.38° × 1.38°). The search letters (0.6° × 0.4°) were black and presented in a circle (1.7° in radius). Participants searched for an “X” or “N” target letter among either nontarget letters (F, H, K, Z, M, high load) or black dots (low load; shown here) and pressed 0 for “X” and 2 for “N” using the numerical keypad. Participants were asked to also detect a small (0.3° × 0.3°) gray shape presented at 5.4° eccentricity on 50% of the trials randomly selected (shown on bottom left corner of the search-task display) and press “S” on the keyboard for “present” or “A” for “absent” upon presentation of a question mark. The memory probe was always presented in one of the occupied memory-set positions. Participants pressed “S” to indicate a match to the memory set in color and position or “A” to indicate “different.” Stimuli are not drawn to scale.

As customary in signal detection and memory paradigms, participants were instructed to perform as accurately as possible, and performance was not speeded.

Results

The results of the search and memory tasks are presented in Table 1.3 In Experiment 1a, the search task accuracy was significantly reduced with higher perceptual load, t(7) = 5.51, p = .001, d = 1.49. Thus, perceptual load was manipulated effectively. VSTM task accuracy did not differ between the two perceptual load conditions, t(7) = 0.85, p = .43, d = .36. Importantly, detection sensitivity was significantly reduced in the high (vs. low) load condition, t(7) = 2.38, p = .049, d = .86 (see Figure 2). There was no effect of load on response bias (β; p = .10). These results replicate Macdonald and Lavie’s (2008) findings within our new interleaved memory and perception tasks paradigm.

Table 1

Results in the Search and Memory Tasks of Experiments 1 and 2

Figure 2

The results of Experiments 1 and 2. Mean detection sensitivity (d′) is plotted as a function of the level and type of load. Error bars represent + 1 SEM.

In all experiments, analyses of the detection task performance were made on trials with correct performance on the search and memory tasks, and analyses of the search task were made on trials with correct performance on the memory task. Including the incorrect trials in the analyses did not change the results pattern or significance.

As shown in Table 1, in Experiment 1b, VSTM task accuracy was significantly reduced with higher VSTM load, t(15) = 5.66, p = .006, d = 1.47. Thus, VSTM load was manipulated effectively. Search task accuracy did not differ significantly between the low and high VSTM load conditions (t < 1).4 Importantly, detection sensitivity was significantly reduced in the high (vs. low) VSTM load condition, t(15) = 3.29, p = .005, d = .62, as we predicted (see Figure 2). There was no effect of VSTM load on response bias (β; p = .99).

In the present study, search task performance was near ceiling (95% accuracy on average; see Table 1); thus, it was less sensitive than the detection task to reveal any effect of VSTM load. In other words, any available capacity remaining for sensory perception in conditions of high VSTM load is likely to have been sufficient for the search task pop-out detection but not for the lowest priority and more demanding masked-stimulus detection.

Moreover, Figure 3 shows that the reduction in detection sensitivity with higher load could be predicted from the extent to which VSTM was occupied under high load. A correlation between the increase in memory capacity estimates (Cowan’s K)5 and the reduction in d′ with high load indicated that individuals who held more items in memory under high (vs. low) VSTM load also showed a greater reduction of detection sensitivity with higher load (b = .28), t(13) = 2.74, p = .02. The finding of a trade-off between detection sensitivity and the extent to which VSTM capacity was filled under load strongly suggests shared capacity between VSTM and perception. Finally, a between-experiment comparison revealed no difference between the effects of VSTM load and perceptual load on detection, F(1, 22) = 2.12, p = .16, η2 = .09.

Figure 3

Scatterplot showing the effects of VSTM load on detection sensitivity (high-load d′ minus low-load d′) and on memory capacity estimates (calculated using Cowan’s K [K = (hit rate − 0.5 + correct rejection − 0.5) * N], where K is the capacity estimate and N is the number of items presented in the memory set). The line represents the best linear fit. The increase in K with high load explained a significant proportion of variance in the reduction of detection sensitivity scores (R2 = .37), F(1, 13) = 7.51, p = .02.

Cowan’s K = (hit rate − 0.5 + correct rejection − 0.5) * N, where K is the capacity estimate and N is the number of items presented in the memory set (Cowan et al., 2005). The data of one outlier participant with a detection sensitivity reduction of more than 2 SDs under high VSTM load were excluded from the linear regression analysis.

Overall, Experiment 1’s findings confirm our prediction that the effects of VSTM maintenance load on detection would be akin to those of perceptual load.

Experiment 2

Ten new participants (aged 19 to 38 years) were recruited for Experiment 2.

The search and detection task of Experiment 1b was now used within the delay of a “successor naming” WM task. A memory set of nine black digits (1–9; 0.7°× 0.5 ° each) was presented in either fixed numerical order for 500 ms (low-load condition) or random order for 2000 ms (high-load condition). Following the detection response, a single memory probe digit (equally likely to be any of the first eight digits of the trial’s set) was presented at fixation (until response) and participants indicated which digit followed the memory probe digit in the memory sample.

The results of the search and memory tasks are presented in Table 1. WM task accuracy was significantly lower in the high (vs. low) load conditions, t(9) = 6.16, p < .001, d = 1.59, confirming the efficacy of this load manipulation. Search task accuracy was the same across the WM load conditions, t < 1. Importantly, detection sensitivity was now significantly higher in the high (vs. low) WM load condition, t(9) = 2.53, p = .03, d = .53 (see Figure 2). Load had no effect on response bias (β; p = .15).

The opposite effects on detection of WM load (Experiment 2) versus VSTM load and perceptual load (Experiment 1) were confirmed in mixed-model ANOVAs, with the between-subjects factor of load type (WM vs. VSTM; or WM vs. perceptual load) and the within-subject factor of load level (low, high), which revealed significant interactions (for WM vs. VSTM, F[1, 24] = 17, p < .001, η2 = .42; for WM vs. perceptual load, F[1, 16] = 12.24, p = .003, η2 = .43).

General Discussion

The present results demonstrate opposite effects of different types of WM load on visual perception. During search task performance, detection sensitivity for a low-priority search-unrelated stimulus in the periphery was reduced with high visual maintenance load (an effect equivalent to that of perceptual load), but was enhanced with high load on WM cognitive control processes. These findings support our predictions that loading the different WM maintenance versus control functions would have opposite effects on perception, and demonstrate, for the first time, critical roles for these WM functions in detection sensitivity. As we briefly review in this section, previous research has thus far established the effects of WM cognitive control load on distractor processing (measured with behavioral interference effects, neural responses and explicit recognition, e.g., Carmel, Fairnie, & Lavie, 2012; Rissman, Gazzaley, & D’Esposito, 2009). This research cannot inform about the fundamental process of detection sensitivity, and although a recent report that WM cognitive control load reduces the rates of inattentional blindness (De Fockert & Bremner, 2011) is consistent with our findings, inattentional blindness reports are open to alternative accounts in terms of response bias (as we also detail later on in this discussion).

Moreover, some of the previous findings could be attributed to a general increase in task difficulty, whereas our findings that different types of load have opposite effects on stimulus detection cannot be explained in such terms. All high-load conditions were more difficult, but the effect on detection sensitivity critically depended on the type of process loaded. This is important because general effects of task difficulty can explain some of the previous reports that inattentional blindness rates are increased with higher load, irrespective of whether “executive” WM or VSTM load was manipulated (e.g., Fougnie, & Marois, 2007; Todd, Fougnie & Marois, 2005). One likely general effect of increased task difficulty is the adaptation of a more conservative criterion for reports about unexpected, task-irrelevant stimuli. Indeed, the awareness measures used in these previous studies were based on a single subjective response (noticed or not) per participant. Therefore, increased rates of “blindness” reports under higher load may reflect elevation of the response criterion. In contrast, the present effects were established for an expected stimulus with a criterion-free measure of detection sensitivity, and the results showed no effects of load on response bias. Our conclusions are thus immune to these alternative interpretations.

Our findings extend load theory to now accommodate the effects of loading different WM processes of maintenance and cognitive control, and demonstrate a new functional dissociation between the roles of these WM processes in the fundamental perception process of visual detection. Previous evidence for a dissociation of short-term storage and cognitive control WM processes has typically been based on findings that these different WM processes are mediated by different (posterior vs. anterior) brain areas, and that neuropsychological patients show selective deficits (e.g., D’Esposito & Postle, 2000; Smith & Jonides, 1999). Our demonstration is the first to show that loading these different WM processes has opposite effects on the very same measure of perceptual detection sensitivity.

Our proposed dissociation can accommodate previously disparate lines of evidence and resolve apparent discrepancies in the previous work. High WM load has been found to increase distractor processing in some previous studies (e.g., De Fockert et al., 2001; Lavie & De Fockert, 2005; Lavie et al., 2004; Rissman et al., 2009). These findings were obtained with well-established WM tasks that load cognitive control similarly to our Experiment 2. Other studies reported that WM load has led to reduced distractor processing (Bollinger, Masangkay, Zanto, & Gazzaley, 2009; Rose, Schmid, Winzen, Sommer, & Buchel, 2005; Sreenivasan & Jha, 2007). The different effects of WM load on distractor processing appear discrepant on a unitary view of WM. But our proposed framework can accommodate all these findings, as the findings of reduced distractor processing were in fact obtained in tasks that loaded visual maintenance. Bollinger et al. (2009), Rose et al. (2005), and Sreenivasan and Jha (2007) have all compared visual distractor processing (e.g., FFA- and N170-evoked potentials related to distractor faces) while the subjects performed a VSTM task of either low load (maintaining one visual image, e.g., of a place or a colored square) or high load (maintaining two or more such images). These findings are consistent with the VSTM load effects we report. In support of this claim, we have recently found that high VSTM load reduces distractor interference effects measured with the response competition paradigm, whereas WM cognitive control load increases them (King, 2009; Konstantinou, Beal, King & Lavie, 2013).

In the present study, participants performed a visual search task during the delay, thus rendering the detection stimulus of lower priority. This was a critical requirement for the effects of cognitive control load because these are mediated by reduced control over the task-processing priorities under load. When task performance no longer involves the demand to maintain different processing priorities (as when the detection task is the only task performed during the delay), WM cognitive control load has no effects, whereas VSTM load still reduces detection sensitivity, as the effects of reduced perceptual representation capacity do not depend on prioritization conflicts (Konstantinou, Bahrami, Rees, & Lavie, 2012). These findings are in further support of the dissociation we propose between the effects of VSTM load on perceptual capacity and those of cognitive control WM load.

We note that VSTM load also reduces detection sensitivity for stimuli that are presented at the same locations as those of the VSTM sample (Konstantinou et al., 2012, Experiment 2). Thus, the effects are not due to a change in the focus of spatial attention.

Previous research demonstrated that distractor interference depends on whether the content of the WM task involved overlaps with target or distractor processing (e.g., Kim, Kim, & Chun, 2005; Park, Kim, & Chun, 2007). The present research focused on dissociating the roles of visual maintenance and cognitive control in detection rather than differential involvement in processing the search or detection stimuli. We therefore used the same type of stimuli for both search and detection to ensure our effects cannot be due to a greater overlap with one or the other. Future research may extend this line to dissociating cognitive control from other forms of sensory maintenance (e.g., for verbal material or in other sensory modalities), while taking into consideration whether the target and distractor stimuli share the same sensory representations with the content maintained in WM or may involve distinct neural correlates.

In conclusion, our present report clarifies an important distinction between visual maintenance and cognitive control functions of WM, and demonstrates that these functions can be dissociated through the opposite effects of loading them on visual detection. This dissociation resolves apparent discrepancies in the previous literature and emphasizes the importance of carefully considering whether a WM task loads on sensory representations or on cognitive control.