Psychophysical dual-task setups do not measure pre-saccadic attention but saccade-related strengthening of sensory representations.

Visual attention and saccadic eye movements are linked in a tight, yet flexible fashion. In humans, this link is typically studied with dual-task setups. Participants are instructed to execute a saccade to some target location, while a discrimination target is flashed on a screen before the saccade can be made. Participants are also instructed to report a specific feature of this discrimination target at the trial end. Discrimination performance is usually better if the discrimination target occurred at the same location as the saccade target compared to when it occurred at a different location, which is explained by the mandatory shift of attention to the saccade target location before saccade onset. This pre-saccadic shift of attention presumably enhances the perception of the discrimination target if it occurred at the same, but not if it occurred at a different location. It is, however, known that a dual-task setup can alter the primary process under investigation. Here, we directly compared pre-saccadic attention in single-task versus dual-task setups using concurrent electroencephalography (EEG) and eye-tracking. Our results corroborate the idea of a pre-saccadic shift of attention. They, however, question that this shift leads to the same-position discrimination advantage. The relation of saccade and discrimination target position affected the EEG signal only after saccade onset. Our results, thus, favor an alternative explanation based on the role of saccades for the consolidation of sensory and short-term memory. We conclude that studies with dual-task setups arrived at a valid conclusion despite not measuring exactly what they intended to measure.

INTRODUCTION

Visual attention and eye movements are tightly linked. Following the premotor theory of attention, covert visual attention shifts before the onset of the saccadic eye movement to the location where the eyes are going to move (Craighero & Rizzolatti, 2005; Eimer et al., 2007; Moore et al., 2003; Rizzolatti et al., 1987; Wauschkuhn et al., 1998; Zirnsak et al., 2014). In humans, this relation of attention and saccadic eye movements has primarily been investigated with psychophysical dual‐task setups (Baldauf & Deubel, 2008; Belopolsky & Theeuwes, 2009; Born, Ansorge, & Kerzel, 2012, 2013; Deubel & Schneider, 1996; Godijn & Theeuwes, 2003; Hoffman & Subramaniam, 1995; Kowler et al., 1995; Rolfs et al., 2011; for an exception see (Kulke et al., 2016). In these dual‐task paradigms, participants are typically instructed to make saccades to certain target locations, and before saccade onset, a discrimination target is flashed either at the saccade‐target location or at a different location. In addition to making the saccade, participants are asked to report a specific feature of the discrimination target, for instance, whether the discrimination target is the letter E or the digit 3. It is commonly found that discrimination performance is better when saccade target and discrimination target cooccur at the same location compared to when they occur at separate locations. This same‐position discrimination advantage is usually interpreted as the consequence of the pre‐saccadic shift of attention to the saccade target location, which enhances the perception of the discrimination target when it appears at the same location but not when it appears at a different location (e.g., Deubel & Schneider, 1996).

The link between attention and saccades is tight; however, there is some flexibility. The extent to which pre‐saccadic attention shifts to the saccade target location depends on task requirements and statistical regularities of task events (Belopolsky & Theeuwes, 2009). Moreover, the actual saccade target location can deviate from the intended saccade location, in particular when distractors are present close to the saccade target. Under these conditions, pre‐saccadic attention coincides with the intended and not the actual saccade location (Van der Stigchel & de Vries, 2015). In addition, attention and saccades influence trans‐saccadic adaptation effects in different ways (Melcher, 2009). These findings demonstrate how particular task settings can loosen the link between attention and saccades. In line with this idea, electrophysiological studies in monkeys provide evidence that saccade planning and attention are neuronally dissociable (Wardak et al., 2011). In addition, dual‐task settings increase saccadic latencies (Kristjánsson et al., 2001; Pashler et al., 1993) and increased saccade latencies could in principle allow for more extensive pre‐saccadic allocation of attention. Thus, it is likely that a dual‐task setup requiring saccade execution and a concurrent discrimination task modulates pre‐saccadic attention in some task‐specific way. It is, therefore, at present completely unclear whether the pre‐saccadic same‐position discrimination advantage observed in dual‐task settings does, indeed, reflect natural pre‐saccadic attention or whether it is instead an artifact from the dual‐task setup.

Eventually, instead of explaining the behavioral same‐position discrimination advantage by a pre‐saccadic shift of attention, it could also be explained by the role of saccades in the consolidation of visual sensory and short‐term memory (Ohl & Rolfs, 2017, 2018). If the discrimination target was retained in visual sensory memory and saccade execution prioritized the percept of the discrimination target if it had occurred at the saccade target location, we should observe a same‐position discrimination advantage, too. However, according to this notion, the same‐position discrimination advantage would not result from the pre‐saccadic shift of attention but would occur with or after saccade execution. This alternative hypothesis would also explain why it was not possible for participants to direct, purportedly pre‐saccadic, attention to a location other than the saccade target even if the discrimination target was known in advance to occur at that other location (Experiment 2 of Deubel & Schneider, 1996): simply because actual saccade execution caused the effect. In addition, this alternative hypothesis would also explain why discrimination performance is lower but still far above chance for locations other than the saccade target (e.g., Born et al., 2012; Deubel & Schneider, 1996): simply because the discrimination target at other locations is also retained in visual memory, but visual information at these locations is not boosted by saccade execution (cf. Ohl & Rolfs, 2017) and, therefore, discrimination performance is eventually not as good as at the saccade‐target location. To sum up, first, it is unclear how the dual‐task setup affects pre‐saccadic attention and, second, it is unclear whether the same‐position discrimination advantage actually results from a pre‐saccadic shift of attention.

Our previous research, using combined electroencephalography (EEG) and eye‐tracking, suggests that in a single‐task visual search design, neurophysiological markers of visuospatial selective attention precede the saccade onset (Huber‐Huber et al., 2016). We found differences in event‐related potentials (ERP) between contra‐ versus ipsilateral electrode sites with respect to the saccade target location similar to the N2pc component which is supposed to indicate attentional target selection (Eimer, 1996, 2014; Luch & Hillyard, 1994; Luck & Hillyard, 1994). Some research, however, shows that the N2pc component is time‐locked to stimulus onset (Weaver et al., 2017) and not to saccade execution which suggests that this component is different from the contra‐ipsilateral differences that preceded saccades in our previous study (Huber‐Huber et al., 2016). Still, both effects occur at the same electrode sites and follow the same logic in that they present a lateralized component with an attention shift to one side of the visual field, despite a visually balanced display (Eimer, 1996), which suggests that also our lateralized effect reflected some form of pre‐saccadic attention. However, even if we previously reported evidence for pre‐saccadic shifts of attention in a single‐task setup (Huber‐Huber et al., 2016), we cannot yet conclude that the same‐position discrimination advantage in a dual‐task setup results from this pre‐saccadic shift of attention.

To find out whether a dual‐task setup enhances pre‐saccadic visuospatial attention and to determine whether the same‐position discrimination advantage results from this pre‐saccadic attentional shift, we conducted the same experiment under dual‐task and single‐task instructions, while measuring neurophysiological markers of visuospatial attention with combined EEG and eye‐tracking (for a similar approach see Kulke, 2019). Participants were instructed to make saccadic eye movements to color‐defined target stimuli, while we also presented a discrimination target for 82 ms after target onset and before the saccade could be executed. Our experimental design was similar to Born et al. (2012). In Block 1, however, participants only made saccades, and the discrimination target was not even mentioned to them (single‐task). In Block 2, participants were informed about the discrimination target, an asymmetric cross, and they were instructed to report whether the vertical element of the cross was shifted to the left or to the right (dual‐task). A final block, Block 3, with single‐task trials as in Block 1, concluded the experiment to determine whether the dual‐task setting might have changed pre‐saccadic processing persistently.

To compare pre‐saccadic visuospatial attention in single‐ versus dual‐task setups, we assessed contra‐ipsilateral differences at posterior‐lateral electrodes PO7 and PO8 time‐locked to saccade onsets (Huber‐Huber et al., 2016). If a dual‐task setting affected pre‐saccadic visuospatial attention, the difference in the ERP between contra‐ and ipsilateral sites with respect to the saccade‐target location is expected to vary with dual‐task compared to single‐task blocks. Such a finding would suggest that the dual‐task setup artificially modulates (e.g., invites) pre‐saccadic attention shifts. In addition, if the same‐position discrimination advantage resulted from the pre‐saccadic shift of attention, any same‐versus‐different‐location effect in the ERP should arise from the pre‐saccadic period. Our experimental design contained four possible target locations, two on each side of the screen. Considering that visually evoked contra‐ipsilateral differences are sensitive to the side of the screen at which stimuli occur (e.g., Eimer, 1996), the critical contrast for any same‐versus‐different‐location effect was whether saccade and discrimination target occurred on the same or different sides of the display. In contrast, if the same‐position discrimination advantage resulted from saccade execution, any corresponding effect in the ERP should occur after saccade onset at the earliest.

In addition, we ensured that systematic differences in gaze behavior between conditions did not confound the ERP by analyzing saccade amplitudes, saccade durations, and saccade target fixations. We also compared saccade latencies between single‐ and dual‐task setups to see whether our motivation that dual‐task setups could provide a prolonged time window for pre‐saccadic shifts of attention was justified (Kristjánsson et al., 2001; Pashler et al., 1993). In the dual‐task condition, Block 2, we checked the typical same‐position discrimination advantage in behavioral data.

METHOD

Participants

In total, data from 36 participants were collected. Four participants were excluded because, for them, too few trials were left in each condition (less than 10 per cell of the design). Of the remaining 32 participants, 20 participants were female, five left‐handed, nine left‐eye dominant as determined by the hole‐in‐the‐card test, and the mean age was 22.5 years, ranging from 19 to 30 years. All participants had a normal or corrected‐to‐normal vision, intact color vision, and gave written informed consent prior to the experiment proper. The study was approved by the institutional review board of the University of Vienna.

A sample size of 32 participants achieves 0.95 power to detect within‐participant effects with a partial eta squared ηp 2 of around 0.10 for two‐level factors and around 0.08 for three‐level factors. This power calculation was calculated with the G*Power software (version 3.1.9.3; Faul et al., 2009) using the default settings for correlation among repeated measures, nonsphericity correction, and the conversion between effect sizes f and ηp 2. Note that the power to detect an interaction effect can be obtained by considering the power for the particular contrast that defines the interaction.

Apparatus

Visual stimuli were presented on a 19‐inch color VGA monitor, with a refresh rate of 85 Hz at 1,024 by 768 pixels screen resolution. Participants were seated 58 cm from the monitor, with their heads supported by a chin rest. An EyeLink 1,000+ eye‐tracker (SR Research, Ontario, Canada) recorded the participants’ gaze in the desktop mount setup. The experiment was programmed in MATLAB (The MathWorks, Inc.) using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) with the EyeLink extension (Cornelissen et al., 2002). Manual responses were collected with a computer keyboard.

Stimuli

Four circles, one per quadrant of the visual field, denoted possible saccade‐ and discrimination‐target locations, each placed equidistantly at a radius of 5.92° visual angle from the screen center. Each circle had a diameter of 2.56° visual angle and an edge width of 0.12° visual angle. A grey square with the same edge width was placed inside each circle with a size of 1.77°. The saccade‐target circle was defined by its fixed color, which was counterbalanced across participants. The equiluminant and about equidistant colors in the Lab color space were red (66.8, 28.5, 40.6, measured by an X‐Rite I1 color measurement device, X‐Rite, Grand Rapids, Michigan, USA), yellow (66.3, −38.7, 61.6), green (66.8, −60.7, 13.7), and blue (67.4, 15.8, −73.3). The non‐target circles were of the remaining three non‐target colors of this set. The background was grey (82.3, 4, −11.30), as were the squares inside the circles (67.3, 4.4, −9.5). For the discrimination target display, all squares changed to crosses of the same grey and size. One of the four crosses was the discrimination target and had its vertical element shifted either to the left or to the right. The amount of shift was determined by a staircase procedure (see below). Critically, the discrimination target could be at the same or at a different location than the saccade target. The frequency of all possible arrangements and layouts of the stimuli was balanced across the experiment.

Procedure

Stimuli and procedure were similar to the study by Born and colleagues (2012). An example trial is depicted in Figure 1. Each trial started with a fixation cross (0.39° × 0.39° visual angle). Fixating the cross for at least 750 ms within 1° visual angle triggered the saccade‐target display. The saccade‐target display was presented for 82 ms (seven frames). After this presentation time, the saccade‐target display remained on screen and the discrimination target was added to the display, which means that all the squares inside the circles turned into crosses. This combined saccade‐ and discrimination‐target screen was also presented for 82 ms (seven frames), leading to a total saccade target presentation time of 164 ms (14 frames). The discrimination target was defined as the cross that had its vertical element shifted to the left or to the right. Shift direction was determined randomly in each trial. At the end of saccade‐target presentation, that is, when the colored circles turned grey, all crosses were replaced by squares. This final grey masking display was presented until fixation on the saccade target. Correct fixation was determined online as the first 200‐ms period within which 90% of the individual gaze samples lay within a 1° visual angle from the saccade target center. Offline correct fixation trials were defined as trials in which the first fixation after stimulus onset was within a 2° visual angle from the saccade target location.

FIGURE 1

Experimental procedure. In all three experimental blocks, participants made saccades to a color‐defined target circle (in the trial illustrated here: bottom left location, yellow). In Block 2, they also reported whether the vertical element of the discrimination target (here: top right location in the third display) was shifted to the left or to the right (here: to the left). Stimuli are not drawn to scale. Figure adapted with permission from Born et al. (2012)

In Block 1, participants were instructed to look at their color‐defined saccade target as quickly as possible after saccade‐target onset. The discrimination target was not mentioned. After this single‐task block with 208 trials, participants were informed about the discrimination target. For Block 2, the instruction was again to make an eye movement to the saccade target but additionally report with manual button press after the saccade whether the vertical bar of the discrimination target had been shifted to the left or to the right. We emphasized that for this discrimination task, only correctness counted and response speed was irrelevant. After 416 trials in this dual‐task setting, participants performed Block 3 consisting again of 208 single‐task trials. Participants were familiarized with the single‐task procedure in a set of training trials at the beginning of Block 1 and with the dual‐task procedure in a set of training trials at the beginning of Block 2. We selected this sandwich design instead of counterbalancing the order of single‐task and dual‐task blocks across participants because there was no reason to assume that the single‐task setup would have an effect on dual‐task performance; instead the opposite, that dual‐task instructions had an influence on subsequent single‐task performance, appeared much more likely. Therefore, the within‐participants sandwich design provided higher power with respect to the hypothesized effects than a counterbalancing design which implied a between‐participants factor.

The amount of displacement of the vertical bar was constant at 0.31° in Blocks 1 and 3. In Block 2, the amount of displacement was adjusted by a staircase procedure (QUEST algorithm, QuestCreate function, Watson & Pelli, 1983) with the following parameters: standard deviation 3, step size 1, minimum 2, maximum 18 pixels (corresponding to about 0.56° visual angle), beta 0.5, delta 0.01, gamma 0.5. In each trial, the optimal amount of deviation was determined with the QuestQuantile function (Psychtoolbox Version 3.0.12).

Trials were separated by a grey screen for a random interval of 800 to 1,200 ms (uniformly distributed), and the order of trials was randomized within blocks. However, due to a mistake in the randomization procedure, the location of the discrimination target changed from trial to trial regularly clockwise across all four possible locations, while the saccade‐target location followed a less predictable, but still regular and clockwise sequence in which every fifth step in the clockwise loop was skipped. Note that 16 turns in this sequence contained every possible saccade‐target and discrimination‐target arrangement exactly one time. The data of 20 participants were recorded with this mistake. Another set of 12 participants was recorded with proper randomization of the trial sequence. Although this complicated sequence is probably very difficult to learn, we considered a between‐participants factor Group (fixed, random) in all analyses and report if this factor significantly affected the results.

Electroencephalography and eye‐tracking data recording and analysis

Gaze behavior was recoded monocular with an EyeLink 1,000+ eye‐tracker (SR Research, Ontario, Canada) in desktop mount mode at a sampling rate of 1,000 Hz. The EEG data were recorded at 1,000 Hz with 64 active electrodes (Brain Products, actiCAP system) placed in an elastic cap (EASYCAP GmbH, Herrsching, Germany) at a subset of locations of the 10/10 system. The amplifier was a full‐band DC‐EEG system (neuroConn GmbH, Ilmenau, Germany). Electrode impedances were kept below 5 kΩ (Kappenman & Luck, 2010). The ground electrode at AFz served as an online reference and the offline reference was the average of both mastoids. The signal was digitally low‐pass filtered at 40 Hz with a finite impulse response filter (cut‐off frequency of 45 Hz, −6 dB, transition band width of 10 Hz, filter order 330).

The concurrently recorded eye‐tracking data were synchronized with the EEG data with the help of the EYE‐EEG toolbox (version 0.41, Dimigen et al., 2011) for EEGLAB (version 13.3.2, Delorme & Makeig, 2004) in Matlab (version 2014a, The MathWorks, Inc.). The continuous data were segmented with respect to saccade onset. Baseline correction was conducted with respect to the 200 ms period before saccade‐target onset. A trial was excluded if the first fixation that commenced after saccade‐target onset was not within 2° visual angle from the center of the saccade target and we applied a filter on saccadic response times to exclude trials with implausibly early or late responses. Separately for each participant and cell of the design (see below), a trial was excluded if its saccadic response time was more distant from the median than three times the median absolute deviation (Leys et al., 2013). We did not apply any additional procedures or thresholds to remove artifacts in the EEG signal because the criteria for the saccade to the target together with the gaze‐contingent procedure avoided artifacts in the first place. For the gaze‐contingent setup to work properly, the eye‐tracker had to have continuously good calibration. It is normal that after some time, depending on how still the participant could keep the head in the chinrest, the eye‐tracker calibration deteriorates and recalibration is required. Therefore, the experimenter monitored the eye‐tracking signal during the experiment and initiated recalibration in case the experiment did not continue. Consequently, the eventually analyzed data only contained eye movements from the screen center to the saccade target. A total of 21,289 trials entered the analysis.

For the analysis of ERPs, the single‐trial EEG data were swapped across hemispheres for trials with saccade targets on the left side, rendering electrodes in the left hemisphere contralateral and electrodes in the right hemisphere ipsilateral to the saccade‐target location for all trials. Average ERPs were computed for the three arrangements of saccade‐target and discrimination‐target location relevant for contra‐ipsilateral differences, for the same location, the same side but different locations, and opposite side. This variable was captured in the three‐level within‐participant factor Position (same, same‐side, opposite‐side). Further within‐participant factors were Block (1 single‐task, 2 dual‐task, 3 single‐task) and Laterality (contra‐, ipsilateral with respect to saccade target). The additional between‐participants factor Group (fixed, random) coded for each participant whether they did the fixed or the properly randomized sequence of trials (see Procedure). This factor did, however, not show any significant effects. Average ERPs were analyzed with mixed repeated‐measures analyses of variance (ANOVAs), including all main effects and interactions. If Mauchly’s test of sphericity was significant for factors with more than two levels, corresponding p values were calculated with Greenhouse‐Geisser corrected degrees of freedom.

In order to rule out possible confounds from eye gaze behavior in the EEG data, we analyzed saccade amplitudes and target‐fixation durations of exactly the same trials and with the same factors as the EEG data, of course except for the factor Laterality.

RESULTS

Discrimination performance

To determine the same‐position discrimination advantage, we analyzed discrimination performance in Block 2, which featured the additional discrimination task, in a repeated‐measures ANOVA with the within‐participant factor Position (same, different) and the between‐participants factor Group (fixed, random). In contrast to the ERP‐ and eye‐tracking data analyses, here, we merged the data of all trials in which the discrimination target was not at the saccade‐target location, regardless of whether it was at the same or the opposite side, to make our analysis consistent with previous research (Born et al., 2012).

As expected, participants were better in discriminating the shift (left or right) of the vertical element of the discrimination target when the discrimination target had appeared at the same location (84.1% correct) as the saccade target compared to at a different location (67.5% correct), F(1, 30) = 82.75, p < .001, ηp 2 = 0.73. However, an interaction with participant group, F(1, 30) = 4.73, p = .038, ηp 2 = 0.14, suggested a larger same‐position discrimination advantage for the group with the properly randomized trial sequence (20.7%) compared to the group with the fixed trial sequence (12.7%). This result might have indicated that participants in the fixed sequence group did, indeed, learn the regular, although complicated, trial sequence to some degree, which allowed them to predict the discrimination target‐location leading to lesser performance differences between same‐ versus different‐location conditions. However, in all other analyses, the factor group did not show any significant effects, which limits the evidence for differences between groups to this particular finding. Eventually, both groups exhibited a significant same‐position discrimination advantage, F(1, 11) = 60.04, p < .001, ηp 2 = 0.85, for the random group and, F(1, 19) = 29.33, p < .001, ηp 2 = 0.61, for the fixed group, which shows that the theoretically most relevant aspect of better discrimination performance in the same‐ versus different‐location trials is present, regardless of the fixed or random trial sequence.

Gaze behavior

Saccade latencies

One motivation of the present study was the idea that a secondary task delays saccadic latencies (similar to Kristjánsson et al., 2001; Pashler et al., 1993), which can create a time window facilitating pre‐saccadic shifts of attention. According to this hypothesis, saccadic latencies were expected to be longer in a dual‐task than in a single‐task setting. This hypothesis was confirmed in a Task (single, dual) × Group (fixed, random) ANOVA showing only a significant main effect of task, F(1, 30) = 5.73, p =.023, ηp 2 = 0.16, with 275 ms in the dual‐task and 250 ms in the single‐task setting. This particular result supports the idea that the dual‐task setup extended the pre‐saccadic time‐window for about 25 ms, which could provide additional time for pre‐saccadic shifts of attention. However, we also analyzed saccade latencies in the same design as later on the EEG data. This second analysis revealed a clear effect of Block, F(2, 60) = 7.47, p =.007, ηp 2 = 0.20, which indicated that saccade latencies were particularly fast in the last single‐task Block 3 (241 ms), not that fast in the very first Block 1 (260 ms), and slowest in the dual‐task Block 2 (274 ms) with Bonferroni‐corrected post hoc comparisons being significant for Block 3 versus 2 t(31) = 3.79, p = .002, d = 0.67 and Block 3 versus 1 t(31) = 5.29, p < .001, d = 0.93, but not for Block 2 versus 1 t(31) = 1.35, p = .560, d = −0.24. A significant Group × Block × Position interaction, F(4, 120) = 2.88, p = .026, ηp 2 = 0.09, suggested that this effect of Block was particularly present for the fixed group, F(2, 38) = 8.98, p = .003, ηp 2 = 0.32, but not for the random group, F(2, 22) = 1.53, p = .243, ηp 2 = 0.12, although the pattern was numerically in the same direction in both groups. Post hoc comparisons targeting the contribution of the factor Position to the three‐way interaction were not significant. This pattern of results shows that saccade latencies tended to be slower in the dual‐task setting, although the pattern involving the first and the last single‐task block was slightly different and suggests some additional influence of practice or time on task. Overall, however, saccade latencies are in line with our rationale that visual attention could operate differently in dual‐task compared to single‐task setups and those physiological methods avoiding a dual‐task setup can provide valuable insights.

Saccade amplitudes

It is known that saccadic amplitudes affect the fixation‐related EEG (e.g., Dimigen et al., 2011; Kaunitz et al., 2014; Ries et al., 2018). To rule out possible confounds, we analyzed saccade amplitudes with exactly the same design as the EEG data and the same set of trials. As Figure 2b shows, there was a trend toward increasing saccade amplitudes across the blocks of the experiment, main effect of Block, F(2, 60) = 8.85, p < .001, ηp 2 = 0.23. However, Bonferroni‐corrected post hoc t tests revealed a statistically significant difference only between Blocks 1 and 3, t(31) = 4.23, p < .001, d = 0.75, but not for the other pairwise comparisons, Block 1 versus 2, t(31) = 1.95, p = .180, d = 0.35, and Block 2 versus 3, t(31) = 2.29, p = .087, d = 0.40. No other effects were significant. Considering this pattern of results together with saccade latencies, it might be tempting to think that in Block 2, saccades were executed later but that they were also more precise in the sense that they were closer to the target. However, not the saccades in Block 2 but in Block 3 corresponded most closely to the saccade target, as can be seen from the horizontal line in Figure 2b at 5.92°, indicating the distance between screen center and saccade target. Moreover, this line illustrates that saccades in all conditions tended to be slightly shorter than to the target center.

FIGURE 2

Gaze behavior for the three experimental blocks and the three saccade‐target and discrimination‐target position relations relevant for the EEG analysis. (a) Saccade latencies with respect to saccade target onset. (b) Saccade amplitudes in degrees of visual angle. The horizontal black line at 5.92° indicates the distance between the screen center and center of the saccade target. (c) Saccade duration in milliseconds. (d) Duration of the first fixation on the saccade target in milliseconds

Saccade durations

Different saccade amplitudes between conditions could also mean differences in the duration of the saccades which could have presented a further confound for the post‐saccadic ERP. The end of a saccade is the onset of a new fixation and a new fixation triggers a new fixation‐related potential which means that differences in saccade durations between conditions could have led to differences in the ERP between conditions which could not have been attributed directly to the experimental manipulation. To rule out this confound, we analyzed saccade durations in the same way as saccade amplitudes. Figure 2c shows that saccade durations were very uniform. No effects were significant, all p > .127, all ηp 2 < 0.07, which means that there is no evidence for the contamination of the post‐saccadic ERP by differences in the timing of new fixation onsets.

Fixation duration

Similar to saccade amplitudes and saccade durations, a difference in the duration of the fixation on the saccade target can account for potential differences in the post‐saccadic EEG. We, therefore, also analyzed the duration of the first fixation on the saccade target with the same design as the EEG data for the same set of trials. As can be seen from Figure 2d, fixations were longer in dual‐task Block 2 compared to the single‐task Blocks 1 and 3. This was statistically confirmed by a main effect of Block, F(2, 60) = 9.53, p < .001, ηp 2 = 0.24, and Bonferroni‐corrected post hoc t tests for Block 1 versus 2, t(31) = 2.71, p = .033, d = 0.48, and Block 2 versus 3, t(31) = 4.28, p < .001, d = 0.76. There was no evidence for a difference between Blocks 1 and 3, t(31) = 1.66, p = .322, d = 0.29. These results suggest that the dual‐task setup influenced not only visual attention but also post‐saccadic target processing. Importantly, to avoid artifacts in the ERPs due to systematic variation in fixation durations and follow‐up eye movements, we subsequently limited the ERP analysis in time to a maximum of 200 ms after saccade onset (see below). Note that 200 ms after saccade onset corresponds to the first fixation duration of about 150 to 160 ms, considering the mean saccade duration of 45 ms in this task. Thus, for the effects in the ERPs, fixation durations do not present a serious source of confound. Moreover, they provide additional evidence that the dual‐task setup modulates behavior, in particular, they provide evidence that the additional discrimination task affected how participants carry out the saccade task.

Pre‐saccadic event‐related potentials

To determine the amount of evidence for pre‐saccadic attentional selection of the saccade target, we analyzed the difference between contra‐ and ipsilateral potentials in the EEG signal time‐locked to the saccade onset. Figure 3 shows saccade‐locked ERPs at electrodes PO7/8 sorted into contra‐ and ipsilateral locations with respect to saccade direction, separately for the three experimental blocks (Figure 3a‐c) and the corresponding contra‐ipsilateral difference waves (Figure 3d). As can be seen from this figure, a very small difference between contra‐ and ipsilateral sites emerged before saccade onset, suggesting the attentional selection of the saccade target. A repeated‐measures ANOVA on mean amplitudes from −50 to −10 ms confirmed a significant Laterality main effect, F(1, 30) = 6.71, p = .015, ηp 2 = 0.18, demonstrating a more negative potential at the site contralateral to the saccade target. This pre‐saccadic time period largely overlapped with the time period during which the discrimination target was present, which can be seen from the temporal distribution of discrimination target onsets in Figure 3e considering the discrimination target presentation time of 82 ms. Importantly, there was no evidence that this pre‐saccadic indicator of attentional selection differed between Blocks because the Laterality x Block interaction was not significant, F(2, 60) = 1.32, p = .276, ηp 2 = 0.04; all other interactions involving Laterality and Block were not significant, with F < 1.09. Irrelevant to our argument, a main effect of Block, F(2, 60) = 4.69, p = .013, ηp 2 = 0.14, suggested a generally more positive pre‐saccadic potential in Block 1 compared to the other Blocks 2, t(31) = 2.65, p = .038, d = 0.47, and Block 3, t(31) = 2.77, p = .028, d = 0.49. The difference between Blocks 2 and 3 was not significant, t(31) = 0.39, p = .999, d = 0.07. This blocking effect might just have reflected some form of learning or adaptation in the course of the experimental session.

FIGURE 3

Saccade‐onset locked event‐related potentials (ERPs) at electrode pair PO7/8 sorted into contra‐ and ipsilateral sites with respect to saccade‐target location for the three experimental blocks (Panels A, B, and C) and the differences between the contra‐ and ipsilateral waveforms (Panel D). Waveforms were averaged across the position of the remaining factors (same, same‐side, opposite‐side) and group (fixed, random). Negativity plotted upwards. Baseline correction was conducted with respect to the 200 ms period before saccade‐target onset (not shown). Panel E shows the distribution of discrimination target onsets within each block. Panel F shows the distribution of target fixation offsets as the proportion of all trials. Time point 0 is saccade onset

Early post‐saccadic event‐related potentials

As Figure 3 shows in particular in Panel D, the contra‐ipsilateral difference increased substantially after saccade onset. A repeated‐measures ANOVA on mean amplitudes from 20 to 80 ms showed a highly significant Laterality effect, F(1, 30) = 119.34, p < .001, ηp 2 = 0.80, an effect of Block, F(2, 60) = 4.35, p = .017, ηp 2 = 0.13, and their interaction, F(2, 60) = 6.80, p = .002, ηp 2 = 0.18. Bonferroni‐corrected post hoc t tests for this interaction confirmed a larger contra‐ipsilateral difference in the dual‐task Block 2 than in both single‐task Blocks 1 and 3, t(31) = 2.61, p = .041, d = 0.65 and t(31) = 3.71, p = .002, d = 0.93, respectively. There was no evidence for a difference between the single‐task Blocks 1 and 3, t(31) = 1.42, p =.494, d = 0.36. Note that any contra‐ipsilateral differences after saccade onset are strongly confounded by artifacts from the eye‐movement itself and from the change in gaze position. To rule out this possible confound, we analyzed saccade amplitudes, which showed some tendency to increase across experimental Blocks, while only being significantly different between Blocks 1 and 3 (Figure 2b). Importantly, however, saccade amplitudes would have had to show a different pattern in order to explain the finding in saccade‐onset locked ERPs. The contra‐ipsilateral difference was largest in Block 2 and about the same in Blocks 1 and 3. It is, thus, clear that the difference in saccade amplitudes cannot explain the larger contra‐ipsilateral difference in the dual‐task Block 2 versus single‐task Blocks 1 and 3. In contrast to saccade amplitudes, saccade latencies showed a pattern that could be related to the early post‐saccadic contra‐ipsilateral difference. Saccade latencies tended to be longer in the dual‐task Block 2 compared to the single‐task Block which could suggest that this tendency toward a later onset of the saccades in Block 2 allowed for the larger contra‐ipsilateral difference in Block 2.

Interestingly, in the early post‐saccadic time window of 20 to 80 ms the three‐way interaction Laterality x Block x Position was also significant, F(4, 120) = 3.25, p = .014, ηp 2 = 0.10, which indicated that not only the saccade target but also the discrimination target had an effect on contra‐ipsilateral differences. The average contra‐ipsilateral differences for the nine conditions are illustrated in Figure 4a. To better interpret this interaction, we calculated two‐by‐two interaction effect contrasts. We calculated the difference in contra‐ipsilateral differences for pairs of position conditions and conducted a pairwise t test on these values comparing block conditions. Each of the nine post hoc tests was Bonferroni‐corrected. The outcome was, however, mixed. There was no evidence for a difference between same and same‐side conditions across any of the blocks. The same and opposite‐side conditions differed for Block 1 versus Block 2, t(31) = 3.90, p = .004, d = 0.98, and the same‐side versus opposite‐side conditions differed for Block 2 versus Block 3, t(31) = 3.12, p = .036, d = 0.78. All other pairwise comparisons were not significant. Despite this statistically rather unclear pattern, Figure 4 suggests that in particular the same and same‐side position trials contributed to the overall larger early post‐saccadic contra‐ipsilateral difference in Block 2 (see also Figure 3). This result suggests that in the dual‐task Block 2, but not in the single‐task Blocks 1 and 3, the saccade target location was processed more in‐depth if that location also contained the discrimination target. Thus, the discrimination target modulated the ERP with a dual‐task setup but not with a single‐task setup. Importantly, however, this effect of saccade‐target and discrimination‐target position occurred after saccade onset.

FIGURE 4

Average saccade‐locked event‐related potential amplitudes of contra‐ipsilateral differences in the early (20–80 ms, Panel A) and the late (100–200 ms, Panel B) post‐saccadic time windows, separately for all Blocks (1 single‐task, 2 dual‐task, 3 single‐task) and Positions of saccade and discrimination target (same, same‐side, opposite side). Error bars represent Morey‐factor corrected within‐participant confidence intervals

In our design with four saccade‐target locations, the post‐saccadic event‐related potentials can be confounded by an imbalance in the frequency of trials for each saccade‐target location in the finally analyzed data set. To rule that out, we compared the counts of trials for each saccade‐target location with a mixed Poisson regression model (identity link) containing the same predictors as in the EEG analysis and the additional four‐level factor saccade‐target location (left top, right top, right bottom, left bottom, see Figure 1). The model had a random factor for participants and was run with the lme4 package in R (Bates et al., 2013; R Core Team, 2013) using successive difference contrasts from the MASS package. This analysis confirmed the larger number of trials in Block 2 compared to Block 1, ß = 11.99, SE = 0.33, p < .001, and compared to Block 3, ß = 11.78, SE = 0.33, p < .001, as well as the larger number of trials for opposite‐side position trials compared to same‐side position, ß = 13.67, SE = 0.34, p < .001, but not between same‐side and same position trials, ß = −0.07, SE = 0.27, p = .782. These effects were expected based on the design of the experiment (see Methods section). In addition, this analysis revealed a main effect of saccade‐target location showing about 1 trial less for the right bottom location compared to the right top location, ß = −1.13, SE = 0.367320, p = .002, and about 1 trial less for the left bottom location compared to the right bottom location, ß = −0.814, SE = 0.357, p = .023. Importantly, the factor saccade‐target location did not interact with any of the other factors which means that a difference in the number of trials across saccade‐target locations cannot explain the ERP effects.

Late post‐saccadic event‐related potentials

The post‐saccadic ERP exhibited a second phase of contra‐ipsilateral differences, which we statistically evaluated from 100 to 200 ms. This time window was chosen to capture most of the effect, while avoiding too much contamination by secondary eye‐movements after the end of the first fixation (Figure 3f). In this second phase, there was again a very clear Laterality main effect, F(1, 30) = 49.92, p < .001, ηp 2 = 0.62, and a Laterality x Block interaction, F(2, 60) = 6.96, p = .002, ηp 2 = 0.19, which confirmed a larger contra‐ipsilateral difference in Block 2 compared to Block 1, t(31) = 4.64, p < .001, d = 0.82. There was, however, no evidence that Block 3 differed from Block 1, t(31) = 1.83, p = .230, d = 0.32, or from Block 2, t(31) = 1.99, p = .165, d = 0.35, which reflects that, in this later phase, the waveform of Block 3 appeared numerically between Blocks 1 and 2 (Figure 3d). Similar to the earlier post‐saccadic phase, also the Laterality × Block × Position interaction was significant in this later phase, F(4, 120) = 3.95, p = .005, ηp 2 = 0.12. The contra‐ipsilateral differences for the corresponding nine conditions are plotted in Figure 4b and were followed‐up in the same way as for the earlier post‐saccadic effect. Here, the influence of the discrimination target position was statistically much clearer than in the earlier phase. The contra‐ipsilateral difference for same position versus opposite‐side varied between Blocks 1 and 2, t(31) = 4.36, p = .001, d = 0.77, as well as between Blocks 2 and 3, t(31) = 3.31, p = .014, d = 0.59. This difference was the same for same‐side versus opposite‐side conditions comparing Blocks 1 and 2, t(31) = 3.58, p = .011, d = 0.63, and Blocks 2 and 3, t(31) = 3.04, p = .029, d = 0.54. Again, there was no evidence that the contra‐ipsilateral difference varied across Blocks 1 and 3 or across same and same‐side conditions.

DISCUSSION

Concurrently recording EEG and eye‐tracking data, we investigated the relation of visuospatial selective attention and saccadic eye movements in single‐task and dual‐task settings. We replicated the same‐position discrimination advantage in the dual‐task Block 2. Participants were better in reporting the left versus right shift of the vertical element of the discrimination target when it had appeared at the location of the saccade target in contrast to a different location. As expected, however, the additional discrimination task prolonged saccade latencies, which confirms our suspicion that the dual‐task setting did, indeed, affect saccade execution. These findings suggest that previous studies only using a dual‐task setting to investigate the link between attention and saccades altered the object under investigation by investigating it. Although the prolonged saccade latencies in the dual‐task block could have provided a time window for more extensive pre‐saccadic shifts of attention as we conjectured, there was no evidence from ERPs for this fact. Before saccade onset, contra‐ipsilateral differences appeared to be the same in the single‐task blocks compared to the dual‐task block. This finding suggests that, although the dual‐task setting modulated saccade execution, the extent of pre‐saccadic shifts of attention was the same. Interestingly, before the saccade, there was also no evidence for an effect of the relative positions of the saccade‐ and discrimination target; in other words, it did not matter whether the discrimination target was at the same or at a different location than the saccade target. Only after saccade onset, an effect of position emerged, but crucially only in the dual‐task block. With the concurrent discrimination task, the contra‐ipsilateral difference after saccade onset was larger if saccade and discrimination target were at the same position or at the same side of the display compared to at opposite sides. This finding is crucial because it suggests that the same‐position discrimination advantage observed in the behavioral data does not result from the pre‐saccadic shift of attention but instead originates from processes after saccade onset.

The same‐position discrimination advantage is traditionally considered to arise from the pre‐saccadic shift of the attentional spotlight to the saccade target location. If the discrimination target occurs at the same location, the perception of the discrimination target is enhanced, leading to better discrimination performance (Born et al., 2012, 2013; Deubel & Schneider, 1996). We found evidence for a pre‐saccadic shift of attention, which could, in theory, result in a same‐position discrimination advantage. If this was the case, however, the relative positions of saccade and discrimination target should either not be reflected in the ERP at all or, if they have an effect, they should have an effect already before the saccade onset. Instead, in our data, saccade and discrimination target positions affected the ERP only after the saccade onset. Therefore, we argue for an alternative explanation in terms of sensory and short‐term memory (Gegenfurtner & Sperling, 1993; Ohl & Rolfs, 2017, 2018; Zerr et al., 2017). Ohl and Rolfs (2017) showed that saccade execution crucially affects the content of visual short‐term memory. A saccade, executed after a memory array had been shown, increased memory performance for the saccade target item even if other items in the array were more likely to be probed for the subsequent memory task. We argue that the same‐position discrimination advantage results from the same mechanism. Our idea is that, after presentation, the discrimination target display is retained in sensory memory and executing a saccade prioritizes the saccade‐target location. If the discrimination target had occurred at the same location, its memory trace is better conserved compared to when it had appeared at a different location, which eventually leads to the same‐position discrimination advantage in behavior. These findings are also in line with the critical role of visual working or short‐term memory for visual discrimination (Kiyonaga et al., 2017; Scimeca et al., 2018; Teng & Kravitz, 2019) and for saccade execution (Hollingworth et al., 2013; Schneegans et al., 2014). In sum, our results are consistent with the idea of pre‐saccade allocation of visuospatial attention to the saccade target location; they only question the pre‐saccadic origin of the behavioral same‐position discrimination advantage and suggest instead that this effect results from a prioritization of the saccade target location in sensory memory due to saccade execution (cf. Teng & Kravitz, 2019).

In addition to explaining our main finding, the sensory memory account (e.g., Ohl & Rolfs, 2017) can explain three more findings of the same‐position discrimination advantage. First, the discrimination target was presented 82 ms after saccade‐target onset and it remained on screen for another 82 ms, which means that it disappeared 164 ms after‐saccade target onset. Saccades started on average 262 ms after saccade‐target onset with a minimum of 152 ms, the first quartile at 216 ms, and the median at 242 ms. The pre‐saccadic contra‐ipsilateral difference did not appear much earlier than 50 ms before saccade onset. Therefore, it is likely that in most of the trials the discrimination target had already disappeared from the screen by the time the pre‐saccadic shift of attention was initiated. This aspect is more difficult to accommodate in the pre‐saccadic account for the same‐position discrimination advantage and requires to posit some form of sensory memory as well to which the pre‐saccadic shift of attention can be directed. In contrast, according to the sensory‐memory account, the whole discrimination target display is kept in sensory memory after it disappeared and saccade execution subsequently prioritizes the saccade target location, maybe akin to more covert read‐out biases in attention (cf. Shiu & Pashler, 1994).

Second, the sensory memory account explains the purportedly obligatory coupling between saccades and attention that has been inferred from dual‐task setups equally well as the pre‐saccadic account. In their Experiment 2, Deubel and Schneider (1996) presented the discrimination target always at the same location and informed participants about that. Still, discrimination performance at that location was not as good as at the saccade target location, which suggested that participants were not able to direct pre‐saccadic attention to a location different from the saccade target. According to the sensory memory account, prior knowledge about the discrimination target location does not matter; only saccade execution determines the location that is better retained in visual short‐term memory, consequently leading to the same‐position discrimination advantage.

Third, discrimination performance is better at the saccade target location, but it is not at chance level for all other locations. This fact is slightly more difficult to accommodate in the pre‐saccadic account than in the sensory memory account. According to the pre‐saccadic account, attention facilitates discriminating the feature of the discrimination target, which explains why target discrimination performance is better at the saccade target than at other locations, but it does not provide an explicit explanation for why discrimination performance is not at the chance level for the other locations. The sensory memory account additionally explains that aspect. According to the sensory memory account the discrimination target is retained in memory even if it appeared at other locations; therefore, discrimination performance is above chance also for these locations; just that the saccade target location is prioritized by the executed saccade leading to relatively better discrimination performance for the saccade target location compared to the other locations.

Importantly, we considered confounds from eye gaze behavior in the ERP results by analyzing saccade amplitudes and first fixation durations on the saccade target. Saccade amplitudes appeared to increase across blocks being eventually slightly larger in Block 3 compared to Block 1. This pattern, however, cannot confound the ERP results, because larger amplitudes would be expected to lead to larger contra‐ipsilateral differences, but they were, after saccade onset, largest in Block 2 and of similar size in Blocks 1 and 3. Eventually, the variation in saccade amplitudes might have been too small to show an effect in the ERP. Confounds from fixation durations were largely avoided by limiting the ERP analysis in time to the period before most of the fixations ended.

Whereas contra‐ipsilateral differences before the saccade can be interpreted as an attention‐related component given suitable experimental conditions, effects after saccade onset have to be interpreted very cautiously. Every eye‐movement causes an artifactual difference in the electrical potential between contra‐ and ipsilateral electrode sites, because of the electrostatic corneoretinal dipole of the eyeball (Dimigen et al., 2011; Lins et al., 1993; Plöchl et al., 2012). Importantly, our effects in the contra‐ipsilateral differences after saccade onset could not be explained by a rotation of the eye as measured in terms of saccade amplitudes. The contra‐ipsilateral difference after saccade onset was larger in Block 2 than Blocks 1 and 3, particularly in the same and same‐side position conditions. This finding contrasts saccade amplitudes which differed only between Blocks 1 and 3 and did in particular not show any effects of position. However, the corneoretinal artifact precludes interpreting the contra‐ipsilateral difference after saccade onset per se as an indicator of attention.

Our sandwich design with the dual‐task Block 2 in between two single‐task Blocks 1 and 3 showed that the effect of the dual‐task setting was limited mainly to Block 2 and did not transfer to the subsequent single‐task. Only the later post‐saccadic ERP hinted at a less clear difference between Blocks 2 and 3, which might have indicated some sort of transfer from the dual‐task block. However, regarding the pre‐saccadic and the early post‐saccadic time windows, there was no evidence for transfer effects from the dual‐task to the trailing single‐task block. Following our sensory memory account, this result suggests that the prioritization of a spatial location by saccade execution depends on the task instructions and is not an automatic process accompanying every saccade. Interestingly, similar task‐related flexibility has been attributed to the link between attention and saccades (Belopolsky & Theeuwes, 2009).

One group of participants had performed the sequence of trials in a fixed, yet complicated, order because of a mistake in randomization. Considering that this might have affected our results, we included a between‐participants factor in our design. This factor eventually only affected discrimination performance and, importantly, both groups of participants showed the expected same‐position discrimination advantage, which suggests that this flaw in randomization can safely be disregarded.

To sum up, we corroborated the idea of pre‐saccadic attentional selection of the saccade target location measuring ERPs time‐locked to saccade onsets in single‐saccade task and dual saccade‐and‐discrimination task designs. However, our results suggest that, in contrast to the traditional explanation, the same‐position discrimination advantage does not result from the pre‐saccadic allocation of attention to the saccade target location but from a prioritization of the saccade target location in the sensory memory representation of the discrimination target display.

CONFLICT OF INTERESTS

The authors declare no conflict of interests.

AUTHOR CONTRIBUTION

Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Resources; Software; Supervision; Validation; Visualization; Writing‐original draft; Writing‐review & editing: Huber‐Huber. Data curation; Investigation; Resources; Writing‐review & editing: Steininger. Software; Writing‐review & editing: Grüner. Conceptualization; Formal analysis; Investigation; Methodology; Project administration; Resources; Supervision; Writing‐review & editing: Ansorge.