Oculomotor interference of bimodal distractors.

When executing an eye movement to a target location, the presence of an irrelevant distracting stimulus can influence the saccade metrics and latency. The present study investigated the influence of distractors of different sensory modalities (i.e. auditory, visual and audiovisual) which were presented at various distances (i.e. close or remote) from a visual target. The interfering effects of a bimodal distractor were more pronounced in the spatial domain than in the temporal domain. The results indicate that the direction of interference depended on the spatial layout of the visual scene. The close bimodal distractor caused the saccade endpoint and saccade trajectory to deviate towards the distractor whereas the remote bimodal distractor caused a deviation away from the distractor. Furthermore, saccade averaging and trajectory deviation evoked by a bimodal distractor was larger compared to the effects evoked by a unimodal distractor. This indicates that a bimodal distractor evoked stronger spatial oculomotor competition compared to a unimodal distractor and that the direction of the interference depended on the distance between the target and the distractor. Together, these findings suggest that the oculomotor vector to irrelevant bimodal input is enhanced and that the interference by multisensory input is stronger compared to unisensory input.

Introduction

Every day we perceive an abundant amount of sensory information. All this information is rarely perceived via a single sense. We can, for example, see and hear a car approaching, people talking to each other, and birds flying by. To make sense of all this sensory input, information from different senses is combined into a single percept (Stein and Stanford, 2008, Stevenson et al., 2012). As such, our senses contribute to a single sensory experience of the environment. Multisensory integration can lead to behavioral benefits such as faster eye movements to multisensory as compared to unisensory stimuli (e.g. Colonius and Arndt, 2001, Hughes et al., 1994).

Whereas some combinations of stimuli in the environment are relevant to us, other sensory input distracts us from our current behavioral goals. In order to properly interact with the environment we need to determine what sensory information belongs together. The integration of information from different senses is governed by two main principles: (1) stimuli from different senses tend to integrate when they are spatially aligned (i.e. the spatial rule, Stein & Meredith, 1993; see Stevenson et al., 2012 for a review), and (2) multisensory integration depends on whether stimuli from different senses are presented within a temporal binding window of about 150 ms (Leone and McCourt, 2013, Meredith and Stein, 1986, Stevenson et al., 2012, Wallace et al., 1996).

Because we can only execute one eye movement at a time, the oculomotor system might be especially dependent on the precise selection of the relevant sensory information. To date, most of the studies in which the effects of information from different senses on the oculomotor system have been investigated, the focus has been on either on bimodal targets (audiovisual) or on unimodal target and distractor configurations (auditory or visual; (Bolognini et al., 2005, Colonius and Arndt, 2001, Corneil and Munoz, 1996, Corneil et al., 2002, Engelken and Stevens, 1989, Hershenson, 1962, Hughes et al., 1994, Konrad et al., 1989, Lee et al., 1991, Ten Brink et al., 2014, Todd, 1912). In everyday life, however, distractors consisting of a variety of combinations of sensory input can influence the oculomotor system. So, whereas the effect of unimodal distractors on saccadic behavior has been studied extensively, the influence of crossmodal distractors on the oculomotor selection process is largely unknown (Coren and Hoenig, 1972, Coëffé and O’Regan, 1987, Doyle and Walker, 2001, Findlay, 1982, Findlay and Blythe, 2009, Ottes et al., 1984, Van der Stigchel et al., 2012, Van der Stigchel and Theeuwes, 2005, Walker et al., 1997).

The presence of a unimodal distractor interferes with the correct programming of a saccade to a target. This is reflected in both the temporal and spatial properties of the saccade. The distance between a target and a distractor modulates these effects. In the temporal domain it is known that distractors that are positioned at a remote location from the target evoke longer latencies as compared to distractors close to a target (Walker et al., 1997). This well-known Remote Distractor Effect (RDE) is thought to be the result of lateral inhibition between target and distractor. The lateral inhibition between the two locations slows the speed at which the response threshold of the saccade to the target is reached (Godijn & Theeuwes, 2002).

The spatial influence of a distractor on saccade properties is reflected in changes in the saccade endpoint and saccade trajectory. Distractor effects have been explained in terms of the programming of an oculomotor vector towards the target and distractor locations. The oculomotor system has to resolve the competition between these two vectors in order to determine the goal of the next eye movement (Meeter et al., 2010, Trappenberg et al., 2001). In this process, the vector programmed towards the distractor needs to be inhibited in order to avoid making an eye movement to the distractor (Rizzolatti et al., 1987, Sheliga et al., 1995). Based on Tippers population coding theory, inhibition of a distractor implies that a subset of neurons is down-tuned which decreases its effect on the overt response (Tipper, Howard, & Jackson, 1997). Presenting a distractor close to the target makes it impossible to completely inhibit the distractor because this would result in (partially) inhibiting the response vector to the target. The result is that the distractor is only weakly inhibited and causes the resultant vector, and therefore the saccade endpoint and saccade trajectory, to deviate towards the distractor. The partial inhibition of a vector that is evoked by a close distractor can therefore result in saccade averaging (e.g. ‘global effect’, Heeman et al., 2014, Van der Stigchel et al., 2006, Van der Stigchel and Nijboer, 2011, Van der Stigchel et al., 2012, Van der Stoep et al., 2012) or trajectory deviation towards the distractor (Van der Stigchel, 2010, Van der Stigchel et al., 2006).

In contrast to weak inhibition, strong inhibition of a vector can occur in the presence of a remote distractor, causing the resultant vector to deviate away from the distractor. This is often observed when the distractor is located at a position remote from the target (Doyle and Walker, 2001, Tipper et al., 2001, Tipper et al., 1997, Van der Stigchel et al., 2006). The directional shift of the saccade endpoint or the saccade trajectory away or towards the distractor location reflects the outcome of the competition between target and distractor (Walker et al., 1997). A larger saccade endpoint deviation or trajectory deviation either away or towards a distractor implies a stronger influence of the distractor.

Support for the idea that crossmodal distractors can influence the spatial properties of saccades in a similar way as unimodal distractors comes from various studies. For example, when an auditory and a visual stimulus were presented simultaneously in close spatial proximity the saccade landed in between the two stimuli (Lueck, Crawford, Savage, & Kennard, 1990). In addition, saccade trajectories of saccade towards a visual target are modulated by the presence of an auditory signal (Frens, Van Opstal, & Van der Willigen, 1995). When an auditory distractor was presented in the vicinity of a visual target, the saccade was initiated in the direction of the midpoint between the target and distractor before curving towards the visual target. Evidence that saccade metrics were influenced by auditory stimuli was also provided by Doyle and Walker (2002). Their study indicated that when participants made a saccade to a visual target the auditory distractor had a significant effect on saccade trajectory deviation. The trajectory curved towards the auditory distractor when it was presented close to a target while the saccade curved away from the auditory distractor when the distractor was presented more distant from the target. More recently, Ten Brink and colleagues (2014) showed that visual distractors also affect saccades to auditory targets, but this strongly depended on whether the target and distractor were horizontally or vertically aligned.

Although the effects of unimodal (crossmodal and intramodal) distractors on saccade metrics have been clearly demonstrated, it is currently unknown how multisensory distractors affect saccades to unimodal visual targets. The fact that saccades to multisensory targets are much faster than to unisensory targets suggests that multisensory stimuli are more potent than unisensory stimuli and result in reaching the response threshold earlier (Bolognini et al., 2005). Further support for this notion comes from studies in which it was shown that multisensory exogenous cues were able to attract attention under both high and low cognitive load, whereas unimodal exogenous cues only attracted attention under low cognitive load (see Spence & Santangelo, 2009 for a review). If multisensory stimuli are indeed represented more strongly compared to unimodal stimuli they might also prove to be stronger distractors in the oculomotor system.

In the current study the effects of unimodal and bimodal distractors on oculomotor selection were investigated. We hypothesized that the bimodal distractors influenced saccade metrics more than unimodal distractors and that this increase can be explained in terms of oculomotor competition. Saccade endpoint deviation and saccade trajectory deviation were adopted as a proxy of the influence of different distractor modalities. In the current study targets and distractors were presented in configurations similar to studies investigating the influence of unimodal visual distractors located either close to or remote from the target (e.g. Heeman et al., 2014, Van der Stigchel et al., 2012). As such, different spatial and temporal modulatory effects of unimodal and bimodal distractors on saccade endpoint deviation, saccade trajectory deviation and saccade latency can be investigated.

Material and methods

Participants

Nine participants (21–39 years old; M = 27.8, SD = 6.8; 4 males) took part in the experiment. For participating they either received money or course credits. All participants had normal or corrected-to-normal visual acuity and correctly reported hearing the auditory stimuli when they were presented during the training block. Informed consent was obtained from all individual participants included in the study. This study has been carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans.

Apparatus

Eye movements were recorded using the Eyelink 1000 eye tracker system (SR Research Ltd., Mississauga, Ontario, Canada) in the desktop mount configuration. This infrared video based eye tracker has a 1000 Hz temporal resolution and a spatial resolution of 0.2°. The left eye was recorded all participants. The experiment was projected onto a thin cotton screen using a TLP-T3 LCD data projector (Toshiba Corporation, Fukaya, Japan) positioned above the participants’ head in 60 Hz mode (1024 × 768 pixels). The size of the projection was 57.5 × 43 cm. At 0.25 cm behind the screen, 8 active speakers (mono, full range) were aligned with the visual display to present the auditory stimuli. The stimulus computer and speakers were linked by a Fast Track Ultra 8R USB audio interface (M-Audio, Irwindale, California, United States). The participants’ head was stabilized using a chinrest at 73 cm from the display. The experiment was performed in a dimly lit, sound-attenuated-room of 2 × 4 × 2.6 m. The average background noise was 46 dBA. The experiment was programmed in C++ using Microsoft Visual C++ Express Edition 2006. Fig. 1 shows the experimental setup.

Fig. 1

The experimental setup showing a front view (left picture) of the projection screen, projector, eye tracker, participant seat and eye tracker; and a rear view (right picture) of the speaker positioning.

Sound localization pilot

Saccade metrics cannot be influenced if the spatial location of the sound source cannot not be inferred. Therefore, prior to the experiment, we ran a sound localization pilot experiment1 to ascertain whether participants were able to determine the location of the different sound sources in our setup (see Footnote 1 for a description).

Stimuli, design and procedure

At the center of the projection, at approximately eye-level, a medium gray plus sign (1.35° × 1.35°, 61.8 cd/m2) was displayed on top of a dark gray background (3.8 cd/m2). The plus sign served as a fixation point. After a random interval of 500–1000 ms a target and a distractor were presented for 500 ms. The target was a light gray filled circle (1° × 1° visual angle, 167 cd/m2). The distractor could be a visual distractor equal to the target, an auditory distractor consisting of a burst of broadband noise (150 Hz–18 kHz) at 62 dB(A) or a combination of the two. Additionally, to be able to determine a baseline for all measurements, the main experiment contained a No Distractor condition in which only a target was presented.

The target was always presented at an eccentricity of 14.3° visual angle from fixation, either above or below the fixation cross. The distractor was presented either close to the target (20° angular distance) or remote from the target (60° angular distance) and always at the same eccentricity as the target (14.3° visual angle, see Fig. 2). The visual and auditory stimulus presentation of the audiovisual distractor were in spatial (<0.25° visual angle) and temporal (±3 ms) alignment. Target and distractor location were randomized with the constraint that the target and distractor always appeared in the same, upper or lower, hemifield. After each trial all stimuli were removed from the screen. Unimodal and multimodal trials were randomized within a block of trials. Fig. 3 shows the procedure for the experiment.

Fig. 2

Possible target and distractor locations. Target and distractor were always presented in the same (upper or lower) hemifield.

Fig. 3

Schematic representation of the trial sequence of the experiment with three possible distractors.

Participants were instructed to fixate on the central fixation cross and make a fast and accurate eye movement to the target as soon as it appeared, while ignoring all distractors. The experiment consisted of a single session of 768 experimental trials. Each of the six experimental conditions (Visual Close, Visual Remote, Auditory Close, Auditory Remote, Audiovisual Close and Audiovisual Remote) was presented an equal number of times (96 trials per condition). The No Distractor baseline condition was presented 192 times. The conditions were intermixed and randomized across trials. The experiment started with a nine-point grid calibration sequence and 32 practice trails. At the start of each trial, fixation offset was measured to ascertain correct fixation by the participant on the central fixation cross.

Data analysis

Saccade endpoint deviation was defined as the angular distance (φ) between the saccade endpoint and the target in degrees (see Fig. 4a). For each trial, the first saccade after target onset was analyzed. Saccade endpoint deviations were calculated relative to the target and distractor positions and collapsed across the upper and lower hemifields. The second measure of saccade metrics that was analyzed was saccade trajectory deviation. Saccade trajectory deviation was defined as the angular distance (φ) between the mean of the saccade trajectory from start to end of the saccade and the straight line from saccade start to target (see Fig. 4b). Saccade endpoint deviation and trajectory deviation towards the distractor was marked a positive and endpoint deviation away from the distractor was marked as negative. Saccade amplitude was defined as the distance between the start of the saccade and the saccade endpoint in degrees of visual angle. Saccade latency was defined as the interval between target onset and the initiation of the first saccadic eye movement.

Fig. 4

Schematic of the definition of saccade endpoint deviation (a) and saccade trajectory deviation (b). Saccade endpoint deviation (φ) is the angular distance between the participants baseline route (- - -) from the start of the saccade (x) to the target location (T) and the direct route (….) from the start of the saccade to the end of the saccade (+). Saccade trajectory deviation (φ) is the angular distance between the mean location of the saccade trajectory sample points (●) and the direct route from the start of the saccade and the target location (- - -). The participants mean trajectory deviation in the No Distractor condition is used as a baseline (β) and, depending on its relative location, added to or subtracted from the trajectory deviation of each trial (α). In case of the presence of a distractor a shift of the endpoint towards the distractor (D) is positive (φ) and a shift away from the distractor is negative (−φ).

Trials with a saccadic latency of less than 80 ms were excluded because they are regarded as anticipatory and are not a response to the targets’ appearance (for a discussion of the cut-off values see Wenban-Smith & Findlay, 1991). Also, trials with a latency of more than 400 ms were excluded from the analysis because they are too slow to be influenced by the distractor (Heeman et al., 2014). Trials were excluded when the endpoint deviation or amplitude of the saccade exceeded two and a half standard deviations from the participants’ mean, as they were regarded as outliers. Each participants’ mean endpoint deviation, trajectory deviation and saccade latency as measured in the No Distractor condition served as a baseline. Individual baselines were subtracted from the mean endpoint deviation, trajectory deviation and saccade latency of the participant.

First, we analyzed saccade metrics by conducting a repeated measures 2 × 3 analysis of variance (ANOVA) on both the Saccade endpoint deviation and Saccade trajectory deviation. Distractor Location (Close, Remote) and Distractor Modality (Visual, Auditory, Audiovisual) were used as factors. Since we were interested in the direction of the effects (i.e. deviation away from vs. deviation towards the distractor), six post-hoc two-tailed one-sample t-tests were used to test both the baseline corrected mean saccade endpoint deviation and mean saccade trajectory deviation against 0 (which indicates no shift). We corrected for multiple comparisons using the Holm-Bonferroni correction method (i.e., correcting for three comparisons in the close and three in the remote distractor condition). In so doing we were able to determine whether shifts were in line with the hypothesis that close distractors cause deviation towards the distractor and remote distractors cause deviation away from the distractor (Van der Stigchel et al., 2006). Subsequently, we compared the saccade endpoint deviations within the close and within the remote distractor condition. Six post-hoc two-tailed paired-sample t-tests (Holm-Bonferroni corrected) were used to determine whether the baseline corrected mean saccade endpoint deviations and mean saccade trajectory deviations in the unimodal and the bimodal conditions differed from each other. Also, the precision of the saccades was tested since the extent of the deviation might be paired with the spread or dispersion of endpoints. Saccade endpoint dispersion was expressed in standard deviation of saccade endpoint deviation. Thereby a repeated measure 2 × 3 ANOVA tested the baseline corrected dispersion (i.e. standard deviation) of the saccade endpoint deviation with Distractor Location (Close, Remote) and Distractor Modality (Visual, Auditory, Audiovisual) as factors. Significant effects were explored with paired sample t-tests (Holm-Bonferroni corrected).

Second, we analyzed the temporal properties of the saccade latency. A repeated measures 2 × 3 ANOVA was run with Distractor location (Close, Remote) and Distractor Modality (Visual, Auditory, Audiovisual) as factors. We showed the relative of influence of each experimental condition was by performing three post-hoc two tailed one sample t-tests against 0, one on each baseline corrected condition, and three two tailed paired-sample t-test comparing the three different conditions. All p-values were Holm-Bonferroni corrected.

Finally, we investigated whether any deviation effects could be explained by latency or by different latency characteristics induced by the modality of the distractor. We isolated the latency range that contained trials for all distractor modalities and compared the faster half of the trials to the slower half of the trials with a 4 × 2 repeated measures ANOVA with Distractor Modality (No distractor, Visual, Auditory, Audiovisual) and Latency (Short, Long) as factors.

In cases where Mauchly’s test of sphericity indicated a violation of the assumption of sphericity for repeated measure ANOVAs we used the Greenhouse-Geisser correction to correct the degrees of freedom and the p-values. P-values resulting from the ANOVA’s were Bonferroni corrected. The p-values of all t-tests were transduced into adjusted p-values to compensate for multiple comparisons according to the Holm-Bonferroni method for multiple comparisons (Holm, 1979). This method of correcting accounts for the fact that measurements in different conditions are not independent of each other and requires a conservative correction for the smallest p-value while a more liberal correction is applied to the largest p-value.

Results

Based on the exclusion criteria 18.8% of the trials were excluded.

Saccade endpoint deviation

The repeated measures ANOVA revealed a main effect of Distractor Location (F(1,8) = 24.720, p = 0.001, ηp2 = 0.755). The main effect of Distractor Modality was also significant (F(2,16) = 7.145, p = 0.006, ηp2 = 0.472). Furthermore, there was an interaction between Distractor Location and Distractor Modality (F(1.198,9.584) = 11.413, p = 0.006, ηp2 = 0.588, ε = 0.599). Fig. 5 shows the results of the saccade endpoint deviation analysis.

Fig. 5

Baseline corrected saccade endpoint deviation in the conditions with a Visual, Auditory or an Audiovisual distractor. A positive error indicates a shift towards the distractor, a negative error indicates a shift away from the distractor. Asterisks indicate a significant shift in saccade endpoint deviation from zero in the remote audiovisual, the close visual and close audiovisual condition and a significant difference between the close visual and close audiovisual condition. Dashed line is the participants baseline indication no saccade endpoint deviation in either direction. Error bars indicate standard deviation.

Post-hoc two-tailed t-tests (test-value 0) showed that in the presence of a Close distractor, saccade endpoints in the Audiovisual condition (M = 1.67°, SD = 0.98) deviated significantly towards the distractor (Audiovisual: t(8) = 5.113, p = 0.001,  = 0.006, d = 2.41) while saccade endpoints in the Visual condition (M = 0.95°, SD = 0.93) did not produce a significant endpoint deviation (Visual: t(8) = 3.068, p = 0.015,  = 0.06, d = 1.445). In the presence of a Remote distractor the Audiovisual distractor (M = −0.55°, SD = 0.44) evoked a significant deviation away from the distractor (t(8) = 3.692, p = 0.006,  = 0.03, d = 1.768). None of the other conditions: Close Auditory, Remote Auditory and Remote Visual produced significant saccade endpoint deviations (Close Auditory: M = 0.03°, SD = 0.37, t(8) = 0.260, p = 0.802,  = 1, d = 0.183; Remote Auditory: M = 0.06°, SD = 0.27, t(8) = 0.691, p = 0.509,  = 1, d = 0.488; Remote Visual: M = −0.41°, SD = 0.58, t(8) = 2.143, p = 0.064,  = 0.192, d = 1.515).

A two-tailed paired sample post-hoc t-test between saccades in the presence of a Close Visual distractor and those in the presence of a Close Audiovisual distractor showed that the observed difference in endpoint deviation was indeed significantly more towards the audiovisual distractor (t(8) = 4.796, p = 0.001,  = 0.004, d = 0.80).

However, the difference between the endpoint deviation in the Remote Audiovisual condition and the Remote Visual condition (M = −0.42°, SD = 0.58) was not significant (t(8) = 0.826, p = 0.432,  = 0.432, d = 0.29).

The results consistently showed the standard global effect for close visual distractors and a larger distractor effect for bimodal distractors, also in the remote location. The unimodal auditory distractors did not influence saccade endpoints, regardless of location.

With respect to the saccade endpoint dispersion, an ANOVA with Distractor Location (Close, Remote) and Distractor Modality (Visual, Auditory, Audiovisual) as factors showed no effect of Distractor Location (F(1,8) = 0.39, p = 0.549, ηp2 = 0.047), indicating that saccade endpoint dispersion did not differ between the Close and Remote conditions. The ANOVA did show an effect of Distractor Modality (F(2,16) = 7.634, p = 0.005, ηp2 = 0.488) and an interaction between Distractor Location and Distractor Modality (F(2,16) = 4.584, p = 0.027, ηp2 = 0.364). Fig. 6 shows the results of the saccade endpoint dispersion analysis. Post hoc analysis with a two-tailed one sample t-test showed that none of the conditions differed from the No Distractor baseline. This indicates that the presence of a distractor, regardless of its modality did not change the saccade endpoint dispersion compared to conditions in which no distractor was present. The interaction is explained by the smaller saccade endpoint dispersion in the Close Auditory condition (M = −0.06, SD = 0.27) compared to the Close Visual condition (M = 0.42, SD = 0.48) as was revealed by a post-hoc analysis with paired sample t-tests (t(8) = 4.777, p = 0.001,  = 0.006, d = 0.45). None of the other comparisons reached significance: differences in saccade endpoint deviation in the bimodal condition were not paired with differences in saccade endpoint dispersion.

Fig. 6

Baseline corrected saccade endpoint dispersion in standard deviations for all three modalities in the Close and the Remote condition. The dispersion of the no distractor condition served as a baseline (- - -). Error bars indicate standard deviation.

Saccade trajectory deviation

The ANOVA on the Saccade trajectory deviation with Distractor Location (Close, Remote) and Distractor Modality (Visual, Auditory, Audiovisual) as factors showed that both the effects of Distractor Location (F(1,8) = 37.064, p < 0.001, ηp2 = 0.822) and the effect of Distractor Modality were significant (F(2,16) = 4.552, p = 0.027, ηp2 = 0.363). Furthermore, the interaction was also significant (F(2,16) = 14.102, p < 0.001, ηp2 = 0.638). Fig. 7 shows the results of the saccade trajectory deviation analysis.

Fig. 7

Baseline corrected saccade trajectory deviation in the conditions with a Visual, Auditory or an Audiovisual distractor. A positive error indicates a shift towards the distractor, a negative error indicates a shift away from the distractor. Dashed line is the No Distractor condition that served as a baseline (- - -). Error bars indicate standard deviation.

Post-hoc two-tailed t-tests (test-value 0) showed that in the presence of a Close distractor, saccade trajectories in the Audiovisual condition (M = 1.9°, SD = 1.34) deviated significantly towards the distractor (t(8) = 4.248, p = 0.003,  = 0.012, d = 3). In the presence of a Remote distractor both the Visual distractor (M = −0.95°, SD = 0.55) and the Audiovisual distractor (M = −0.98°, SD = 0.3) evoked a significant deviation away from the distractor (Remote Visual: t(8) = 5.222, p = 0.001,  = 0.005, d = 3.69; Remote Audiovisual: t(8) = 9.633, p < 0.001,  < 0.001, d = 6.81). None of the other conditions: Close Visual, Close Auditory, Remote Auditory and Remote Visual produced significant saccade trajectory deviations (Close Visual: M = 0.85°, SD = 1.25, t(8) = 2.041, p = 0.076,  = 0.228, d = 1.44; Close Auditory: M = 0.02°, SD = 0.32, t(8) = 0.198, p = 0.848,  = 1, d = 0.14; Remote Auditory: M = 0.01°, SD = 0.46, t(8) = 0.064, p = 0.95,  = 1, d = 0.05).

A two-tailed paired sample post-hoc t-test between saccade trajectory deviations in the presence of a Close Visual distractor and those in the presence of a Close Audiovisual distractor showed that the observed difference in trajectory deviation shifted significantly more towards the audiovisual distractor (t(8) = 5.047, p = 0.001,  = 0.003, d = 3.57).

Saccade latencies

The ANOVA with Distractor Location (Close, Remote) and Distractor Modality (Visual, Auditory, Audiovisual) as factors showed a main effect of Distractor Location (F(1,8) = 15.024, p = 0.005, ηp2 = 0.653). Fig. 8 shows the results of the saccade latency analysis. A post-hoc paired sample t-test showed that saccades in the presence of a Close distractor were initiated faster (M = 215.8 ms, SD = 10.35) than saccades in the presence of a Remote distractor (M = 220.0 ms, SD = 11.15; t(8) = 4.113, p = 0.003, d = 0.39).

Fig. 8

Saccade latencies per condition. Dashed line is the No Distractor condition that served as a baseline (- - -). Error bars indicate standard deviation.

The results also showed a main effect of Distractor Modality (F(2,16) = 120.993, p < 0.001, ηp2 = 0.938). Post-hoc paired sample t-tests showed that saccade latencies in the Visual condition (M = 230.5 ms, SD = 10.62) were the longest of all three conditions (Visual vs Auditory: t(8) = 17.261, p < 0.001,  < 0.001, d = 2.505; Visual vs Audiovisual: t(8) = 9.978, p < 0.001,  < 001, d = 2.051). Although the absolute difference in saccade latency in the presence of an Audiovisual distractor (M = 208.7 ms, SD = 10.64) and an Auditory distractor (M = 204.1 ms, SD = 10.46) was small, the difference was significant (Auditory vs Audiovisual condition: t(8) = 2.817, p = 0.023,  = 0.023, d = 0.436). Therefore, saccade initiation in the unimodal Auditory condition was the fastest of all three distractor modalities.

Saccade latencies in all conditions that included an auditory signal (Auditory and Audiovisual) were initiated faster than saccades in trials without an auditory signal as is shown by the post-hoc paired sample t-tests testing the No Distractor condition against both auditory conditions (M = 228.1 ms, SD = 11.56; No vs Auditory: t(8) = 9.799, p < 0.001,  < 0.001, d = 2.177 and No vs Audiovisual: t(8) = 6.809, p < 0.001,  < 0.001, d = 1.746). Consequently, saccade latency in the presence of a unimodal Visual distractor did not differ from the No Distractor baseline (t(8) = 1.111, p = 0.299,  = 0.299, d = 0.216).

There was no interaction between Distractor location and Distractor Modality (F(2,16) = 0.466, p = 0.091, ηp2 = 0.091).

Saccade endpoint deviation by latency

We explored the saccade endpoint deviation of trials in the latency range that was covered in all conditions. A repeated measures ANOVA with Distractor Modality (No distractor, Visual, Auditory, Audiovisual) and Latency (Short, Long) as factors of the Close condition showed a significant effect of Distractor Modality (F(1.347,10.776) = 9.503, p = 0.007, ηp2 = 0.543, ε = 0.449). This means that differences in saccade endpoint can be contributed to the differences in distractor modality. There was no effect, however, of short or long Latency (F(1,8) = 4.099, p = 0.078, ηp2 = 0.339) and no interaction (F(3,24) = 1.693, p = 0.195, ηp2 = 0.175). In the Remote condition we again saw an effect of Distractor Modality (F(3,24) = 3.139, p = 0.044, ηp2 = 0.282) but no effect of Latency and no interaction (Latency: F(1,8) = 1.494, p = 0.256, ηp2 = 0.157; Interaction: F(3,24) = 1.607, p = 0.214, ηp2 = 0.167). These results confirm that the differences in saccade endpoint deviation are explained by the modality of the distractor and not by the differences in time course between conditions.

General discussion

When executing an eye movement to a target, an irrelevant distracting stimulus can influence the endpoint, the trajectory and the latency of the saccade. The aim of the current study was to investigate the interference of an irrelevant bimodal distractor on oculomotor selection as opposed to unimodal distractors. The task of the observer was to make a fast and accurate eye movement to a target while ignoring any distractors. We used saccade endpoint deviation, saccade trajectory deviation and saccade latency as measures of the outcome of oculomotor competition between target and distractor. Overall, our results show that a bimodal distractor triggered larger saccade endpoint deviation and saccade trajectory deviation than a unimodal distractor. This result was not caused by a spatio-temporal trade-off or differences in saccade endpoint dispersion between the conditions. We conclude that the combined auditory and the visual information in the bimodal distractor condition indeed increased the influence of the distractor on the saccade endpoint deviation and saccade trajectory deviation and evoked stronger oculomotor competition compared to a unimodal distractor.

The direction of the interference was dependent on the spatial layout of the audiovisual scene. A bimodal distractor close to the target resulted in a saccade endpoint deviation and a larger saccade trajectory deviation towards the distractor whereas a bimodal distractor remote from the target resulted in a saccade endpoint deviation and a larger saccade trajectory deviation away from the distractor. The interfering effects of the unimodal distractor are small. We did not find a global effect for unimodal visual distractors. We attribute this to the fact that there are only two target locations. This make the location of the target very predictable and the oculomotor vector towards these locations very strong (He & Kowler, 1989). For remote unimodal distractors, we found no influence on the saccade endpoint. This is not surprising considering the large angular distance (60°) between target and distractor. The bimodal close and remote distractor, however, did influence the saccade metrics which indicates that adding a second modality to the distractor causes stronger interference.

Our observation that the bimodal distractor, both close and remote, caused a larger saccade endpoint deviation as opposed to the unimodal distractor implicates that the added modality of sound lifted the strength of the distractor and in the case of the remote distractor even above the response threshold. The weighted average account states that an eye movement is made to the location of the weighted average of oculomotor activity of two stimuli (Tipper et al., 1997). More salient stimuli (e.g. larger, brighter) have a stronger weight in the averaging (Deubel et al., 1984, Findlay, 1982). This provides an explanation as to why a close bimodal stimulus in the current study could cause more interference on the oculomotor system than the unimodal stimuli. The underlying mechanism that resulted in the increased interference by the remote bimodal distractor does not have such an unequivocal explanation. It has been suggested that deviation away is the result of strong inhibition (Van der Stigchel et al., 2006). Therefore, it seems that the bimodal distractor required stronger inhibition than the unimodal distractors. The appearance of a distractor triggers an inhibitory effect which influences the programmed vector towards the target by shifting it away from the direct path to the target. Therefore, within this framework, if inhibition is strong enough as triggered by the bimodal distractor, the resultant vector is shifted away from the distractor whereas a weaker inhibition as triggered by the unimodal distractor does not have this effect.

One might argue that the effect of distractor modality can be explained by the fact that conditions that included an auditory signal were almost 30 ms faster than visual only conditions and that faster saccades experience more distractor interference. Therefore the analysis of short and long latency saccades that only included saccades in the latency range in which all conditions were present provides valuable insights into this issue. From the latency (long, short) by distractor modality analysis it is apparent that the bimodal distractor has a larger effect on the saccade endpoint compared to the unimodal visual distractor. Although the larger mean saccade endpoint deviation in the audiovisual condition is biased by the speed-accuracy trade-off, it is not the case that this explains the entire added effect of the bimodal distractor as the bimodal distractor effect remains when these fastest trials are excluded from the analysis. Even though latencies in the auditory present trials were shorter than in auditory absent trials, the results of the latency by deviation exploration show that this difference does not explain the overall larger saccade endpoint deviation found in the bimodal distractor condition. This is true for both the whole latency range of the experiment as for the latency range that included trials from all conditions. We therefore conclude that a trade-off between the spatial and temporal domain does not explain the increased effects caused by the presence of the bimodal distractor.

The neural structure that is often implicated in oculomotor competition is the superior colliculus (SC) (Calvert, 2001). This is based mainly on neurophysiological findings in mammals (Chalupa and Rhoades, 1977, Finlay et al., 1978) and primates (Jay and Sparks, 1984, Jay and Sparks, 1987). As part of the neuronal architecture of the SC, some neurons are modality specific and a subset of different neurons respond stronger when they are excited by multiple modalities (Frens and Van Opstal, 1998, Stein and Meredith, 1993). Triggering the multisensory neurons in the SC would therefore increase the strength of the distractor in oculomotor competition. This could be the explanation for our behavioral finding that there is an increased influence of the bimodal distractor on saccade averaging and why stronger inhibition, which resulted in deviation away, was needed in the remote bimodal condition.

An intriguing question that follows from the above mentioned results is whether these effects are the result of actual integration and neuronal co-activation of the visual and auditory stimuli. Although a pilot study provided evidence that the two stimuli could integrate, the task of the observer in this pilot study was to saccade to the bimodal stimulus, making it a highly-attended stimulus. In contrast, the task of the observer in the present experiment was to ignore the bimodal stimulus, resulting in inhibition of the bimodal stimulus. Any evidence for integration from the pilot study can therefore not be extended to the current experiment. For this reason, it is hard to argue that the visual and auditory information were actually integrated. As there was no overt response to the distractor, there is no direct evidence that way of knowing that visual and auditory information was integrated.

In the temporal domain we found the two expected temporal effects of a distractor. For one, remote distractors slow saccadic response time as opposed to close distractors as predicted by the RDE (Walker, McSorley, & Haggard, 2006). Second, in line with existing warning signal literature (Ross & Ross, 1981), the presence of an auditory signal decreased saccade latency. Ross and colleagues (1981) have shown that saccade latency is shorter when an auditory signal accompanies the visual target even when the warning signal is presented simultaneously with the target onset. This decrease in saccade latency is attributed to a warning signal effect that prepares for an upcoming event. Latency facilitation gradually declines and eventually disappears with increasing temporal disalignment of the visual and auditory stimuli (Bolognini et al., 2005, Frens et al., 1995, Stevenson et al., 2012, Van der Stoep et al., 2012).

In our study we did not, however, find an increase nor a decrease of the saccade latency in the presence of a bimodal distractor compared to a unimodal distractors. Possibly the RDE and the warning signal effect counteracted each other in such a way that any added bimodal effects were obscured. But, in light of the paradigm used in the current study, this is not an exception. Previous studies in our lab have also failed to find remote distractors effects in comparable set-ups (Van der Stigchel and Theeuwes, 2008, Van Zoest et al., 2012). This is probably the result of the highly predictable target locations. In the current study, the target was always presented directly above or below fixation. Participants were able to prepare an eye movement in advance to these two relevant target locations. This might have resulted in an absence of any interference in the temporal domain.

We did not find a reliable effect of the unimodal auditory distractor. One might argue that this was the result of a mismatch between the different afferent timings of the visual and auditory input within the oculomotor system. In the current study, however, we address the effects of concurrent sensory input coming from a single source. In previous studies it has been shown that the simultaneous presentation of auditory and visual signals induces reliable multisensory response enhancement (MRE). For instance, in one of the early papers on auditory influences on saccades it was shown that a 0 ms stimulus onset asynchrony (SOA) was effective in interfering with both saccade metrics and saccade timing, while presenting the auditory stimulus 50 or 100 ms after the onset of the visual stimulus reduced the interferences. Furthermore, single cell recordings in animals showed MRE when stimuli of different modalities were presented simultaneously (Meredith, Nemitz and Stein, 1987). In these experiments, the interval between the visual and auditory stimulus was systematically varied. Periods of maximal enhancement occurred when the response trains that were evoked in the SC by the two stimuli overlapped. This was the case when the auditory stimulus was presented from 50 ms before the visual stimulus presentation till 50 ms after the visual stimulus presentation (Wallace et al., 1996). Finally, one pivotal argument for choosing the simultaneous presentation of the visual and auditory signals in the bimodal conditions comes from a study in which the role of physical and physiological simultaneity in multisensory facilitation was investigated in humans (Leone & McCourt, 2013). It was shown that the simultaneous presentation of the different modalities always induced response enhancement.

The lack of interference by auditory distractors may be explained by spatial ventriloquism. Spatial ventriloquism is the phenomenon which causes the dominant stimulus, the visual target in this case, to ‘capture’ the spatial perception of the weaker stimulus, the auditory distractor (Alais & Burr, 2004). Spatial ventriloquism in this experiment is likely to have been facilitated because participants were instructed to saccade to the target that was always above or below the fixation cross. This instruction to focus on the target locations may have encouraged participants to pre-direct endogenous attention to those spatial locations (Posner, Snyder, & Davidson, 1980) making the visual target increasingly dominant over the auditory distractor that could appear in eight different locations (Van der Stigchel et al., 2007, Walker et al., 2006). As a result the auditory distractor is reallocated to the same location as the visual target and the oculomotor vector of the distractor no longer affects the direction resultant vector. Presenting a visual stimulus in conjunction with the auditory signal at the distractor location disrupts the spatial relocation of auditory signal to the target location which explains why the addition of an auditory signal to the visual distractor does enhance the oculomotor vector to the distractor.

To conclude, when making eye movements to a visual target, bimodal distractors located either close or remote from the target influence the saccade endpoint deviation and saccade trajectory deviation more than unimodal distractors. Furthermore, close distractors make the saccade endpoint and trajectory deviate towards the distractor and remote distractors make the saccade endpoint and trajectory deviate away from the distractor. This indicates that bimodal distractors evoke stronger oculomotor competition and that the direction of the interference ultimately depends on the spatial layout of the visual scene.