Emergence of β and γ networks following multisensory training.

Our perceptual reality relies on inferences about the causal structure of the world given by multiple sensory inputs. In ecological settings, multisensory events that cohere in time and space benefit inferential processes: hearing and seeing a speaker enhances speech comprehension, and the acoustic changes of flapping wings naturally pace the motion of a flock of birds. Here, we asked how a few minutes of (multi)sensory training could shape cortical interactions in a subsequent unisensory perceptual task. For this, we investigated oscillatory activity and functional connectivity as a function of individuals' sensory history during training. Human participants performed a visual motion coherence discrimination task while being recorded with magnetoencephalography. Three groups of participants performed the same task with visual stimuli only, while listening to acoustic textures temporally comodulated with the strength of visual motion coherence, or with auditory noise uncorrelated with visual motion. The functional connectivity patterns before and after training were contrasted to resting-state networks to assess the variability of common task-relevant networks, and the emergence of new functional interactions as a function of sensory history. One major finding is the emergence of a large-scale synchronization in the high γ (gamma: 60-120Hz) and β (beta: 15-30Hz) bands for individuals who underwent comodulated multisensory training. The post-training network involved prefrontal, parietal, and visual cortices. Our results suggest that the integration of evidence and decision-making strategies become more efficient following congruent multisensory training through plasticity in network routing and oscillatory regimes.

Introduction

The brain can infer the causal structure of its surroundings by integrating multisensory signals originating from the same physical sources, while segregating those originating from different causes (Parise et al., 2012; Parise and Ernst, 2016; Deroy et al., 2016; Kayser and Shams, 2015; Rohe and Noppeney, 2015; Cao et al., 2019). The resolution of this causal inference problem weighs in the reliability and the degree of correspondence between multisensory inputs (Roach et al., 2006; Spence, 2011; Maddox et al., 2015). In ecological settings, the temporal comodulation of sensory signals helps perceptual scene analysis: for instance, an interlocutor’s mouth movements are temporally coherent with the envelope of the acoustic speech signals providing the listener with strong binding cues for predictive inferences (Grant and Seitz, 2000; van Wassenhove et al., 2005; Schroeder et al., 2008; Nahorna et al., 2015; van Wassenhove, 2013; Maddox et al., 2015). Temporally congruent signals enhance the detectability (Van der Burg et al., 2008; Maddox et al., 2015) and the identification (Kösem and van Wassenhove, 2012; Zilber et al., 2014) of events, whereas temporally incongruent signals hinder their identification (Kösem and van Wassenhove, 2012; Maddox et al., 2015). Herein, we explored the cortical mechanisms by which the internalized temporal structure of coherent multisensory events may subsequently regulate visual (unisensory) processing.

Using magnetoencephalography (MEG), we first characterized the impact of uni- and multi-sensory training history on human brain activity when participants () performed a visual motion coherence task (Fig. 1A). The task consisted in reporting the color of the most coherent cloud of dots amongst two intermixed red and green clouds of moving dots. After initially performing the task with visual stimuli only (PRE), participants were split into three experimental groups for short individualized training during which participants were tested on four strengths of visual coherence centered on each individual’s initial discrimination threshold measured in PRE: one group performed the task with visual stimuli only (V), another one with acoustic textures spectro-temporally congruent with the most coherent visual cloud (AV) and a third one, with distracting auditory noise uncorrelated with any of the two visual clouds (CTRL). After performing the training for 20 min, all participants were again tested with visual stimuli only (POST). Behaviorally, all participants improved their perceptual discrimination with the AV group showing the largest benefits and with an initial analysis of the MEG evoked activity suggesting the implication of a large-scale brain network following training (Zilber et al., 2014).

Fig. 1

Experimental procedure and Methods. (A) Left panel: each trial started with the presentation of a fixation cross lasting 0.6–0.8 s followed by the presentation of two intermixed clouds of dots moving incoherently. One cloud was red, the other, green. After a variable delay ( s) of incoherent motion, one of the clouds (here, red) moved coherently for 1 s while the other remained fully incoherent (here, green). Seven possible strengths of motion coherence were tested; the direction and color were randomized across trials. Participants selected which of the red or green cloud was most coherent (Videos S1 and S2 in (Zilber et al., 2014)). Right panel: schematic operationalization of the motion coherence discrimination task entailing the integration of motion and color for decision-making. (B) MEG recordings were collected from 36 participants, who performed the task described in (A) in the PRE and POST blocks. Between the PRE and POST, participants were split in three experimental training groups who performed the task visually (V), with acoustic textures congruent with the most coherent cloud of dot (AV), or with auditory noise uncorrelated with the visual stimuli (CTRL). All new analyses were carried out on the PRE and POST blocks, when all participants were performing the visual task described in (A). All participants improved their behavioral scores in POST as compared to PRE blocks: the AV group showed the largest perceptual benefit followed by the V and the CTRL group (Zilber et al., 2014). Following preprocessing and source estimations, (C) univariate time-frequency analysis and (D) multivariate functional connectivity analyses were performed to provide (E) complementary insights on the oscillatory mechanisms implicated in the effect of (multi)sensory training history on unisensory processing.

With this in mind, we assessed the changes of brain activity between PRE and POST blocks, when all participants performed the unisensory task with visual stimuli only (Fig. 1B). We thus did not directly focus on the feedforward integration of multisensory features or on selective attention, both of which could only occur during the AV and CTRL training. Here, we thus do not contrast unisensory vs. multisensory processing per se but rather focus on the subsequent effects of multisensory integration on a visual only task. Nevertheless, our analytical approach builds on seminal work suggesting the implication of distinct neural oscillatory coupling within large-scale networks (Senkowski et al., 2008; Keil and Senkowski, 2018; Hipp et al., 2011). The dynamic regimes mediating the binding of multisensory information across brain regions have started being characterized (Lakatos et al., 2008; Senkowski et al., 2008; Keil and Senkowski, 2018; van Atteveldt et al., 2014), yet little is known regarding the oscillatory networks which may actively contribute to supramodal or multisensory object representations (van Wassenhove, 2013; Zilber et al., 2014; Bizley et al., 2016).

Hence, in the present work, we re-analyzed previously collected data and asked how different perceptual histories changed the functional networks hypothesized in (Zilber et al., 2014). First, initial results suggested that selective attention in this task could not be the primary cause of multisensory benefits considering that the contrasts were ran only when visual stimuli were present. Hence, we did not expect changes in the alpha (α) band network to be the major factor in possible effects of perceptual history in this experimental paradigm. Second, all groups showed behavioral improvements in the task irrespective of their perceptual training; we thus expected changes in the bottom-up (perceptual) analysis of sensory inputs, as captured by high frequency analysis (likely gamma activity, γ). Third, as the AV group improved most, we also expected a strong top-down drive in the POST compared to the PRE for this group. As current research assigns an important role to beta (β) activity in the shaping of top-down predictions and decisional values (Engel and Fries, 2010; Siegel et al., 2011; Spitzer and Haegens, 2017; Haegens et al., 2017; Bressler and Richter, 2015), beta networks were expected to be a major differential driver between the three groups.

To characterize the different oscillatory networks, we estimated oscillatory activity within, and across, experimental groups using univariate time-frequency analyses (Fig. 1C) and large-scale functional connectivity (FC) measures based on the weighted phase lag index (wPLI) (Vinck et al., 2011) (Fig. 1D). We investigated a large network including prefrontal, parietal, occipital and temporal cortices with regions orthogonally selected for their functional relevance in the task (cf (Zilber et al., 2014), see Methods). Among regions of interest were the ventro-lateral prefrontal cortex (vlPFC), a massive site of convergence for visual, auditory and multisensory information processing (Romanski, 2007; Romanski and Hwang, 2012), whose neurons selectively respond to the color of visual objects (Romanski, 2012) and low-level abstraction (Wutz et al., 2018); the intra-parietal sulcus (IPS), which plays a central role in multisensory processing (Andersen, 1997; Bolognini and Maravita, 2011; Pasalar et al., 2010) and visual motion area (MT), sensitive to perceptual changes in this task (Zilber et al., 2014). Both IPS and MT are known to interact in the β range during perceptual decision-making (Donner et al., 2009). We first started by exploring the modulation of local oscillatory activity during visual motion discrimination (Siegel et al., 2006, 2011), and followed up with the exploration of changes in functional connectivity as a function of sensory history in training.

Materials and methods

Participants

36 healthy human participants were recruited for the study (age range: 18 to 28 y. o.; mean age: 22.1 2.2 s.d.; 3 groups of 12 participants each: V: 4 females; AV: 6 females; CTRL: 6 females). All participants were right-handed, had normal hearing and normal or corrected-to-normal vision. Before the experiment, all participants provided a written informed consent in accordance with the Declaration of Helsinki (2013) and the local Ethics Committee on Human Research at NeuroSpin (Gif-sur-Yvette, France). Prior to the MEG acquisition, participants were randomly split into 3 experimental groups (V, AV, and CTRL) as detailed below.

Task

The MEG experiment consisted of interleaved MEG blocks alternating between rest and task. The first resting block occurred prior to any task or training and will be thereafter referred to as REST. REST was used as baseline for functional connectivity analysis. The six task blocks included: a 12 min pre-training block (PRE) consisting of the visual coherence discrimination task; a 20 min training (4 successive blocks of 5 min each) on the same task using purely visual stimuli (V group), congruent audiovisual stimuli (AV group) or incongruent audiovisual stimuli (CTRL group); a 12 min post-training block (POST) consisting of the same visual coherence discrimination task as in PRE. Thus, the PRE and POST blocks consisted of the same visual only coherence discrimination task for all three experimental groups and using the exact set of visual stimuli. Only the training was either visual or audiovisual. The task requirements in PRE, training, and in POST were otherwise identical in all runs: two clouds of colored dots were intermixed on the screen and participants had to tell which of the red or green cloud of dots was the most coherent. In PRE and POST, participants also rated their confidence on a scale of 1–5 after they provided their main response regarding the color of the most coherent cloud of dots.

In PRE and POST, the initial and final motion coherence discrimination threshold of each participant was assessed by testing seven strengths of visual motion coherence (15, 25, 35, 45, 55, 75 and 95). 28 trials for each strength of visual motion coherence were collected in PRE and in POST for a total of 196 trials in each block. In the training (4 blocks, 5 min each), four visual coherence levels were tested corresponding to and of an individual’s discrimination threshold computed in PRE (see (Zilber et al., 2014) for more details). 28 trials for each strength of visual motion coherence were presented for a total of 112 trials in a given training block. These data were not considered as our main question focused on contrasting brain activity to identical experimental conditions given a different training history. Further experimental details can be found in (Zilber et al., 2014).

To localize the visual motion area, we used a passive MEG localizer (120 trials) after POST. Participants were presented with a fully incoherent visual cloud lasting 0.5 s and followed by either a highly coherent ( of coherence) or an incoherent ( of coherence) interval of 1 s (60 trials each). During the localizer, participants were asked to passively view the visual motion stimuli.

Stimuli

Visual stimuli consisted of intermixed red and green clouds of dots (Fig. 1A) calibrated to isoluminance using heterochromatic flicker photometry on a per individual basis prior to MEG data acquisition. A white fixation cross was at the center of a gray mask disk and dots were presented within an annulus of to of visual angle. Dots had a radius of . The motion flow was 16.7 dots per deg × 2 s with a speed of s and its direction confined within an angle of around the azimuth. of the trials were upward coherent motion and the remaining of the trials were downward coherent motion. The color and the direction of the most coherent cloud of dots were thus pseudo-randomized across trials, and both the color and the direction of dots were orthogonal to the task goal.

The V group underwent training using visual only stimuli. The AV group underwent training using temporal comodulated audiovisual associations comparable to those used in sensory substitution devices such as the vOICe (Meijer, 1992) and the EyeMusic (Levy-Tzedek et al., 2012), with intuitive perceptual associations between sensory modalities (Melara and O’brien, 1987; Maeda et al., 2004). Here, we used parametric sounds or acoustic textures (cf (Overath et al., 2010) with sampling frequency kHz, frequency range: 0.2–5 kHz) which enabled to pair each visual dot with a linear frequency-modulated acoustic sweep whose slope depended on the direction taken by the visual dot (see (Zilber et al., 2014) for more details). The maximal slope was 16 octaves/s corresponding to motion directions of . A visual dot moving upwards was associated with an upward acoustic ramp, whereas a downward moving dot was associated with a descending acoustic ramp. The duration of a ramp was also identical to the life-time of a visual dot. The CTRL group underwent training with acoustic noise of the same duration and amplitude as the acoustic textures used for the AV group. Unlike acoustic textures in which the dynamical properties of the fine spectral acoustics were matched with the dynamical properties of the visual dot motion, the acoustic noise used for the CTRL group was fully uncorrelated with the visual coherent motion. This served as a control so that participants trained with audiovisual signals could either hear a sound designed to be temporally predictive of visual coherence (under the temporal comodulation hypothesis, automatic mapping between the spectral coherence in acoustics and visual motion coherence; AV group) or a random acoustic noise (the lack of spectral coherence in the acoustics could not map on visual motion coherence and may act as a distractor). In sum, the CTRL group was included to test the specificity of audiovisual associations in this task and the benefit of temporal comodulation in audiovisual training.

In the task and for all experimental groups, a given trial started with a variable duration (0.3 to 0.6 s) mixing both red and green clouds of dots being fully incoherent (0 of coherent motion). Then, one cloud of dots became more coherent than the other for a duration of 1 s. In PRE and POST, the coherence level taken by the most coherent cloud was one of seven possible values described in the Task section. During training, the coherence level taken by the most coherent cloud took one of four values described in the Task section. Inter-trials intervals (ITI) varied from 0.6 to 0.8 s. Samples of the video trials can be experienced (Movies S1 and S2 in Zilber et al., 2014).

MEG and MRI data acquisition

Electromagnetic brain activity was recorded in a magnetically shielded room using a 306 MEG system (Neuromag Elekta LTD, Helsinki). MEG signals were sampled at 2 kHz and band-pass filtered between 0.03 and 600 Hz. Four head position coils (HPI) were used to measure the head position of participants before each block; three fiducial markers (nasion and pre-auricular points) were used during digitization as a reference for coregistration of anatomical MRI (aMRI) immediately following MEG acquisition. Electrooculograms (EOG) and electrocardiogram (ECG) were recorded simultaneously with MEG. Five minutes of empty room recordings were acquired before each block for the computation of the noise covariance matrix.

The T1 weighted aMRI was recorded using a 3-T Siemens Trio MRI scanner. Parameters of the sequence were: field-of-view: 256256176 mm3 (transversal orientation), voxel size: mm; acquisition time: 466 s; echo time ms, inversion time ms, repetition time ms and flip angle (FA): . For each participant, cortical reconstruction and volumetric segmentation of T1 weighted aMRI was performed using FreeSurfer.1 Once cortical models were complete, deformable procedures were executed using the MNE software (Gramfort et al., 2014) to register source estimates of each individual onto the FreeSurfer average brain for group analysis.

MEG preprocessing

The analysis of the MEG data was carried out using the MNE-python toolbox (Gramfort et al., 2013). After applying an anti-aliasing FIR filter (low-pass cutoff frequency at 130 Hz), MEG data were down-sampled to 400 Hz, and preprocessed (Fig. 1B) to remove external and internal interferences, in accordance with accepted guidelines for MEG research (Gross et al., 2013). Signal Space Separation (SSS) was applied with MaxFilter to remove exogenous artifacts and noisy sensors (Taulu and Simola, 2006). Ocular and cardiac artifacts (eye blinks and heart beats) were removed using Independent Component Analysis (ICA) on raw signals. ICA were fitted to raw MEG signals, and sources matching the ECG and EOG were automatically found and removed before signals reconstruction following the procedure described in (Gramfort et al., 2014).2 On average, and over the 36 participants: 39.149.91 components were extracted and 3.250.86 components were zeroed out for the REST conditions; 418.79 components were extracted and 4.301.39 components were zeroed out for TASK.

Univariate time-frequency analysis in sensor space

Briefly, to identify significant changes in oscillatory activity associated with task performance and task improvements, we performed non-parametric cluster-level 1 sample t-tests for each frequency band. Second, in PRE, we performed non-parametric cluster-level paired t-test on time-frequency epochs (single-trial analysis), contrasting high and low motion coherence as well as correct and incorrect trials. Third, we performed non-parametric cluster-level paired t-tests on time-frequency epochs, contrasting brain activity to the same stimuli in PRE and POST blocks. The details of each statistical test is provided below.

The oscillatory activity in α, β and γ ranges was established using a univariate time-frequency analytical approach in sensor space (Fig. 1C). Oscillatory activity in the post-stimulus period was contrasted with the pre-stimulus period. Both the duration of the initial incoherent portion of stimuli (300–600 ms) (Fig. 1A) and the decision time reflected in reaction times (RTs) were variable. As such, we locked the epochs according to three different events in the sequence of stimuli, each relevant for our ad-hoc working hypotheses and inherent to our experimental design. A first epoching ranged from ms to ms post-incoherent motion onset thus fully capturing the incoherent portion of the stimuli. The second epoching focused on the brain activity following the onset of motion coherence per se and ranged from 0 ms to ms post-coherent motion onset. The third epoching focused on the decision-making analysis and was anchored on RTs from to ms around the button press (RT). For all three sets of epochs, the 600 ms interval preceding the incoherence onset served as baseline activity.

For each set of epochs, a group-level non-parametric spatio-temporal cluster analysis was computed on single-trial time-frequency transforms obtained with Morlet complex-valued wavelets and averaged in each frequency band of interest. The number of cycles in the Morlet wavelet was defined for each frequency (f) as . To assess the statistical significance of the obtained clusters we randomly flipped times the sign of the time-frequency transformed data, and our cluster-level correction for multiple comparisons was based on the maximum statistic method (Maris and Oostenveld, 2007). The spatio-temporal clustering was used to identify the sensors showing significant event-related activity in PRE and in POST. One of these sensors was used for subsequent univariate analyses to ensure that any inference made on a particular frequency band was first determined independently of our ad-hoc working hypothesis motivating the subsequent contrasts. Specifically, time-frequency cluster analyses were used in PRE to perform group-level statistics (, low vs. high MC, and correct vs. incorrect trials). Moreover, a statistical contrast was performed between POST and PRE blocks, pooling all participants together () as well as considering each group separately (). Statistical significance for all these contrasts was assessed using random permutations as discussed above.

To evaluate the extent to which oscillatory activity could significantly contribute to the observed behavioral measures (performance, RTs, confidence) and stimulus parameters (strength in visual motion coherence), we used a post-hoc general linear model (GLM) on single trials over the time intervals found to be significant in our cluster analyses. The non-parametric approach to GLM based on random permutations was employed to obtain a robust and unbiased linear regression (Anderson and Legendre, 1999; DiCiccio and Romano, 2017). The GLM to test the linear regression between oscillatory power and the strength of visual motion coherence and different behavioral parameters followed the equation . Here, was the MEG mean power in a significant cluster of sensors; was the vector containing p regressor variables. To find the best fitting model, we tested different combinations of regressors including motion coherence, reaction times, correctness, confidence ratings and their interactions. Each regressor was first tested in an independent linear model, and significant explanatory variables were subsequently tested in the same model, together with their interactions in order to identify possible driving effects.

contained the p regression coefficients including the constant term, and ε was the error term. Iteratively reweighted least squares were used to obtain an estimate of and a value of the Wald statistic . A non-parametric approach based on random permutations was used to obtain robust and unbiased significance levels and confidence intervals. Specifically, to test the significance of each estimated regression coefficient , random permutations of the corresponding regressor variable were generated, yielding a distribution of Wald statistics for each partial regression coefficient under the null hypothesis . For each estimated coefficient, the p-value was calculated as the proportion of grater then or equal to , in absolute value. Permutation inference for the GLM in common neuroimaging applications has been proposed as a non-parametric test to relax assumptions on data distributions (Winkler et al., 2014). The 36 participants were pooled together in PRE () whereas group-specific analyses () were performed on POST data to study the effects of (multi)sensory training. This analysis was carried out for the three sets of epochs locked to the three different events (incoherence onset, coherence onset, response).

MRI-MEG coregistration and source reconstruction

The coregistration of MEG data with the individual anatomical MRIs (aMRI) was carried out by realigning the digitized fiducial points with the markers in MRI slices, using MRILAB (Neuromag-Elekta LTD, Helsinki) and tools within MNE (Gramfort et al., 2014). Individual forward solutions for all source reconstructions located on the cortical sheet were computed using a 3-layers boundary element model constrained by the individual aMRI. Cortical surfaces were extracted with FreeSurfer and decimated to about 5120 vertices per hemisphere with 4.9 mm spacing. The forward solution, noise and source covariance matrices were used to calculate the noise-normalized dynamic statistical parametric mapping (dSPM) (Dale et al., 2000) inverse operator (depth = 0.8). The inverse solution was obtained using a loose orientation constraint on the transverse component of the source covariance matrix (loose = 0.4). The estimates of the reconstructed dSPM time series were interpolated onto the FreeSurfer average brain for group-level source space analysis. Only the radial components of the estimated currents were considered for further analysis.

After source estimation, we proceeded by summarizing the results into regions of interest (ROIs). When selecting the ROIs, we encountered the well-known trade-off between computational tractability and signal-to-noise ratio: Too small ROIs (e.g., voxel-wise analysis) may increase the noise and at the same time exacerbate the multiple comparisons problem, while too large ROIs may suffer from signal cancellation, especially if multiple sources are captured in one ROI. In MEG and EEG source localization, additional peculiarities have to be considered. First, source reconstructed spatial maps are coarser and more blurred than in fMRI, hence potentially arguing in favor of using coarser parcellations. Second, the sign of the reconstructed signals follows the curvature of the cortex which may induce cancellation during averaging. This may distort resulting time-courses even if only one single source is captured by the ROI. In practice, ROIs are therefore often selected according to specific data analysis goals (Farahibozorg et al., 2018; Colclough et al., 2016; Khan et al., 2018) as is generally recommended for many elements of MEG and EEG analysis (Jas et al., 2018). In light of these considerations, we chose the rather coarse Desikan-Killiany parcellation (Desikan et al., 2006) from FreeSurfer that covers both hemispheres on each individual cortex with 28 ROIs.3 This set of ROIs has already been established as sufficiently sensitive in previous work from our group. For example, it has been shown to capture multisensory processing, perceptual decision making and motion discrimination (Zilber et al., 2014). To mitigate the risk of potential signal cancellation, we used a weighted averaging approach, which explicitly took into account the cortical curvature through the surface normals. The resulting ROIs covered the frontopolar regions (FP), frontal eye field (FEF), ventro-lateral prefrontal cortex (vlPFC), premotor cortex and supplementary motor region (BA6), primary motor cortex (PMC), intra-parietal sulcus (IPS), inferior temporal cortex (ITC), auditory cortex (AUD), superior temporal sulcus (aSTS, mSTS and pSTS), middle temporal visual area (MT), visual area V4, and primary and secondary visual cortices (V1–V2). The average activities over all the vertices within each of these cortical regions (labels) were used for the subsequent functional connectivity analysis.

Functional connectivity analysis

Adjacency matrices

Functional interaction between brain regions was assessed by evaluating the similarity of brain activity across remote brain areas, namely functional connectivity (FC) (Fig. 1D). Several studies have compared a subset of FC methods with respect to their ability to correctly detect the presence of simulated connectivity schemes in a multivariate data set (Ansari-Asl et al., 2006). The outcomes showed that the performance of the measures depended both on the characteristics of the dataset and the methods. No single method outperformed the others in all cases. A practical and reasonable approach thus consisted in predetermining the FC method according to the plausible ad-hoc working hypotheses of the experimental study under scrutiny. To characterize FC in the absence of a priori knowledge about its nature and the generating model, non-parametric measures could first be used.

The notion of phase coupling derives from the study of oscillatory nonlinear dynamical systems. Based on this notion, Phase Lag Index (PLI) (Stam et al., 2007) aims at quantifying in a statistical sense the phase delay between such systems from experimental data (Sazonov et al., 2009) according to the following formula:where quantifies the instantaneous phase difference between two source reconstructed time series and at time point . In Eq. (1), the expectation is typically replaced by the empirical mean over consecutive time points. PLI was shown to be robust with respect to instantaneous linear mixing effects which may lead to the detection of spurious functional couplings not caused by brain interactions (instantaneous linear mixing effects) (Stam et al., 2007).

Moreover, PLI has the advantage of not being influenced by the magnitude of phase delays. Weighted PLI (wPLI) also solves the problem of discontinuity around zero (Vinck et al., 2011), by using the magnitude of the imaginary part of the cross-spectrum as weights. To measure pairwise interactions between the extracted cortical labels, we used the definition of wPLI in the frequency domain, exploiting the phase of the Fourier-based cross-spectrum of two time series and :where stands for the imaginary part, and the expectations were replaced by their empirical estimates averaged over epochs. Here, f usually spans a specific frequency band such as oscillatory regimes (α, β or γ). Therefore, each FC observation consisted of a symmetric adjacency matrix of size . 10 instances of FC were obtained for each participant and each block performing a partition of epochs into 10 non-overlapping subsets. In order to ensure the balance of the number of epochs used to obtain each FC instance for the different participants, the total number of epochs was set to the minimum observed across participants.

Statistical analysis of FC

A widely employed approach to extract the FC network of interest from an adjacency matrix consists in applying a threshold to the strength of the estimated connections (Fig. 4A). The threshold is obtained according to a suitable criterion (De Vico Fallani et al., 2014). The resulting FC patterns correspond to the strongest connections, which do not necessarily reflect the most significant differences between experimental conditions. Additionally, while such approach is particularly suitable for graph-theoretic network analysis, it does not allow direct quantitative comparisons, owing to the variability of significant connections. Here, our goal was to separately investigate the FC changes that were task-dependent (i.e. significant changes in the contrast PRE or POST vs. REST) and the cortical interactions subsequent to (multi)sensory training in each group. Hence, the comparison between FC estimates obtained for the three experimental groups (V, AV, CTRL) was addressed using a different approach. First, adjacency matrices were separately averaged over each frequency band of interest, each block (REST, PRE and POST) and each participant (Fig. 4A). Second, for each frequency band and each experimental group (V, AV and CTRL), the task-relevant networks were extracted by performing a group-level permutation t-test between FC estimated in REST and FC estimated in task blocks (PRE, POST) (Fig. 4B). Third, considering only the subset of task-related connections common to PRE and POST blocks (i.e. the connections significantly changing both in PRE and POST as compared to REST), the variability driven by the perceptual history training (POST vs. PRE) was evaluated using a permutation t-test (Fig. 4C, top).

Fig. 4

Overview of statistical contrasts performed to extract functional oscillatory networks. (A) Functional connectivity (FC) estimates during the presentation of coherent motion, in PRE and in POST, contrasted with resting-state FC patterns (REST). For illustration, we report the full and thresholded α oscillatory networks separately for PRE, REST and POST (top to bottom panels, respectively). Statistical contrasts were based on non-parametric permutation t-tests and performed on the 28 cortical regions. (B) Pairwise phase couplings were contrasted to show significant differences of weighted Phase Lag Index (wPLI) values characterizing the task-related FC network (PRE vs. REST and POST vs. REST). (C) The FC patterns computed in (B) were compared to assess the variability of the task-related FC in PRE and in POST, as well as to characterize the appearance of new connectivity patterns in POST. FP: frontopolar; FEF: frontal eye field; vLPFC: ventro-lateral prefrontal cortex; PMC: primary motor cortex; BA6: supplementary motor cortex; IPS: intra-parietal sulcus; ITC: inferior temporal cortex; AUD: auditory cortex; aSTS: anterior superior temporal sulcus; mSTS: middle STS; pSTS: posterior STS; MT: middle temporal visual motion area; V4: visual area 4; V1–V2:primary and secondary visual cortices.

All statistical tests were corrected for multiple comparisons using the maximum statistic method (Maris and Oostenveld, 2007). Finally, the reorganization of FC in POST was addressed by highlighting the emergence of new task-relevant FC in POST, which were not observed in PRE (Fig. 4C, bottom). Importantly, this approach considered the FC at REST as the baseline for all other FC analyses. This allowed to better disentangle the different FC patterns and their changes between PRE and POST. Hence, a linear correlation analysis based on Pearson’s correlation coefficient was performed between the average increase of post-specific interactions from PRE to POST, and the corresponding increase of confidence ratings, for each frequency band and each training group separately.

Topological analysis of FC

A complementary and conventional topological analysis of FC networks was also carried out (Bullmore and Sporns, 2009) to investigate the degree of interaction between each brain region per oscillatory regime. Specifically, the networks with density threshold given by , where is the number of regions (De Vico Fallani et al., 2017), were first extracted for each participant, each block and each frequency band. The weighted node degree , a topological property which is a conceptually simple measure of centrality of a node i within a network, was then computed for each label in the extracted networks, according to the formula:where K is the number of nodes in the network (cortical labels), and is the estimated FC value between nodes i and k. Permutation t-test were performed to evaluate the differences of node degree values between REST and task blocks (i.e. PRE or POST) as well as between PRE and POST. Again, the maximum statistic method was used to correct the statistical tests for multiple comparisons (Maris and Oostenveld, 2007).

Results

We first assessed the broad-band oscillatory activity following the presentation of visual motion stimuli. For this, we combined single trials in PRE, which were evoked by all motion coherence levels in all three experimental groups (), and performed a time-frequency analysis of the MEG responses. A spatio-temporal clustering permutation test corrected for multiple comparisons (see Experimental Procedures) on post-stimulus time-frequency activity (Fig. 2A, left panel) revealed a significant decrease of α (alpha: ) and β () power (, starting 0.09 s post-incoherence onset to 0.31 s post-response; 38 sensors) with a significant increase in the power of broadband high γ (, , starting 0.04 s post-incoherence onset to 0.62 s post-response; 24 sensors) as compared to baseline. The significant clusters observed for both the sustained decrease in α power and the increase in high-frequency γ power were mostly localized in the occipital sensors (Fig. 2A, middle panel). This pattern lasted throughout the presentation of visual motion coherence. Consistent with the topographical pattern at the scalp level, source estimations of the α and the high γ responses suggested generators located in bilateral visual cortices (Fig. 2A, right panel). This time-frequency pattern during unisensory visual motion coherence was consistent with previously reported time-frequency responses induced by visual motion stimuli (Donner et al., 2007; Siegel et al., 2006). The significant increase in γ band during visual motion coherence was also consistent with a previous report of visual motion eliciting a stronger γ response than stationary visual stimuli (Swettenham et al., 2009). We then asked whether the post-stimulus power changes in α, γ, and β were linked to the strength of visual motion coherence in PRE (for all participants) and in POST (as a function of the experimental group), and then proceeded with the exploration of the β band.

Fig. 2

Occipital low-frequency suppression and motion strength (and POST-accuracy) dependent broadbandγincrease. (A) Significant occipital time-frequency clusters of low-frequency ( Hz) power suppression and γ ( Hz) band increase were found during the presentation of motion coherence (left panel). The time-frequency analysis was locked to the onset of incoherent motion (first black vertical line) and to the coherence motion onset (first white vertical line; the second white vertical is the offset) as well as response-locked (second black vertical line). The three separate analyses were stacked together to provide the full unfolding of oscillatory activity during the trial. The group average () time-frequency response of the PRE trials showed a sustained decrease of low-frequency power with an increase in broadband γ power: this is illustrated for one occipital sensor in the obtained spatial clusters (highlighted yellow sensor in the central panel; see also Inline Supp. Mat. A). Source estimates of α ( Hz) power and broadband γ revealed the implication of visual and parietal cortices (right panel). (B) In occipital sensors, we found no significant modulations of α power as a function of motion coherence in PRE (top panel) or in POST (bottom panel). Bars are 1 s. e.m. (C) High γ activity increased with motion coherence in PRE (left top panel) and in POST (left bottom). High γ activity also increased with accuracy but only in POST (right bottom). Bars are 1 s. e.m. (D) Source estimates showed a significant linear relationship between high γ and motion coherence in occipital cortices.

α suppression is independent of sensory evidence and training history

In PRE, i.e., prior to any training, we used the grand average data () and assessed changes in α power from the onset of motion coherence (Fig. 2A, left panel, white demarcation lines) as a function of the strength in motion coherence (Fig. 2B) using non-parametric statistics and a GLM. We found no significant relationships between α power and motion coherence. The same regression analysis was performed in the POST data, independently for each experimental group () in order to preserve the distinct training history of each group. Again, we found no significant relationships between α power and motion coherence, and no significant differences in α power between PRE and POST experimental blocks. Overall, we found no substantial evidence that α power varied as a function of visual motion coherence strength or perceptual history. While the absence of systematic α modulation limits the functional specificity of α in this task, the general decrease of α power notably seen in posterior sensors during the presentation of visual stimuli was generally consistent with the inhibitory gating of visual information (Jensen and Mazaheri, 2010; Zumer et al., 2014) thereby a decrease in α power could be taken as an index of selective attention (Foxe and Snyder, 2011).

Broadband high-γ power increases with the strength in visual motion coherence and post-training performance

Before training (PRE, ), the occipital broadband high γ following the presentation of visual motion coherence showed a significant increase with the strength of visual motion coherence (; Fig. 2C, left top panel). A similar analysis performed on POST data separately for each experimental group () revealed a significant linear relation between the post-stimulus γ power increase and the increase in stimulus motion coherence. This effect was seen in all three groups irrespective of training history ( , ; Fig. 2C, left bottom panel). This observation was consistent with the important role of high γ power during motion discrimination and its modulation by the strength in visual motion (Siegel et al., 2006). In PRE, no other effects or interactions were found when adding participants’ behavioral correctness (C), reaction times (RT), or confidence ratings (CR) to the GLM (see Experimental Procedures; additional information regarding behavioral outcomes provided in (see Figure 3 in Zilber et al., 2014). To the contrary, in POST, a positive interaction between correctness and motion coherence drove the regression analysis on its own (, , ). In fact, irrespective of training history, the interaction between the strength of motion coherence and participants’ performance explained the linear relationship between participants’ correctness and occipital broadband high γ power (, ; see Fig. 2C, right panels). Subsequent source estimations (see Experimental Procedures) suggested that the increased modulation of high γ band activity likely originated in visual cortices (Fig. 2D). This observation was in general agreement with previous findings linking local γ band activity to the encoding of sensory evidence (Von Stein and Sarnthein, 2000) during a visual motion discrimination task (Siegel et al., 2006, 2011). This observation also suggested that, irrespective of sensory history during training, the reliability of visual sensory evidence contributed to successful task performance.

In addition to the post-stimulus α and γ effects found in PRE (), we also observed two significant β clusters () partially overlapping over the frontal sensors during the presentation of coherent motion: a bilateral early increase in β band power (, from 0.26 s pres-coherence onset to 0.65 s post-coherence onset, 40 sensors) was subsequently followed by a significant decrease (, 21 sensors) over the left hemispheric sensors. The decrease in β band power started around 0.57 s following the onset of visual motion coherence (Fig. 3A). The same analysis performed on POST data () showed, overall, that the decrease in β power was left-lateralized and occurred more strongly over left frontal sensors. In what follows, we thus further investigate changes in β power.

Fig. 3

The modulations ofβpower activity indicate a gain in efficiency. (A) Group average () time-frequency maps during the PRE block. The analysis was separately performed on trials locked to the incoherent stimulus onset (white vertical line), locked to the coherent motion (first black vertical line) and locked to the response (second black vertical line). All three analyses were stacked for illustration and provided for a left hemispheric sensor (red dot on left topographic map). Time-frequency permutation clustering statistics revealed two β ( Hz) components partially overlapping over frontal sensors (topography and represented sensor reported on the bottom right corner) during the presentation of the coherent dot motion: a significant bilateral early increase of β power (red) was followed by a significant decrease solely over the left hemispheric sensors. (B) Statistical contrasts tested the changes in β power between correct vs. incorrect trials (top panel), high vs. low motion coherence trials (middle panel) and POST vs. PRE trials (lower panel). All three contrasts revealed a stronger decrease of β power. See also Inline Supp. Mat. B (C) Consistent with contrasts in (B), β power linearly decreased with increasing motion coherence in PRE and in POST (left top and bottom, respectively) but linearly increased with RT in PRE and POST (right top and bottom, respectively). (D) The AV group showed the strongest β power decrease from PRE (gray) to POST (red) for all strengths of visual motion coherence (left panel). Bars are 1 s. e.m. Thus, during motion coherence stimuli the strongest overall β power decrease from PRE to POST was observed for the AV group (histogram on the right). Source estimates of β power showed a significant decrease in POST as compared to PRE over (E) the motor and parietal cortices. This effect was strongest for the AV group.

Distinct β power effects

In a first working hypothesis, we considered prior work showing that changes in β power contribute to perceptual decision-making (Donner et al., 2009; Alavash et al., 2017) and can index functional inhibition during perceptual discrimination tasks engaging different sensory modalities (Cassim et al., 2001; Bauer et al., 2012). We also took into account that motor ​β ​power can be modulated by attention when anticipating motion coherence onset (Saleh et al., 2010). To test whether these affected the observed the significant β power suppression in our study, we performed cluster permutations on time-frequency data locked to the onset of motion coherence, and devised three contrasts of interest (Fig. 3B): correct vs. incorrect trials in PRE as in (Donner et al., 2009) (top panel), high vs. low motion coherence in PRE (middle panel) and PRE vs. POST trials (bottom panel). In all three contrasts, we found a significant decrease of β power so that the a priori easiest trials yielded a larger suppression of β power compared to the more difficult trials. Specifically, we found a systematic late decrease of β power in the correct vs. incorrect trials (, starting 0.28 s post-coherence onset; Fig. 3B, top) and in the high vs. low motion coherence contrast (, starting 0.22 s post-coherence onset; Fig. 3B, middle). A similar, yet longer-lasting, left-lateralized frontal β effect was found in the POST vs. PRE contrasts (, 0.08 s pre-stimulus onset; Fig. 3B, bottom).

As the decrease in β power was found locked to the coherence onset but late in the trial – i.e. just before participants’ responses –, it may have reflected the seminal β suppression preceding movement onset (Pfurtscheller and Da Silva, 1999) seen when locking epochs to the individuals’ reaction times (RTs) (Fig. 3A, second black line). To test for the possibility that the observed β suppression reflected β event-related desynchronization shaped by motor readiness and action execution (Mima et al., 1999; Jenkinson and Brown, 2011), we thus locked the trials to participants’ RT and tested the same contrasts as those performed previously on the trials locked to the onset of motion coherence (i.e., correct vs. incorrect in PRE data, high vs. low motion coherence in PRE data and PRE vs. POST). The correct vs. incorrect, and the high vs. low motion coherence response-locked contrasts did not reveal a decrease; rather they showed a small but significant increase of ​β ​power before movement preparation ( starting 0.6 s before movement onset). This pattern was only detected when locking the data to the RT (Fig. 3B first two rows on the right) and was distributed over the posterior and frontal sensors. This effect appeared to converge with previous observations (Siegel et al., 2011), in which β power was suggested to mediate stages of decision-making linking sensory evidence encoding with choice-related action execution. We did not observe significant differences when contrasting the PRE and POST activity for this effect (Fig. 3B third row on the right), and thus did not pursue the analysis of this specific effect which had been previously investigated in details (Siegel et al., 2011). Importantly however, this response-locked β activity did not seem to be shaped by sensory history in training, and the changes in β power locked to the onset of visual motion coherence (Fig. 3A and Fig. 3B) were thus considered distinct from the seminal response-locked effect.

β power is sensitive to integrated evidence during decision-making

Considering that the modulations of β power suppression were not specific to the presentation of visual motion coherence, but rather, and also, sensitive to the correctness and the type of training participants underwent, we tested whether, in the absence of a task, the same β power decrease could be seen. For this, we used the localizer data during which participants passively attended the coherent motion stimuli in the absence of a task. We contrasted brain responses elicited by the presentation of coherent motion with those obtained in response to the incoherent motion. We found no significant β power changes in this contrast, suggesting that being engaged in the discrimination task was necessary to observe the β suppression effects.

We then performed a separate regression analysis (GLM) on the PRE () and the POST data (independently for each experimental group, ) (Fig. 3C, top and bottom panels, respectively). We used the strength of motion coherence and three behavioral variables (correctness, RT, confidence ratings) as regressors. With this approach, we assessed which of the stimulus motion coherence or of the three behavioral outcomes, contributed most to the variance of the observed modulation in β power. We found that β power significantly decreased with increasing strength in motion coherence in PRE (, , ), (Fig. 3C, left top panel). We also found a significant positive interaction between motion coherence and RT (, , (Fig. 3C, right top panel);): in other words, for a given strength of visual motion coherence, we observed a decrease of β power with faster RT. Altogether, we thus observed that the strongest visual motion coherence and the fastest RT showed the lowest β power.

We then applied the same regression analysis on POST data separately for the three experimental groups (). To make the group-specific results comparable, β power from PRE data were separately subtracted from each individual group’s POST data. This analysis revealed a decrease in the slope of the regression between β power and the strength of visual motion coherence in all three experimental groups (Fig. 3C, bottom left). This was consistent with the fact that all participants improved their performance after training with increased accuracy, decreased RT, and increased confidence rating (Zilber et al., 2014). Similar to the PRE effects, we found a positive interaction between the strength of visual motion coherence and RT with β power (Fig. 3C, bottom right). Crucially, an overall decrease of β power from PRE to POST was consistently observed for all levels of visual motion coherence in the AV group as compared to the V and the CTRL groups (Fig. 3D). That motion coherence and RT were the main contributors to the β power variability was consistent with the observation that all experimental groups were faster in POST as compared to PRE. Interestingly however, that the AV group displayed the largest decrease of β power overall after training was also consistent with its overall better performance compared to the other groups (and not with a faster response as the RT were comparable across groups (Zilber et al., 2014). In other words, each group showed an overall decrease of β power as a function of the strength of visual motion coherence, which may indicate an overall gain in stimulus processing efficiency as the regression slopes across groups were comparable (Fig. 3D, left panel). Additionally, this decrease was shifted down for the AV group as indicated by the histogram in Fig. 3D, which shows the overall difference (mean and s. e.m. over subjects) of β power between POST and PRE for each group, separately. Finally, the performance on the task showed a significant correlation with β power but only when using an independent linear regression model, suggesting that motion coherence and RT contributed most to the β power effects, which in turn affected performance.

To sum up our observations on β power locked to the onset of visual motion coherence: the task-related decrease in β power over the frontal sensors got generally stronger with integrated evidence to perform the task. Additionally, congruent multisensory training (AV) induced a larger (POST-PRE) decrease of β power than other unisensory (V) or conflicting audiovisual (CTRL) trainings. Consistent with the sensor data, the POST vs. PRE statistical contrasts of source estimates showed a strong β power decrease over parieto-central regions especially for the AV group; this decrease was also observed in the V and in the CTRL groups to a smaller extent (Fig. 3E). The observed β power suppression during motion coherence discrimination converges with previous literature reporting a central role of β power during perceptual decision-making tasks (Wyart et al., 2012; Donner et al., 2009; Alavash et al., 2017).

As interim summary for the univariate oscillatory analysis, we observed that α and broadband γ responses during the presentation of visual coherent motion were not significantly affected by training history, in contrast to β oscillatory activity seemingly affected by the degree of integrated evidence during training. β oscillations may play an important role in (multi)sensory perceptual discrimination consistent with its role in mediating interactions across distant structures during perceptual decision-making (Siegel et al., 2011). To disentangle the possible networks mediating these effects, we turned to multivariate functional connectivity (FC) analysis and investigated whether medium- and long-range interactions between cortical regions could provide complementary insights on the specificity of oscillatory regimes as a function of sensory history in training (Fig. 1E).

Task-related network synchronization during visual coherent motion discrimination

To characterize the functional connectivity (FC) induced by (multi)sensory training in the different oscillatory regimes, we estimated the PRE and POST activity during the presentation of motion coherence (i.e., excluding the initial incoherence interval of the stimuli) using 28 cortical regions (ROIs; Fig. 4A). Hence, the bivariate FC was estimated using the weighted phase-lag index (wPLI) (Fig. 4A) in three main synchronization regimes (α, β and high γ). ROIs were selected in a manner orthogonal to the contrasts of interest, mainly by performing a source estimation of the grand average data across all experimental conditions (Zilber et al., 2014, see Methods). All statistical contrasts (Fig. 4B) were based on non-parametric permutation t-tests. Only phase coupling values showing significant differences () were retained in the resulting functional networks reported herein.

First, we estimated the functional connectivity pattern during PRE and POST (i.e. during task), which significantly differed from resting-state (PRE vs. REST and POST vs. REST; Fig. 4B). The subtraction of the resting-state FC from PRE and POST was used as an equivalent of baseline in univariate analyses, and was performed to ensure that we characterized the task-relevant FC in both POST and PRE relative to the resting-state network. A direct comparison of POST vs. PRE FC without consideration of the initial resting-state FC would be a confounding factor in the interpretation of the results, and could falsely assign significant changes of FC to training effects, when they may have resulted from transitioning from REST to task. We then considered the task-relevant networks common to PRE and POST (Fig. 4C, top) and explored the effects of training on the changes of cortical interactions and oscillatory couplings.

Consistent with the occipital decrease in α band power observed in the univariate time-frequency analysis, we found a significant uncoupling of the α oscillatory network in task (both in PRE and in POST) as compared to REST (Fig. 5A, bottom left panel). A relative increase in synchronization modulated by sensory history (Fig. 5A, left column) was found from PRE to POST, involving a large network comprising occipital, temporal and parietal regions. This relative significant increase in α synchronization was observed in the V and in the AV groups, but not in the CTRL group. A similar analysis was performed for the β and the γ oscillatory regimes. Contrary to the α desynchronization pattern, the β and γ activity showed a strengthening of large-scale coupling from REST to task (PRE, POST) (Fig. 5A, bottom panels). The task-related β network implicated vlPFC, IPS and MT in all three groups but showed a significant strengthening from PRE to POST solely in the AV group (Fig. 5A, middle column). The significant relative increase of task-related FC (POST vs. PRE) was also observed for the AV group in the high γ regime implicating the auditory regions and the pSTS. In sum, all three groups displayed a characteristic desynchronization of the α network when engaged in the task, but a higher relative synchronization of the α network in POST as compared to PRE for the AV and V groups. Conversely, an increased synchronization of β and γ networks was found in all three groups from resting-state to task (PRE, POST), but only in the AV group did we see an increase of β and γ synchronization following training.

Fig. 5

Fluctuations of the task-related networks in PRE- and POST-training. (A) Source estimation was performed to obtain cortical activity in 28 regions of interest (see Section 2.7 for details). Pairwise cortical interactions based on wPLI, and averaged in each frequency band of interest, were estimated for each condition (REST, PRE and POST). POST vs. PRE contrasts of cortical interactions (lines connecting two regions in the figure) within the task-related FC network common to PRE and POST were separately studied for α (left), β (middle), and γ (right) and for the 3 training groups (AV:top, V: middle; CTRL: bottom). A qualitative description of FC changes (significant increases and decreases of interactions) is provided at the bottom, showing that increases from PRE to POST were relative to REST, with an initial desynchronization of α from REST to task (POST,PRE), and a relative synchronization of β and γ from REST to task (POST,PRE) was found in all three groups. The AV and V groups showed a relative increase of α FC from PRE to POST. Although a significant β phase-coupling in task-related FC network linking IPS, vLPFC and MT was found in all groups, only the AV group showed a significant strengthening of the β network following training. (B) Topological changes in FC networks from REST to PRE (top) and from PRE to POST (bottom). POST vs. PRE contrasts were performed for each training group separately. In the β network, the node degree in PRE increased in frontal and parietal regions, whereas it decreased in occipito-temporal regions as compared to REST. The reverse pattern was observed in the β network from PRE to POST in the AV group. See also Inline Supp. Mat.C.

Brain network analysis and topological differences in regional connectivity

To investigate the degree of interaction of each brain region, a brain network analysis was carried out using a measure of centrality as index (cf Eq. (3)). This analysis allowed investigating whether specific regions played a central role by assessing the topology of the estimated FC networks based on the number of phase coupling values (connections) over a specific threshold for each ROI (i.e. node degree, see Experimental Procedures). This quantification revealed distinct patterns for each oscillatory regime (Fig. 5B), all corroborating our previous analyses (Fig. 5A). The changes in the node degree within the estimated FC networks were assessed with the statistical contrasts PRE vs. REST combining all groups, and POST vs. PRE on a per group basis.

First, a general task-related decrease of node degree from REST to PRE (Fig. 5B, α blue nodes, top left) was observed in parietal, occipital and temporal regions for the α oscillatory network. This observation was consistent with the global α desynchronization during task as compared to REST. The same contrast for the β network (Fig. 5B, β, top middle) showed an increase of node degree in PRE in frontal and parietal regions (red nodes), but a decrease in occipito-temporal regions (blue nodes) as compared to REST networks. This pattern was expected considering that long-range cortical interactions in the β band are known to involve fronto-parietal regions during perceptual decision-making (Donner et al., 2007; Siegel et al., 2011). Motor cortices also showed a higher node degree in PRE than in REST, reflecting the information flow during task execution mediated by β oscillatory networks. The same contrast for the γ network (Fig. 5B, γ, top right) showed mainly a left-lateralized decrease of node degree but an increase in posterior regions.

We then investigated the changes in FC between POST and PRE as a function of training (Fig. 5B, bottom rows). For the AV group, the analysis of β oscillatory networks revealed a clear reversal of the node degree pattern in the POST vs. PRE contrast as compared to the PRE vs. REST contrast: an increase of node degree from PRE to POST was observed in occipito-temporal regions (mainly in the right hemisphere), while a decrease was found in frontal regions (mainly in the left hemisphere). The node degree value of β oscillatory networks implicating motor cortices also decreased with training in all three experimental groups. Conversely, the right mSTS region, which showed a decreasing node degree from REST to PRE, now consistently increased from PRE to POST in all three groups. This suggested the implication of the mSTS during actual training, the synchronization of which got stronger and more extensive (up to visual regions V4) following training.

In the topological analysis of high γ oscillatory networks, frontal regions exhibited opposite dynamics as compared to our observation in the β band. The node degree of the left frontal BA6 region (pre-motor and supplementary motor regions) decreased from REST to PRE, and increased from PRE to POST for the three groups. These results were in line with previous literature (Donner et al., 2007) showing that high γ and β choice-predictive activities showed opposite changes during perceptual decision-making. In the same study (Donner et al., 2007), oscillatory activities build up gradually during stimulus evidence encoding to reflect the integration of high γ activity in MT. Here, on the other hand, the node degree in mSTS and MT regions selectively increased after congruent multisensory training, consistent with the observed selective implication of these regions in the task (Zilber et al., 2014).

Emergence of β and γ functional networks following training with coherent audiovisual motion

The potential emergence of new functional coupling of cortical brain networks following training (i.e., POST-specific) was addressed on a per group basis (Fig. 4C, bottom). In Fig. 6A, we report cortical interactions that were specific to post-training, i.e., phase couplings among brain regions that were not significantly seen in the PRE vs. REST contrast but which significantly emerged after training in the POST vs REST contrast (see Methods). The POST-specific couplings between cortical regions emerged in a training-selective manner in the β band and only for the AV training group (Fig. 6A). This suggested that multisensory training with temporally comodulated audiovisual stimuli could subsequently affect the organization of cortical interactions during a purely visual discrimination task. The observed reorganization notably encompassed long-range interactions in POST-specific networks around temporal cortices, consistent with the results of the topological analysis shown in Fig. 5B. Additionally, the emergence of high γ phase-coupling after training was found mainly in the AV and the V training groups, while POST-specific α connectivity emerged in the CTRL group. Intriguingly, the sole behavioral variable relevant to the emergence of the β connectivity was participants’ confidence ratings: a significant linear correlation (, , ) was observed so that an increase in POST-specific β band connectivity solely observed in the AV group was commensurate with an increase in these participants’ confidence ratings on the task.

Fig. 6

Emerging oscillatory networks following training. (A) The phase synchronization based on pairwise wPLI between brain regions (lines connecting two regions in the figure) in α, β, and γ oscillatory regimes was investigated separately as a function of training group. The central finding was the emergence of novel mid-to long-range cortical interactions (POST-specific network, not present during PRE) in the β and γ networks especially in the AV group. The CTRL group trained with incongruent AV stimuli showed the largest synchronization in α. (B) A linear correlation between the average increase of the β band POST-specific interactions from PRE to POST and the increase in participants’ confidence ratings was observed solely for the AV group (top panel). The β band POST-specific network was mainly characterized by fronto-occipital and temporo-parietal interactions (bottom panel).

Discussion

In this study, we asked how internalized content, established on the basis of temporally coherent audiovisual signals, subsequently benefit the discrimination of visual motion coherence. During the presentation of visual motion stimuli, the spectral signatures of brain responses included a decrease in occipital α and frontal β power, and an increase of occipital γ power. While the occipital γ correlated with the strength in visual motion coherence and the post-training performance, α activity showed no functional modulation as a function of stimulus property, sensory evidence, performance or training. Additionally, several contrasts revealed that the local β power captured an integrated aspect of evidence based decision-making as a function of training history. Second, multivariate functional connectivity analysis based on oscillatory phase coupling showed a relative global increase of α ( Hz) phase synchronization post-training in the V and AV groups as compared to the CTRL group; this was found in the context of a general decrease of α FC as compared to REST. Third, and importantly, we report the emergence of long-range β ( Hz) and γ ( Hz) synchronization networks implicating temporal, prefrontal, parietal and visual cortices. The emergence of the β and γ networks was essentially observed following congruent (but not incongruent) multisensory training and the β network was indicative of participants’ confidence rating in post-training. Despite the limit represented by the number of participants in this study (), altogether, our results suggest that sensory history in training can subsequently strengthen decision-making networks through the regulation of large-scale oscillatory synchronizations. It would thus be beneficial in the future to increase the number of participants for robust estimation of network changes and characterization. It would also be interesting to test whether similar pattern of β network can be seen in sensory-impaired populations. The patterns observed in the present study arose from a selected number of cortical regions of interest investigated, which represent a second limit to overcome in future research.

Interplay between top-down α and feedforward γ

Attending stimuli increases both the local and large-scale synchronization of rhythmic neuronal activity in the γ band (Engel et al., 2001; Fries et al., 2001; Varela et al., 2001; Wang, 2010; Fries, 2015). An increase γ power has been reported during binding (Singer and Gray, 1995; Fries et al., 1997), multisensory integration (Bhattacharya et al., 2002; Mishra et al., 2007) and semantic congruence across sensory modalities (Yuval-Greenberg and Deouell, 2007; Schneider et al., 2008). Here, we observed an increase in occipital high γ band during the presentation of visual motion which, consistent with previous work (Sokolov et al., 1999; Siegel et al., 2006, 2011), increased with increasing strength in visual motion coherence. This effect was seen in visual cortices for all three experimental groups, both before and after their respective training. This pattern converged with the notion that γ power provides a spectral index of sensory evidence encoding (Siegel et al., 2006, 2011), and here, may further be a significant indicator of participants’ correct perceptual discrimination following training. As expected (Fries et al., 2001; Fries, 2009; Siegel et al., 2012; Jensen and Mazaheri, 2010), we also observed a concomitant decrease in occipital α power. Seminal work has suggested that α suppression was stronger for the detection of meaningful objects than for scrambled ones (Vanni et al., 1997), and associated with visuo-spatial (Sauseng et al., 2005) and object-based (Foxe and Snyder, 2011) selective attention. An increase in pre-stimulus α power is also known to impair detection (Busch et al., 2009; Van Dijk et al., 2008; Hanslmayr et al., 2007; Haegens et al., 2011; Jones et al., 2010; Zhang and Ding, 2010; Grabot et al., 2017) and, conversely, an increase in α power is often observed in unattended modalities (Busch and VanRullen, 2010; Kelly et al., 2006; Worden et al., 2000). α oscillations are deemed instrumental for selective attention and the top-down control of information (Sadaghiani and Kleinschmidt, 2016). The modulation of occipital α was previously shown to correlate with behavioral improvements of visual motion discrimination in presence of congruently moving sounds (Gleiss and Kayser, 2014a). Here, no systematic changes in the occipital α were found as a function of experimental or behavioral variables, and this was likely due to differences in paradigmatic and methodological approaches: the most notable one being that we did not directly contrast unisensory vs multisensory stimulations per se. Rather, the stable level of occipital α power over experimental conditions parsimoniously indicated that all participants were effectively attentive to the stimuli irrespective of the strength of motion coherence or training history. Recent work has also suggested that γ and α (and β) activity were markers of feedforward and feedback propagation, respectively (Van Kerkoerle et al., 2014; Wang, 2010; Lee et al., 2013; Bastos et al., 2012, 2015; Buffalo et al., 2011; Michalareas et al., 2016; Richter et al., 2017). Given the pattern of stable α suppression and increased γ power as a function of motion coherence (and post-training correctness), one possible working hypothesis is that, given a stable and sustained endogenous attentional control exerted in the α range, perceptual training may improve the efficiency with which γ activity propagates sensory evidence up the hierarchy.

The effect of sensory history in training on α and γ network phase-synchronization

In line with the notion that α oscillations actively contribute to the selection of cortical regions during task (Palva and Palva, 2007, 2011), a decrease of global α phase synchronization was observed from resting-state (REST) to task (PRE, POST) in all groups. Yet, and perhaps more importantly, a relative global increase in α synchronization engaging a parieto-occipital network was observed in post-training as compared to pre-training in both the AV and V groups. This pattern was interestingly not observed in the CTRL group, who was trained with distracting uncorrelated sounds. Rather, the CTRL group (and to some extent, the V group) showed a fronto-parietal α synchronization post-training. The presence of large-scale α synchronization in the CTRL group could be primarily explained by the selective function of α networks, which may help decouple brain regions during conflicting inputs (CTRL). Specifically, at the same time we observed these patterns in the α FC, we also observed a global strengthening of the post-training γ synchronization network. γ-band synchronization in brain networks is fundamental in cortical communication as phase-coupling across brain regions may promote the transmission of information across large-scale neuronal networks (Fries, 2009, 2015; Bastos et al., 2015). Global γ synchronization is notably considered to denote “effective, precise and selective” communication (Fries, 2015). In this study, training may have improved long-range γ synchronization with a possible gain in communication efficiency consistent with the general improvement on the task observed in all groups (Zilber et al., 2014). Sensory history in training affected the general increase of γ phase synchronization so that the group with the largest behavioral improvement, i.e. the AV group, also showed the strongest increase followed by the V and the CTRL group. Altogether, theses results suggest that the type of sensory inputs during a very short training (here a total of 20 min) can selectivity affect the coordination of brain regions implicated in the endogenous control of information processing.

β oscillations as integrated evidence

Very recently, β oscillations have been proposed to be markers of internal content (Spitzer and Haegens, 2017) and supramodal processing (Haegens et al., 2017): following learning, rhythms such as β oscillations may regulate the feedback processing of sensory analysis (Brincat and Miller, 2016) based on abstract categorical representations. These internal network dynamics were observed in prefrontal cortices, and operated in the α/β bands (Brincat and Miller, 2016). An important finding in our study is the fundamental role of β oscillations, both as local power decrease during sensory encoding and decision-making in all groups, and as an emergent large-scale network following congruent multisensory AV training. Several studies have reported an increased coherence or synchronization in the β band associated with multisensory stimulation (von Stein et al., 1999; Mercier et al., 2015) and, consistent with the implication of the β band in sensorimotor processing (Engel and Fries, 2010), β activity was related to participants’ response speed in multisensory context. Gleiss and Kayser (2014b) reported early differences in β band activity but did not find any correlation with behavior. Consistent with another study (Mercier et al., 2015), we found that local modulations of β power were observed when participants were engaged in the task but not during passive viewing, and that RT was the main contributor for this effect: the decrease in local β power was found for contrasts in which evidence-based decisions were most successful (i.e., for strongest as compared to weakest visual motion coherence, for correct as compared to incorrect trials and for post-, compared to, pre-training trials). Under the working hypothesis that abstract internal content (Brincat and Miller, 2016; Wutz et al., 2018) has been learned to drive the processing of incoming sensory information, the strengthening of the large-scale β coupling in the group that has received congruent AV training would further suggest that performing the task with temporally coherent audiovisual events strengthened the ability to predict motion coherence in vision. In other words, the changes in β power and network synchronization may capture endogenous top-down activations of task-relevant (supramodal) cortical representations, which facilitate communication between brain regions (Kopell et al., 2000; Siegel et al., 2011; Spitzer and Haegens, 2017; Haegens et al., 2017; Bressler and Richter, 2015).

In this context, recent predictive coding models drawing from audiovisual speech processing (van Wassenhove et al., 2005; van Wassenhove, 2013; Arnal et al., 2011) and neurophysiological work (Richter et al., 2017; Bressler and Richter, 2015) have pushed forward the notion that prediction errors from one sensory modality to another may be communicated in the γ range, whereas top-down predictions may be mediated by β oscillations (Arnal et al., 2011). As the emergent β network was solely seen for the AV group in which it was linearly related to confidence rating, we speculate that the hypothesized combined effects of increased communication efficiency in a feedforward γ network, the endogenous selective routing in the α network, and the predictive β propagation in the AV group may all contribute to the local selective changes previously reported in the human motion area MT as a change in the neurometric threshold (Zilber et al., 2014). The notion that internal content (as supramodal or abstract representations) may constrain sensory analysis early on provides additional evidence for the implication of large-scale neural oscillations in integrative and predictive brain functions.

Limitations

It is perhaps noteworthy that our γ-band results do not necessarily imply nor preclude the presence of underlying oscillatory sources. Several studies have cast doubt on the idea that task-related changes or ongoing background activity in γ-band dynamics are consistently due to oscillations (Ray and Maunsell, 2011; Manning et al., 2009; Scheffer-Teixeira et al., 2013). Accordingly, the power spectrum does not always show a distinct peak that could unequivocally index oscillations. Instead broad-band power changes are often observed that may exhibit 1/f scale-free behavior (Voytek et al., 2015) and may be accounted for in terms of short-lived stochastic spiking (Miller et al., 2014). We want to emphasize that our study does not possess the requisite statistical power or experimental paradigm to tell apart these concurrent interpretations. We, therefore, carefully suggest to view our γ–band findings as neuronal activity in the wider sense, leaving open the precise physiological generative mechanism. Our phase-based analysis in the γ-band may therefore suffer from specificity, and has to be regarded as pragmatic approximation that may get revised in the future upon the availability of more precise computational tools and an extended study-design.

Conclusion

Taken together, our results support the notion that cortical computations encompass sensory-based processing and that, consistent with the role of prefrontal cortices shifting activity from feedforward inputs to internal dynamics (Brincat and Miller, 2016), the internal content shaped by multisensory inputs during short training can strengthen the selectivity of large-scale oscillatory networks for later adaptive purposes.