Deactivation in the posterior mid-cingulate cortex reflects perceptual transitions during binocular rivalry: Evidence from simultaneous EEG-fMRI.

Binocular rivalry is a phenomenon in which perception spontaneously shifts between two different images that are dichoptically presented to the viewer. By elucidating the cortical networks responsible for these stochastic fluctuations in perception, we can potentially learn much about the neural correlates of visual awareness. We obtained concurrent EEG-fMRI data for a group of 20 healthy human subjects during the continuous presentation of dichoptic visual stimuli. The two eyes' images were tagged with different temporal frequencies so that eye specific steady-state visual evoked potential (SSVEP) signals could be extracted from the EEG data for direct comparison with changes in fMRI BOLD activity associated with binocular rivalry. We additionally included a smooth replay condition that emulated the perceptual transitions experienced during binocular rivalry as a control stimulus. We evaluated a novel SSVEP-informed fMRI analysis in this study in order to delineate the temporal dynamics of rivalry-related BOLD activity from both an electrophysiological and behavioral perspective. In this manner, we assessed BOLD activity during rivalry that was directly correlated with peaks and crosses of the two rivaling, frequency-tagged SSVEP signals, for comparison with BOLD activity associated with subject reported perceptual transitions. Our findings point to a critical role of a right lateralized fronto-parietal network in the processing of bistable stimuli, given that BOLD activity in the right superior/inferior parietal lobules was significantly elevated throughout binocular rivalry and in particular during perceptual transitions, compared with the replay condition. Based on the SSVEP-informed analysis, rivalry was further associated with significantly enhanced BOLD suppression in the posterior mid-cingulate cortex during perceptual transitions, compared with SSVEP crosses. Overall, this work points to a careful interplay between early visual areas, the right posterior parietal cortex and the mid-cingulate cortex in mediating the spontaneous perceptual changes associated with binocular rivalry and has significant implications for future multimodal imaging studies of perception and awareness.

Introduction

Bistable stimuli have long been used to investigate the behavioral aspects of consciousness, and more recently through the use of imaging, the neural correlates of consciousness. During binocular rivalry, one’s conscious awareness spontaneously fluctuates between two different images that are presented simultaneously to the visual system, one to each eye (Tong, 2003). Low-level competition between eye-specific neurons in the lateral geniculate nucleus and primary visual cortex has been implicated in the resolution of binocular rivalry (Blake, 1989; Rees, 2007). Still other studies have suggested that binocular rivalry is the result of competition between conflicting neural patterns at later stages in the visual processing stream, versus simply being the result of competition between monocular neurons (Leopold and Logothetis, 1996; Freeman, 2005). Such theories of rivalry assume only partial suppression of eye-specific information at early visual processing levels, allowing for residual neural signals to reach brain regions involved in higher level, top-down cognitive processing (Wilson, 2003).

Indeed, increased fronto-parietal BOLD activity has been previously associated with binocular rivalry. Several studies have associated cortical activity in multiple regions of the posterior parietal cortex and prefrontal cortex with perceptual transitions during binocular rivalry (Knapen et al., 2011; Wilcke et al., 2009; Britz et al., 2011). Structural differences in the posterior parietal cortex have also been associated with individual differences in perceptual dominance durations during rivalry (Kanai et al., 2010). Furthermore, prefrontal regions that have been implicated in rivalry are generally associated with high level cognitive processes such as decision making, selective attention and goal selection. These top-down networks can modulate activation of neural populations in earlier visual processing areas through feedback projections (Tong et al., 2006). Thus, fronto-parietal networks may be important for integrating high level pattern representations of the visual stimuli with low level neuronal competition in early visual networks for the resolution of rivalry.

A number of studies have suggested a role of both bottom-up and top-down cortical networks in the processing and interpretation of bistable stimuli (Zhang et al., 2011; Meng and Tong, 2004). This bidirectional model emphasizes the recruitment of widespread neural networks during rivalry and reiterates the importance of fronto-parietal regions for its resolution. However, this view has been challenged by more recent work showing that much of the fronto-parietal activity associated with rivalry disappears when bistable changes in visual competition become perceptually invisible/unreportable (Zou et al., 2016; Brascamp et al., 2015). Additionally, it has been suggested that frontal activity during rivalry predominantly reflects introspection and action, not perception, using optokinetic nystagmus and pupil size to track perceptual alternations (Frassle et al., 2014). Given these findings, it is possible that fronto-parietal activity during rivalry lacks a causal role in driving visual competition and is more related to processes involved in visual awareness and attention. Regardless, the specific neural mechanisms of binocular rivalry and their spatiotemporal characteristics remain elusive.

The goal of this study was to delineate neural networks involved in rivalry using simultaneous EEG-fMRI functional imaging techniques. It was hypothesized that changes in BOLD activity during binocular rivalry could be directly correlated with simultaneous fluctuations in occipital EEG activity related to eye-specific competition of frequency-tagged steady-state visual evoked potentials (SSVEPs). Specifically, we predicted that fronto-parietal BOLD increases during rivalry would be greatest during perceptual transitions, as has been previously reported (Knapen et al., 2011). We employed a novel EEG-informed fMRI analysis approach for assessing event-related BOLD activity specifically correlated with SSVEP dominance, when one of the SSVEP signal envelopes peaked while the other was maximally suppressed, and SSVEP crosses, when the amplitudes of the two SSVEP envelopes were approximately equivalent. Using this integrative approach, we found that BOLD activity in a right-lateralized fronto-parietal network was differentially elevated throughout rivalry with peak activation during perceptual transitions, compared to both SSVEP peaks and crosses. In contrast, when we directly compared BOLD activity differences between SSVEP envelope peaks and crosses, this fronto-parietal activity disappeared, suggesting that rivalry is predominantly resolved within early visual regions. Additionally, we found strong deactivation in the posterior mid-cingulate cortex during rivalry-related perceptual transitions, suggesting that this region may be critical for mediating information transfer between early visual and fronto-parietal networks during visual processing. Overall, our findings reiterate that multiple cortical networks are differentially activated during binocular rivalry due to multiple cognitive processes operating concurrently for its resolution.

Methods

Subjects

All experiments were carried out following procedures approved by the Institutional Review Board of the University of Minnesota, Twin Cities. We recruited 20 healthy human subjects for simultaneous EEG-fMRI data acquisition during binocular rivalry. Prior to EEG-fMRI sessions, subjects were introduced to the binocular rivalry stimulus in EEG-alone sessions. This allowed us to evaluate subject-specific perceptual dominance durations prior to the main experiments, to ensure that they were of sufficient length for adequate SSVEP envelope extraction.

Stimulus paradigm

Subjects were presented with a dichoptic rivalry stimulus which consisted of a rotating green and black circular checkerboard presented to one eye, and a rotating (in the opposite direction) red and black checkerboard presented to the other eye as shown in Fig. 1. During replay blocks the same rotating image was presented to both eyes at any time. Subjects were instructed to report their current perceptual state using three different buttons: B1 for green, B2 for red and B1+B2 for perceptually mixed states (i.e. perceptual transitions). Images rotated at a rate of 11.25 degrees/second, and each image was presented flickering at a different frequency within an annular window from 0.5–3.75 degrees visual angle. The green and red images flickered at 6.67 and 8.57 Hz, respectively, and were presented on a gray background with a central fixation cross for subjects to attend to during experimental blocks. Stimulus frequencies were determined prior to EEG-fMRI experiments based on our group’s experience with EEG-alone and behavioral studies of rivalry-related SSVEP stimuli (Zhang et al., 2011; Jamison et al., 2015). Frequencies were chosen to be perceptually similar in order to avoid potential stimulus bias that could arise from significantly different rates of contrast reversal. Tagged frequencies were implemented with the MR’s stimulus presentation screen using a 60 Hz refresh rate. Additionally, the chosen frequencies resulted in sufficient dominance durations (2–5 seconds) during rivalry across all subjects. This was important for reliable SSVEP envelope reconstruction during later EEG analyses. For each subject, each experimental block of rivalry or replay consisted of 5 consecutive 42 second trials of continuous stimulus presentation followed by 12 seconds of rest, and 5 total blocks for each stimulus type. Green and red visual stimuli were randomly alternated between the eyes across trials of rivalry and counterbalanced across subjects. This ensured that data sets related to the green and red percepts were approximately equal in size and avoided any sort of visual entrainment that could occur from repeated exposure of each eye to the same visual stimuli.

Fig. 1

Binocular rivalry stimulus design. During rivalry blocks (left) each eye was presented with a different flickering image in the center of the visual field. Replay blocks (right) simulated both instantaneous and smooth perceptual transitions for comparison with the rivalry condition. Green and red checkerboards were used to induce SSVEPs at 6.67 Hz and 8.57 Hz, respectively.

During all experimental blocks, subjects viewed visual stimuli through prism lenses. These lenses were 10PD 37 mm acrylic squares (Bernell Corporation, Mishawaka, IN) and used to redirect the two stimulus images on the left and right sides of the stimulus presentation screen to the center of the subject’s visual field. Additionally, a vertical divider covered with black felt was placed between the head coil mirror and the center of the stimulus presentation screen located in the back of the scanner bore. This prevented cross-talk between the two visual stimuli, ensuring that each eye was only seeing one image at a time, despite both images being presented in the center of the visual field. At the beginning of each EEG-fMRI experiment, SSVEP stimuli were first balanced for mean luminance, after which small adjustments were made for each subject to equalize perceived brightness of the green and red checkerboard stimuli.

Importantly, we employed two different replay conditions: smooth replay and instantaneous replay. Previous studies have shown the importance of utilizing a smooth or duration-matched replay condition when investigating binocular rivalry (Knapen et al., 2011). For the smooth replay condition, the presented images were switched between the green and red checkerboards using a smooth, expanding wedge to approximate the gradual perceptual transitions associated with real rivalry. For each simulated transition, a small wedge of the target checkerboard would smoothly expand to cover the non-target checkerboard over the course of one second. Wedges expanded outwards in both directions at an equal pace, with the starting point of each expanding wedge randomized with respect to the center of the visual field. In contrast, images were immediately switched for the instantaneous replay condition. The instantaneous replay condition was included in order to reproduce previous fMRI studies of rivalry and ensure the overall quality of the fMRI dataset. For each subject, percept durations for the replay condition were “played back” based on a previous block of rivalry for the same subject. If multiple blocks of rivalry were recorded during a session prior to a replay block, the rivalry block chosen for playback was randomly chosen from them.

Simultaneous EEG-fMRI sessions

Simultaneous EEG-fMRI experiments were carried out in a 3 T Skyra Scanner (Siemens, Human Connectome Project) at the Center for Magnetic Resonance Research at UMN. Following the informed consent process and MRI safety screening, subjects were seated in the MR control room during the setup of the EEG cap. We utilized a 64-channel MRI-compatible EEG system (BrainAmp MRplus, BrainProducts) to continuously record both EEG and ECG over the course of the experiment (Im et al., 2007; Im et al., 2006; Liu and He, 2008; Liu et al., 2009; He and Liu, 2008; He et al., 2011). The cap was prepared using a conductive gel in order to reduce EEG electrode impedances to below 20 kOhm before placing the subject inside the scanner. ECG was recorded using a single electrode placed on the subject’s upper back, for later removal of the cardioballistic artifact in the EEG. Additionally, we utilized a SyncBox in order to synchronize the 10 MHz clock on the MRI gradient computer with the EEG system. This ensured that gradient-induced artifacts in the EEG were as consistent as possible, for later removal. EEG data were obtained using a sampling frequency of 5000 Hz and EEG electrode positions in 3-dimensional space were digitized (FasTrak, Polhemus) at the end of each experiment once the subject was out of the scanner and back in the MR control room.

Concurrent with EEG recordings, whole-brain fMRI BOLD data were acquired during the presentation of visual stimuli using a standard echo planar imaging (EPI) pulse sequence (FA=90°; TR=2200ms; TE=30ms; 3mm isotropic voxels; 36 axial slices; fat suppression on). A 16-channel head-coil was used for data acquisition through which the EEG electrode cables could be routed through an opening in the coil near the top of the head. Subjects were placed in the scanner in a head-first, supine position and viewed visual stimuli using a mirror that was mounted on the head-coil prior to entering the subject into the scanner bore. In addition to the functional data, we obtained a high-resolution MPRAGE structural image of each subject’s brain. Heartbeat was monitored using an infrared sensor placed on the subject’s pointer finger, and respiratory activity was monitored using a flexible band which was wrapped around the subject’s torso prior to entering the bore.

EEG preprocessing

All EEG data were analyzed using MATLAB including the EEGLAB toolkit. EEG data were first re-referenced to the average of all channels and downsampled to 250 Hz prior to band-pass filtering between 0.5–30 Hz. An optimal basis set algorithm based on principal component analysis (PCA) was utilized for removal of the substantial EPI-related gradient artifacts present in the EEG data (Niazy et al., 2005). Next, bad channels and epochs were identified and excluded from further analysis. Noise contaminated electrodes were rejected following visual inspection of a histogram displaying average power for each electrode. Bad epochs were defined as periods when the mean global field power exceeded five standard deviations above the mean. In addition to the gradient artifacts, substantial cardioballistic artifacts (CBAs) related to blood flow in the scalp were present in the EEG. We addressed the CBA by using a multi-step procedure previously developed by our group (Jamison et al., 2015). This method first optimized the alignment of the occipital CBA with each subject’s heartbeat and then utilized independent component analysis (ICA) to remove EEG components most strongly correlated with the occipital CBA. Importantly, this technique utilized a mutual-information metric for identifying noise components most strongly associated with the CBA artifact. This effectively accounted for subject-specific and trial-to-trial variability in the latency of the CBA artifact, with respect to subject heartbeats. Finally, ICA was again used to remove remaining EEG artifacts related to eye blinks, eye movement and muscle activity, based on visual inspection of the independent component time-courses and weight maps.

EEG analysis

Analysis of the frequency-tagged EEG signals was performed based on methods previously developed by our research group (Zhang et al., 2011; Jamison et al., 2015). In order to maximize occipital SNR of the f1 (6.67 Hz) and f2 (8.57 Hz) frequency-tagged SSVEP signals, we employed a gradient-descent spatial optimization filter (Niazy et al., 2005). A five-fold cross validation technique was used to determine the number of iterations necessary for optimizing SSVEP-related topographies for each subject. Each iteration of the gradient-descent filter aims to maximize the quality of the occipital SSVEP data, based on the total SNR of cortical responses in the visual cortex that are directly related to the frequency-tagged visual stimuli. In this sense, the cost function is maximizing the spectral SNR for each SSVEP envelope iteratively. We have previously demonstrated that this optimized spatial filter significantly improves SSVEP envelope reconstruction from EEG-fMRI data compared to a conventional Laplacian spatial filter (He, 1999).

After spatial optimization, two different frequency-specific adaptive recursive least squares (RLS) filters were used to extract the time-courses of the tagged frequencies from each subject’s occipital EEG data, usually from channel Oz or POz (Brown and Norcia, 1997). The adaptive RLS filter operates in a sliding window fashion and utilizes a pair of sine and cosine matched filters for estimating the magnitude and phase of the target SSVEP envelope over time. Such an approach was analogous to estimating the short-time Fourier transform (STFT) for each SSVEP frequency and window.

Following SSVEP envelope reconstruction, we compared SSVEP activity between the rivalry and replay conditions for both button-aligned and peak-aligned SSVEP averages. SSVEP envelope peaks related to the green (f1) and red (f2) stimuli were identified based on the local maxima (peaks) of each envelope’s respective timecourse. Reaction times between SSVEP envelope crosses and button-presses related to perceptual dominance were calculated for the different experimental conditions based on the button-aligned averages.

In order to localize frequency-specific EEG activity, Curry 7.0 was used to carry out source localization analysis of the EEG data. Topographies related to the two tagged frequencies of interest were aligned to button presses and averaged across subjects prior to source localization for determining specific cortical sources of frequency-tagged EEG signals. The results of the SSVEP source analysis were compared to the BOLD data to determine the extent of co-localized electrical and hemodynamic activity in early visual regions. Source localization analysis was mainly carried out to ensure the overall quality of the EEG dataset in terms of localization of visual activity.

fMRI analysis

All fMRI analysis was carried out in MATLAB, including the SPM12 Toolkit. All functional data were aligned to MNI space for visualization and statistical analysis. Functional data were first motion corrected, co-registered and spatially smoothed based on standard procedures previously employed by our group (Jamison et al., 2015). We assessed BOLD activity related to ongoing changes in visual perception using a novel SSVEP-informed general linear model (GLM) for fMRI analysis. Individual subject contrast images based on t-statistics were calculated at the first level using a [1,−1] weight vector for each contrast of interest, prior to being evaluated with one-sample t-tests at the second (group) level. Group level contrast maps were then subject to FDR cluster correction (primary clustering threshold: p < 0.001 uncorrected, p < 0.05 FDR cluster corrected) prior to visualization (Woo et al., 2014; Eklund et al., 2016). The SPM Anatomy Toolbox was used for identifying peak MNI coordinates and t-stats for regions of activation. The regressor creation process for the GLM evaluated is summarized in Fig. 2.

Fig. 2

Binocular rivalry EEG-informed fMRI analysis pipeline. A novel SSVEP-informed GLM approach was utilized in order to delineate the spatio-temporal dynamics of BOLD activity during binocular rivalry. Event-related regressors were generated based on the timings of SSVEP envelope peaks, SSVEP envelope crosses and subject reported button presses related to perceptual transitions. Displayed are the 3 regressors for a single scan (run) of rivalry.

In order to investigate specific differences in cortical activity associated with the behavioral and electrophysiological correlates of rivalry, we utilized event-related regressors obtained from both analysis of the SSVEP data and subjective reports of perceptual mixing (Fig. 2). First, we assessed differences in global BOLD activity related to the overall contrast between the rivaling, frequency-tagged SSVEP signals in early visual cortex. For this novel EEG-informed fMRI analysis, the absolute value of the difference between the f1 and f2 SSVEP envelopes was first calculated for each stimulus block, using the equation abs(f1–f2). The local maxima time-points (SSVEP envelope peaks) of the resultant abs(f1–f2) time-course were selected for the first regressor while the local minima time-points (SSVEP envelope crosses) of the same time-course were selected for the second regressor. More generally, SSVEP peaks corresponded to when one SSVEP signal was strongly dominant over the other while SSVEP crosses corresponded to times when neither SSVEP was dominant. Thus, the first regressor can be thought of as an electrophysiological analog of rivalry dominance while the second can be thought of as an electrophysiological analog of rivalry transitions. By utilizing this event-based SSVEP-informed fMRI approach we accounted for trial to trial variability associated with the EEG. Additionally, we utilized a third event-related regressor which was based on button-presses related to perceptual transitions, similar to previous fMRI imaging studies of rivalry. Perceptual transitions were defined as the midpoints of subject reported perceptual mixing periods. Regressors for SSVEP peaks, SSVEP crosses and perceptual transitions were modeled as three different series of impulse functions convolved with the canonical hemodynamic response function.

For each session, each EPI scan recorded data during the presentation of either rivalry, smooth replay, or instantaneous replay visual stimuli. Thus, there were three different experimental conditions assessed, with three different regressors modeling SSVEP peaks, SSVEP crosses, and perceptual transitions in each scan, for a total of 9 regressors (3 per run type). For most subjects/experimental sessions, we acquired data for 5 runs (scans) of the rivalry task, 5 runs of the smooth replay task, and 2 runs of the instantaneous replay task, in random order. However, there was a small amount of variability in the total number of runs across subjects ( ± 1 run per condition). Regardless, the typical first-level GLM used for analysis was comprised of 12 rows, corresponding with 1452 volumes acquired over 12 runs (121 volumes per run, ~270 seconds of data each). It was of block-diagonal form where each diagonal entry comprised a run-specific design matrix of 3 columns. For each run, the first column represented the SSVEP peaks regressor, the second column the SSVEP crosses regressor, and the third column the perceptual transitions regressor, for the respective experimental condition. Additional regressors for motion correction were also added to regress out movement related activity during each run.

Contrast vectors used were [1 0 0 −1 0 0 0 0 0] and [1 0 0 0 0 0 −1 0 0] comparing SSVEP peaks between the rivalry and two replay conditions (smooth and instantaneous, respectively). Similarly, for comparing SSVEP crosses between rivalry and the two replay conditions, the contrast vectors used were [0 1 0 0 −1 0 0 0 0] and [0 1 0 0 0 0 0 −1 0]. Finally, contrast vectors of [0 0 1 0 0 −1 0 0 0] and [0 0 1 0 0 0 0 0 −1] were used to compare perceptual transitions between the rivalry and two replay conditions. Additionally, we assessed {SSVEP Peaks-SSVEP Crosses} within each experimental condition using contrast vectors of [1 −1 0 0 0 0 0 0 0], [0 0 0 1 −1 0 0 0 0] and [0 0 0 0 0 0 1 −1 0] for the rivalry, smooth replay and instantaneous replay conditions, respectively. Finally, we assessed {Perceptual Transitions-SSVEP Crosses} within each experimental condition using contrast vectors of [0 −1 1 0 0 0 0 0 0], [0 0 0 0 −1 1 0 0 0] and [0 0 0 0 0 0 0 −1 1] for the rivalry, smooth replay and instantaneous replay conditions, respectively. We have included an exemplary first-level design matrix in the supplementary results to further clarify the first-level analysis procedure (Fig. S1). In addition to contrasting SSVEP peaks with SSVEP crosses to find the main effects within each experimental condition, we also contrasted the {SSVEP Peaks-SSVEP Crosses} maps for the rivalry and smooth replay conditions. A similar analysis was carried out to assess differences between the rivalry and smooth replay conditions for the {Perceptual Transitions-SSVEP Crosses} contrast.

Results

Subject perceptual reports

Perceptual dominance durations during rivalry were sufficiently long to allow SSVEP analysis in all 20 subjects (for durations shorter than 1 sec, it is difficult to measure SSVEP amplitude). Fig. 3 shows the pooled percept durations across all subjects, for all button press events across all trials. For each subject, the average green, red and mixed durations across trials were calculated prior to group level statistical analysis of the behavioral data. For the rivalry condition, a one-way analysis of variance (ANOVA) revealed a significant main effect (F(2,54)=85.494, p < 0.001) of event type (comparing subject reported green, red and mixed perceptual blocks). A post-hoc multiple comparisons analysis showed no significant difference between the average dominance duration for the green percept (M=2.65 s, SE=0.12 s) and the average dominance duration for the red percept (M=2.41 s, SE=0.12 s) across subjects. Additionally, the post-hoc analysis revealed that the average duration of mixed perceptual blocks (M=0.59 s, SE=0.12 s) was significantly shorter than the average durations of both dominance conditions. Button-press events corresponding to perceptual mixing represented ongoing transitions between green and red dominance blocks or between two dominance blocks of the same color.

Fig. 3

Rivalry dominance durations. Distribution of dominance durations associated with the three different button-press events for the rivalry condition (n=20).

SSVEP countermodulation & source localization

We successfully extracted frequency-tagged SSVEP signal envelopes from the occipital EEG data for the rivalry and replay conditions (Fig. 4). As expected, SSVEP envelopes modulated around button presses during rivalry, indicating neural suppression of the perceptually suppressed target (Zhang et al., 2011; Jamison et al., 2015). Similar countermodulation activity was observed for the replay conditions, with increased SSVEP SNR compared to rivalry, as expected. The time-series of the SSVEP envelope was calculated for each tagged frequency, aligned and averaged around the button presses indicating perceptual dominance of the green and red stimuli (Fig. 4, Top Row). Shortly after (or around) the time subjects indicated that each stimulus was dominant, the SSVEP envelope for that stimulus’ frequency reached a peak, while the envelope for the non-dominant stimulus showed a trough. SSVEP amplitudes were similar for both f1 and f2 during perceptual transitions (i.e. subject reported perceptual mixing periods). In this sense, perceptual transitions were similar to SSVEP crosses from an electrophysiological perspective. For all subjects, we found that button presses corresponding to green or red perceptual dominance were usually preceded by the corresponding SSVEP signal crossing into dominance. For green perceptual dominance blocks the green SSVEP signal crossed into dominance an average of 1142 ms (SE=88 ms) prior to the subjects indicating green dominance. Similarly, for red perceptual dominance blocks the red SSVEP signal crossed into dominance an average of 912 ms (SE= 81 ms) prior to the subjects indicating red dominance. This reaction time was much shorter for the smooth replay condition: 312 ms (SE=41 ms) for green percepts and 381 ms (SE=37 ms) for red percepts. The instantaneous replay condition was also associated with short reaction times: 405 ms (SE=36 ms) for green percepts and 468 ms (SE=34 ms) for red percepts. We further aligned and averaged SSVEP envelopes around SSVEP envelope peaks (Fig. 4, Middle Row). Source localization analysis of the f1 and f2 envelope topographies was carried out from the same data. In general, the strongest SSVEP sources were consistently localized to the early visual and extrastriate cortex, specifically the occipital pole and lingual gyrus (Fig. 4, Bottom Row).

Fig. 4

Grand Average SSVEP envelopes aligned to button-presses (top) and SSVEP envelope peaks (bottom) during rivalry. Frequency-tagged SSVEP signals were extracted from channel Oz, aligned and averaged around button presses indicating a transition to dominance of the green (top left) or red (top right) stimulus, respectively. The envelope of the frequency of the dominant stimulus peaks just after the time of the button press (t=0), while the envelope of the other stimulus frequency reaches a trough. The bottom figures show the normalized SSVEP sources (current density reconstructions) for f1 (green) and f2 (red) derived from their respective topographies at B1 and B2.

fMRI results

We assessed the results of a novel SSVEP-informed fMRI analysis in order to delineate the spatio-temporal dynamics of hemodynamic activity related to continuous binocular rivalry. This approach revealed a widespread cortical network involved in visual processing during both the rivalry and replay conditions, which included the primary visual cortex, extra striate cortex, posterior parietal cortex, precuneus, cingulate cortex, angular gyrus, and prefrontal cortex.

The assessed GLM was composed of three event-related regressors for each experimental condition based on both the timings of SSVEP envelope peaks and crosses as well as the midpoints of subject reported perceptual mixing periods, i.e. perceptual transitions. We first contrasted activity differences for perceptual transitions between the rivalry and instantaneous replay conditions in order to assess the quality of the fMRI dataset and replicate previous fMRI findings (Fig. 5). As expected, rivalry was associated with significantly elevated activity in a right-lateralized fronto-parietal network, notably the right superior and inferior parietal lobules. Strong activation in prefrontal regions including the frontal eye fields was also observed. Rivalry-related perceptual transitions were also associated with significantly decreased activity in the primary visual cortex, precuneus and posterior/mid-cingulate cortex, compared with instantaneous replay transitions.

Fig. 5

Rivalry vs Instantaneous Replay Contrast for Perceptual Transitions. The contrast in perceptual transition related activity between the rivalry and instantaneous replay conditions was evaluated using a [1, −1] contrast vector for first-level analysis, followed by second-level analysis and FDR cluster correction (primary clustering threshold p < 0.001 uncorrected, FDR corrected p < 0.05).

We next contrasted differences between the rivalry and smooth replay conditions for each regressor of interest (Fig. 6). Corresponding t-stats and peak MNI coordinates for activated cortical regions are displayed in Table 1. Rivalry-related SSVEP peaks (Fig. 6, A) were associated with increased activity in the right superior and inferior parietal lobules (rSPL/IPL), right superior occipital gyrus, right angular gyrus and right supramarginal gyrus, contrasted with SSVEP peaks for the smooth replay condition. Increased peak-related activity was also observed in the left superior parietal lobule, right precentral gyrus and right inferior frontal gyrus (p. Opercularis) during rivalry, while reduced activity was observed in the left superior and middle frontal gyri.

Fig. 6

Rivalry vs Smooth Replay Contrasts. Contrasts between rivalry and replay for each regressor were evaluated using a [1, −1] contrast vector for first-level analysis, followed by second-level analysis and FDR cluster correction (primary clustering threshold p < 0.001 uncorrected, FDR corrected p < 0.05).

Table 1

Rivalry versus Smooth Replay Contrasts.

Regions of activation:	Peak MNI coordinates:	t-stat:
A. SSVEP Peaks
Positive Cluster 1 (193 voxels, qFDR < 0.001)
R Superior Occipital Gyrus	(24,−61,46)	7.13
R Superior Parietal Lobule	(18,−64,49)	6.21
R Angular Gyrus	(30,−52,43)	5.13
R Inferior Parietal Lobule	(36,−43,46)	4.45
R Supramarginal Gyrus	(42,−34,43)	4.31
Positive Cluster 2 (79 voxels, qFDR < 0.001)
L Superior Parietal Lobule	(−27, −58,52)	6.33
Positive Cluster 3 (33 voxels, qFDR = 0.015)
R Precentral Gyrus	(45,5,31)	5.42
R Inferior Frontal Gyrus (p. Opercularis)	(48,8,28)	5.19
Negative Cluster 1 (50 voxels, qFDR = 0.030)
L Superior Frontal Gyrus	(−21,41,37)	−6.27
L Middle Frontal Gyrus	(−21,23,43)	−3.81
B. SSVEP Crosses
Positive Cluster 1 (69 voxels, qFDR = 0.001)
R Superior Parietal Lobule	(24, −61,49)	5.80
R Inferior Parietal Lobule	(30, −55,55)	5.14
Positive Cluster 2 (56 voxels, qFDR = 0.002)
R Middle Frontal Gyrus	(36,2,61)	5.80
R Precentral Gyrus	(33, −4,49)	5.03
Positive Cluster 3 (39 voxels, qFDR = 0.009)
R Inferior Frontal Gyrus (p. Triangularis, p. Opercularis)	(51,14,25); (54,14,34)	5.32; 4.16
Negative Cluster 1 (42 voxels, qFDR = 0.006)
L Cuneus	(−3, −88,19)	−4.39
Negative Cluster 2 (35 voxels, qFDR = 0.007)
L/R Cuneus	(0, −79,31); (15, −79,28)	−4.94; −4.82
R Precuneus	(3, −70,31)	−4.39
C. Perceptual Transitions
Negative Cluster 1 (214 voxels, qFDR < 0.001)
L Supramarginal Gyrus	(−60, −52,37)	−6.64
L Inferior Parietal Lobule	(−51, −55,49)	−5.96
L Angular Gyrus	(−51, −64,31)	−5.65
Negative Cluster 2 (171 voxels, qFDR < 0.001)
L Midcingulate Cortex	(−3, −40,46)	−5.69
R Precuneus	(9, −46,40)	−4.31
Negative Cluster 3 (157 voxels, qFDR < 0.001)
L Middle Frontal Gyrus	(−39,14,55)	−9.31
L Superior Frontal Gyrus	(−15,26,58)	−5.66
L Precentral Gyrus	(−36,8,40)	−5.33
Negative Cluster 4 (53 voxels, qFDR = 0.005)
L Calcarine Gyrus	(−9, −85,4)	−4.61
L Lingual Gyrus	(−9, −76,4)	−4.10
D. SSVEP Peaks - SSVEP Crosses
Negative Cluster 1 (21 voxels, qFDR = 0.246)
R Middle Occipital Gyrus	(30, −88,1)	−4.94
R Inferior Occipital Gyrus	(33, −85,1)	−4.62
E. Perceptual Transitions – SSVEP Crosses
Negative Cluster 1 (50 voxels, qFDR = 0.011)
L Midcingulate Cortex	(0, −31,46)	−5.37
F. SSVEP Peaks - Perceptual Transitions
Positive Cluster 1 (6 voxels, qFDR = 0.581)
R Caudate Nucleus	(9,17,10)	4.40
Negative Cluster 1 (5 voxels, qFDR = 0.581)
L Mid Orbital Gyrus	(−9,50, −8)	4.07

Rivalry-related SSVEP crosses (Fig. 6, B) were similarly associated with increased activity in the right inferior and superior parietal lobules, contrasted with SSVEP crosses for the smooth replay condition. Increased cross-related activity was also observed in the right middle frontal gyrus, right precentral gyrus and right inferior frontal gyrus (p. Triangularis, p. Opercularis) during rivalry, while reduced activity was observed in the bilateral cuneus and right precuneus.

Moving on to BOLD activity related to subjective reports of perceptual transitions, rivalry was associated with significantly decreased activity across numerous cortical regions, compared to the smooth replay condition (Fig. 6, C). Specifically, rivalry-related perceptual transitions were associated with decreased activity in the left supramarginal gyrus, left inferior parietal lobule, left angular gyrus, left mid-cingulate cortex, right precuneus, left middle frontal gyrus, left superior frontal gyrus, left precentral gyrus, left calcarine gyrus and left lingual gyrus.

We further contrasted {SSVEP Peaks-SSVEP Crosses} between the rivalry and smooth replay conditions (Fig. 7, A). Rivalry peaks were associated with decreased activity in the right middle and inferior occipital gyri, compared to crosses, possibly indicating suppression of eye-specific activity and a net decrease in early visual activity (cluster P). However, this difference was not statistically significant following FDR correction, suggesting little to no difference between the rivalry and smooth replay conditions for this contrast. Additionally, we contrasted {Perceptual Transitions-SSVEP Crosses} between the rivalry and smooth replay conditions (Fig. 7, B). Rivalry transitions were associated with significantly decreased activity in the left mid-cingulate cortex compared to SSVEP crosses. Finally, we contrasted {SSVEP Peaks-Perceptual Transitions} between the rivalry and smooth replay conditions (not pictured). Rivalry peaks were associated with increased activity in the right caudate nucleus and deactivation in a small cluster of the left mid orbital gyrus. However, these small clusters of activity did not reach significance following FDR correction.

Fig. 7

Rivalry vs. Smooth Replay Double Contrasts. Initial contrasts for {SSVEP Peaks-SSVEP Crosses} and {Perceptual Transitions-SSVEP Crosses} were evaluated for the rivalry condition using a [1, −1] contrast vector, and then contrasted against the respective maps for the smooth replay condition for first-level analysis, followed by second-level analysis and FDR cluster correction (primary clustering threshold p < 0.001 uncorrected, FDR corrected p < 0.05). Note that the small cluster of early visual activity shown in A was not statistically significant following a multiple comparisons correction (cluster puncorrected = 0.013).

Discussion

We have conducted a simultaneous EEG-fMRI study during continuous binocular rivalry for a group of 20 healthy human subjects. To the best of our knowledge, this is the first study to investigate both the electrophysiological and hemodynamic correlates of binocular rivalry concurrently using multimodal neuroimaging integrating EEG with fMRI. A novel EEG-informed fMRI analysis was utilized for assessing hemodynamic correlates of the SSVEP signatures of rivalry. Specifically, we assessed hemodynamic activity differences between SSVEP envelope peaks and crosses, for comparison with hemodynamic activity associated with perceptual transitions reported by subjects. This allowed us to evaluate the spatio-temporal dynamics of BOLD activity during binocular rivalry without depending on subjective perceptual reports alone. Our results reveal the importance of both a right-lateralized fronto-parietal network and posterior regions of the default mode network (DMN) for resolving perceptual changes during binocular rivalry. BOLD activity in the right superior parietal lobule (rSPL) was strongest during subject reported perceptual transitions, but otherwise continuously elevated throughout periods of binocular rivalry, compared to replay. Increased BOLD activity in the rSPL and associated network was also associated with perceptual transitions for the smooth, but not instantaneous, replay condition. Differences in fronto-parietal and early visual BOLD activation were minimized when directly comparing SSVEP peaks and crosses. Collectively, these findings suggest that increases in fronto-parietal BOLD activity during rivalry are predominantly related to attentional processes and perceptual awareness of changes in visual activity, while differential BOLD suppression in the DMN, notably in the posterior mid-cingulate cortex, may relate to additional attentional processes mediating information flow between early visual and fronto-parietal networks.

Previous fMRI studies of rivalry have been constrained by the use of button-press related perceptual transitions for assessing BOLD activity related to rivalry, making it difficult to delineate cortical networks which could be modulated at other points during rivalry, namely before or after perceptual transition times. Critically, we employed a novel SSVEP-informed analysis to delineate BOLD activity related to spontaneous alternations in SSVEP dominance for comparison with BOLD activity associated with subjective perceptual transitions. As has been previously observed, we found elevated BOLD activity in right-lateralized fronto-parietal networks during rivalry-related perceptual transitions. Strong activation in the rSPL during subject reported perceptual transitions was observed for both the rivalry and smooth replay conditions, but not for the instantaneous replay condition, as has been reported in the literature (Knapen et al., 2011; Lumer et al., 1998). However, we also observed enhanced activity in the rSPL/IPL throughout rivalry (relative to the smooth replay condition), at both SSVEP peaks and crosses, suggesting a continuous role of the rSPL/IPL in mediating rivalry. Previous studies have suggested that rSPL activation during rivalry is related to attention, which is required for rivalry to occur (Zhang et al., 2011; Brascamp and Blake, 2012). It is possible that the sustained rSPL BOLD activation we observed in our study was predominantly related to maintaining attention throughout the presentation of the visual stimuli.

Structural differences in the rSPL and its connectivity with the surrounding anatomy have also been previously associated with individual differences in perceptual rivalry using bistable stimuli (Wilcke et al., 2009; Kanai et al., 2010). However, a recent study found negligible fronto-parietal BOLD activity related to unreportable changes in visual activity during binocular rivalry, suggesting that regions other than the rSPL are mediating rivalry in early visual areas (Brascamp et al., 2015). Similarly, minimal fronto-parietal activity has been observed in experiments employing continuous flash suppression, where changes in visual rivalry still occur but are invisible to the subject (Zou et al., 2016). Our findings challenge the notion that increased rSPL BOLD activity triggers perceptual transitions, given that we observed differential but sustained activation in the rSPL throughout binocular rivalry which was maximized during subject-reported perceptual transitions. Strong rSPL activity during rivalry-related SSVEP peaks and crosses could reflect the increased attentional load during rivalry, compared to the replay condition.

In addition to elevated activity in the rSPL/IPL, we found significant suppression of the DMN throughout binocular rivalry, compared to both replay conditions. This is not by itself provocative, since it is well known that the DMN becomes suppressed during cognitive engagement. However, we found that activity in the posterior mid-cingulate cortex was specifically reduced during rivalry-related perceptual transitions. We suggest that the mid-cingulate could operate as a mediator of information between early visual areas and fronto-parietal attentional networks for the resolution of rivalry. Indeed, previous studies have shown that different sub-regions of the cingulate cortex are essential for integrating information across widespread cortical regions (Fransson and Marrelec, 2008). The cingulate cortex has been implicated in mediating conscious awareness, and is a key player in a number of intrinsic cortical networks responsible for cognitive control (Vogt and Laureys, 2005; Leech and Sharp, 2014). In particular, the mid-cingulate cortex has been implicated with functions including reward assessment, error detection, pain perception and even awareness of others’ decisions (Tolomeo et al., 2016; Vogt, 2016; Brown and Jones, 2008; Rolls, 2009; Apps et al., 2013). Given the ambiguous nature of the visual stimuli during perceptual transitions, it is plausible that suppression of the posterior mid-cingulate network contributes to perceptual stabilization by directly influencing the information entering one’s conscious awareness following an ambiguity.

On the other hand, deactivation in the posterior mid-cingulate cortex during perceptual transitions could represent the detection of SSVEP envelopes switching dominance. This suggests an error-detection role of the mid-cingulate during rivalry that could be critical for initiating changes in perceptual awareness and subsequent attentional focus. Alternatively, the mid-cingulate could simply be involved in deciding how to characterize what one perceives during perceptual transitions. Mid-cingulate regions have also been associated with motor control, including eye and head movement. During rivalry, increased attentional load and increased eye movements may go hand-in-hand during perceptual mixing periods, necessitating the increased recruitment of the mid-cingulate cortex for focusing the eyes. Given this potential role, it is likely that damage to the mid-cingulate cortex would result in significant deficits in attentional processing, as has been found in patients with lesions to this region (Tolomeo et al., 2016; Amiez et al., 2013).

Several groups have attempted to study the role of the posterior parietal cortex in rivalry using transcranial magnetic stimulation (TMS), a non-invasive method for stimulating the cortex with electromagnetic energy. However, these studies have been inconsistent in their findings likely due to stimulating different sub-regions of the parietal cortex that are involved in rivalry, with some reporting reduced dominance durations following rSPL stimulation (Carmel et al., 2010) and others reporting increased dominance durations following stimulation of the rSPL/rIPS (Kanai et al., 2010; Zaretskaya et al., 2010). It is possible that offline inhibitory rTMS of the right SPL prior to binocular rivalry could reduce overall indices of rivalry, due to disruption of the fronto-parietal attentional network. If we assume that the rSPL is involved in maintaining perceptual stability, inhibitory rTMS could potentially lead to a reduction in overall perceptual stability, increased rates of perceptual transitions and reductions in dominance durations, as was reported by Carmel et al. (2010). Alternatively, offline inhibitory theta burst TMS of the rSPL could potentially reduce perceptual switching rates and enhance dominance durations if the rSPL is involved in triggering perceptual alternations, as was observed by Kanai et al., (2010). Importantly, Kanai et al. (2010) found that effect was hemisphere-specific to the rSPL and not the lSPL. Zaretskaya et al. (2010) also found a significant increase in dominance durations during 2 Hz rTMS over the right intraparietal sulcus (rIPS) only and not the rSPL, suggesting a more prominent role of the rIPS in triggering perceptual alternations. Collectively, it is clear that multiple regions of the posterior parietal cortex may be influencing rivalry dynamics, making target specification for TMS even more important at the individual subject level for future studies of rivalry.

Importantly, we found sustained but differential activation in the rSPL throughout all rivalry blocks, which could explain the varying behavioral outcomes that have been reported in studies transiently administering TMS to different sub-regions of the posterior parietal cortex for the study of rivalry. rTMS at lower frequencies has been shown to have subject specific effects and can be both excitatory and inhibitory, depending on factors such as anatomy, brain state, etc. Thus, transient stimulation of the posterior parietal cortex, regardless of the hemisphere, would likely have unpredictable results if the stimulation is applied without heed to such factors, given that fronto-parietal networks appear to be transiently modulated throughout rivalry. Our findings provide evidence that the rSPL is activated during perceived changes in early visual activity – regardless of stimulus condition, suggesting that activation in this region is not necessarily involved in driving perceptual transitions.

Future studies of rivalry could benefit from additional multimodal imaging approaches, such as combining two imaging modalities, as we did in this study with EEG and fMRI, or combining imaging modalities with non-invasive neuromodulation techniques. Combining real time EEG, fMRI or ultimately EEG-fMRI imaging with TMS or transcranial direct/alternating current stimulation (tDCS/tACS) for modulation of specific cortical regions during rivalry could significantly advance our understanding of the specific role of the observed networks in resolving rivalry. Thus, future research in this area should aim to optimize the stimulation parameter space with the specific goal of modulating rivalry-related activity, both neural and behavioral. Such an effort would significantly advance our knowledge of functional and effective connectivity within and across various brain networks related to perception and awareness.

Conclusion

To the best of our knowledge, this is the first study to utilize an SSVEP-informed fMRI approach for concurrently assessing the electrophysiological and hemodynamic correlates of binocular rivalry in a group of healthy human subjects. Our results point to an interplay between the visual cortex, the right superior/inferior parietal lobules and the posterior mid-cingulate cortex for resolving perceptual changes during rivalry. We suggest that fronto-parietal increases in BOLD activity during rivalry are predominantly related to attentional processes and perceptual awareness of increased conflict in early visual regions, while suppression of the mid-cingulate cortex may relate to additional processes mediating information flow between early visual and fronto-parietal regions necessary for perceptual awareness. Additionally, our findings reiterate that suppression of eye-specific information during rivalry is rapidly resolved within early visual networks. This work highlights the utility of combining information across imaging modalities in order to obtain data-driven measures of cognitive activity and has significant implications for future studies of perception, awareness and the neural correlates of consciousness.