State-dependent visual processing.

The temporal dynamics and anatomical correlates underlying human visual cognition are traditionally assessed as a function of stimulus properties and task demands. Any non-stimulus related activity is commonly dismissed as noise and eliminated to extract an evoked signal that is only a small fraction of the magnitude of the measured signal. We review studies that challenge this view by showing that non-stimulus related activity is not mere noise but that it has a well-structured organization which can largely determine the processing of upcoming stimuli. We review recent evidence from human electrophysiology that shows how different aspects of pre-stimulus activity such as pre-stimulus EEG frequency power and phase and pre-stimulus EEG microstates can determine qualitative and quantitative properties of both lower and higher-level visual processing. These studies show that low-level sensory processes depend on the momentary excitability of sensory cortices whereas perceptual processes leading to stimulus awareness depend on momentary pre-stimulus activity in higher-level non-visual brain areas. Also speed and accuracy of stimulus identification have likewise been shown to be modulated by pre-stimulus brain states.

Introduction

Trial-to-trial and inter-individual variations in response time are generally accepted as noise and are hence largely ignored. More recently, such variations in both behavioral and neuronal responses are receiving increasing interest since they can shed new light on brain function (Kanai and Rees, 2011). A classic view of brain function is that of an input–output system driven by external stimulation which is examined by studying how physical stimulus properties and task demands are reflected in the neuronal response they elicit. This approach has yielded much of our understanding about the functional organization of the visual system such as domain specificity and functional specialization. Visual information progresses very rapidly along the dorsal and ventral streams that are organized in a hierarchical fashion (Thorpe and Fabre-Thorpe, 2001). However, the anatomical connections between the visual areas are reciprocal, and information does not only travel up the visual streams in a bottom-up fashion, but higher-order areas likewise exert top-down influence onto lower ones (Bullier, 2001), and the relative timing between top-down and bottom-up processes nonetheless remains a matter of debate (Pourtois et al., 2005). In addition to visual areas along the dorsal and ventral stream, non-visual brain areas in parietal and frontal cortex contribute to how we make sense of the visual input. Yet, functional hierarchy does not necessarily imply temporal hierarchy, e.g., higher-level orbito-frontal areas can come into play before lower-level visual areas when stimulus identification becomes difficult (Bar et al., 2006; Kveraga et al., 2007). The identification of the rapid sequence of visual information flow requires measures that provide a high temporal resolution, and the identification of its anatomical correlates requires measures with a high spatial resolution.

Most of our understanding about the temporal aspects of visual processing in humans comes from event-related potential (ERP) studies that examine brain function by manipulating stimulus properties and task demands. ERPs are a direct electrophysiological measure of neuronal activity with excellent temporal resolution. ERPs are extracted by time-locked averaging of the ongoing EEG as a function of stimulus properties or tasks, and they require the extraction of a relatively small signal from the ongoing background activity, which is commonly dismissed as noise (Picton et al., 2000). The magnitude of the phase-locked event-related signal is only a small fraction of the amplitude of the background signal: ERPs have a magnitude of several microvolts, whereas the amplitude of the ongoing EEG has a magnitude of about 100 μV. This in turn implies that the largest part of the measured signal variation cannot be accounted for by the stimulus. Something similar holds for fMRI: relatively small signal changes (1–5% at 1.5 T, 2–10% at 3 T; Krüger et al., 2001) have to be extracted from background physiological noise of similar magnitude (Amaro Jr and Barker, 2006).

In this paper we review recent evidence for the notion that the ongoing spontaneous activity measured with EEG is not merely noise but that it is functionally significant activity that strongly influences stimulus processing and that can explain much of the trial-to-trial response variability. These studies strongly support the notion that the brain is modulated rather than driven by external stimulation.

In humans, state-dependent processing can be probed by assessing qualitative and quantitative differences in the treatment of physically identical stimuli, an approach that has received increasing popularity in recent years. Several phenomena allow taxing this property of the brain: sensory detection and discrimination thresholds, perceptual reversals of multi-stable stimuli such as ambiguous figures and during binocular rivalry, and behavioral differences such as accuracy and reaction time for identical stimuli. These phenomena have in common that the same stimuli undergo different perceptual and behavioral fates that cannot arise from the stimulus but rather as a consequence of differences in the momentary pre-stimulus brain state. We will focus on studies that show trial-to-trial and inter-individual variations in stimulus processing as a function of spontaneous fluctuations of the momentary brain state in humans. Electrophysiological methods provide the necessary temporal resolution to assess such variations on a trial-to-trial basis with sufficient temporal accuracy, which is why we will focus primarily on EEG studies.

Two Complementary Approaches to Study Brain State-Dependent Processing with EEG

The EEG represents the temporal evolution of the electrical field generated by the global neuronal activity of the brain at each instant. It represents the summation of all concurrently active intracranial sources irrespective of their frequency and it is reflected by the momentary topography of the scalp voltage maps. The EEG is recorded non-invasively and with a high temporal resolution from multiple electrodes placed on the surface of the scalp. The EEG measurement can be considered a matrix with temporal information in one dimension and spatial information in the other dimension. Conventional analyses of spontaneous EEG consider the temporal dimension and assess the time course of amplitude and frequency power and phase fluctuations at selected electrode sites. Topographic analyses of spontaneous EEG consider the spatial dimension and assess the temporal dynamics of the EEG scalp field.

EEG frequencies

The human EEG signal is generally subdivided into five frequency bands, which have been ascribed different functional roles. Two frequency bands are of particular interest for state-dependent visual processing – the alpha and the gamma band. The alpha band was literally the first EEG rhythm that has been described (Berger, 1929). It is elicited when subjects close their eyes or enter a relaxed state and encompasses frequencies between 8 and 12 Hz, predominantly over occipito-parietal sites. Its functional significance has been most commonly associated with levels of alertness and cortical excitability to which it is inversely related (Pfurtscheller, 1992): higher spontaneous posterior alpha power signifies lower levels of alertness and cortical excitability.

The gamma band on the other hand encompasses frequencies above 30 Hz and has been most prominently linked with feature binding and conscious awareness according to the idea that different neuronal populations encoding different aspects of the same object coordinate by synchronous firing in the gamma band which ultimately yields the emergence of a coherent percept (Gray and Singer, 1989; Keil et al., 1999; Tallon-Baudry and Bertrand, 1999), a view that has been recently challenged (Yuval-Greenberg et al., 2008; Melloni et al., 2009; Yuval-Greenberg and Deouell, 2009). Moreover, intracranial gamma band synchrony has been identified as a fundamental process underlying attention (Fries et al., 2001; Fries, 2009).

In addition to its power, an oscillation is also described by its instantaneous phase. The phase angle of an oscillation varies on a much faster temporal scale than frequency power, and it reflects the momentary fluctuations of a network’s excitability (Buzsaki and Draguhn, 2004; Lakatos et al., 2008).

EEG microstates

Unlike amplitude, phase and power modulations of the EEG waveforms, which are a local and hence reference-dependent measure of brain activity, the EEG scalp-topography is a global and reference-free measure of momentary brain activity. Moreover, unlike amplitude modulations of the EEG waveforms, the scalp-topography does not change at every instant, but it remains stable for periods of about 80–120 ms before it changes into another quasi-stable configuration. These periods of quasi-stable scalp-topography are termed “EEG microstates” (Lehmann et al., 1987). At rest, four dominant classes of microstates can be identified across the life span (Koenig et al., 2002), and their local syntax has shown to vary as a function of psychiatric (Koenig et al., 1999; Strelets et al., 2003; Lehmann et al., 2005) and neurological conditions (Strik et al., 1995; Dierks et al., 1997) and drug administration (Kinoshita et al., 1995). Consequently, they have been termed the “basic building blocks of cognition” or “atoms of thought” (Lehmann et al., 1998). More recently, we have shown a direct link between the EEG microstates and the resting-state activity in large-scale functional networks identified with fMRI (Britz et al., 2010). This might appear surprising because the EEG and fMRI signals are meaningful at temporal scales that are two orders of magnitude apart: ∼100 ms for EEG microstate changes and ∼10 s for BOLD fMRI oscillations. This direct link between the fast EEG and slow BOLD signal could be identified because the EEG microstate time series have monofractal properties, i.e., they have a temporal organization that is neither random nor determined, but clearly structured (Van De Ville et al., 2010). The spontaneous neuronal background activity commonly dismissed as noise appears to follow a clear organizational principle. In summary, the EEG microstates represent the momentary global state of the brain and they reflect the activity in large-scale functional networks that can rapidly reorganize on a sub-second time scale.

Brain State-Dependent Visual Detection and Discrimination

Sensory detection and discrimination thresholds are most commonly used to assess brain state-dependent information processing since a stimulus presented at detection threshold is perceived in roughly 50% of trials. Sensory detection has been found to vary as a function of pre-stimulus alpha power with detected items being preceded by lower pre-stimulus alpha power than undetected items (Ergenoglu et al., 2004). Sensory discrimination performance for briefly presented and subsequently masked stimuli has also been found to vary as a function of pre-stimulus alpha power both between (Hanslmayr et al., 2005) and within subjects (Hanslmayr et al., 2007). Evidence from MEG shows that the visual discrimination ability can be predicted from the pre-stimulus alpha power which is generated in the parieto-occipital sulcus (van Dijk et al., 2008). Similarly, illusory visual percepts (phosphenes) elicited by means of single-pulse transcranial magnetic stimulation (TMS) have been shown to be modulated by both spontaneous trial-to-trial variations (Romei et al., 2008a) and inter-individual differences (Romei et al., 2008b) in pre-stimulus posterior alpha power and alpha phase (Dugué et al., 2011). In addition to alpha power, alpha phase has been related to whether physically identical stimuli can be detected on a trial-by-trial basis. Both the detection of near-threshold stimuli (Busch et al., 2009) and the efficacy of meta-contrast masking have been shown to be modulated as a function of pre-stimulus alpha phase (Mathewson et al., 2009).

Brain State-Dependent Multi-Stable Perception

Not only sensory detection and discrimination, but also higher-level perceptual outcomes such as the perceptual awareness for the alternations of multi-stable stimuli have been found to vary with pre-stimulus differences in alpha frequency power. Multi-stable perception is a powerful tool to dissociate perceptual awareness from sensory processing. It arises when ambiguous figures have multiple and mutually exclusive perceptual interpretations (e.g., the Necker cube, the Rubin’s face–vase illusion or bistable apparent motion of the dynamic dot quartet) or when two dissimilar monocular images are presented to the eyes (binocular rivalry). These perceptual interpretations alternate spontaneously and randomly and cannot be linked to physical stimulus properties, but they appear to arise from differences in brain state.

Perceptual reversals of multi-stable stimuli have been primarily associated with decreases in alpha and increases in gamma power. Decreased alpha power was observed immediately preceding button presses indicating perceptual reversals of a Necker cube (Isoglu-Alkaç et al., 2000; Isoglu-Alkac and Struber, 2006). Increased gamma power has been identified in the 1000 ms preceding button presses to indicate perceptual reversals of a dynamic dot quartet (Basar-Eroglu et al., 1996) and before perceptual reversals during binocular rivalry (Doesburg et al., 2005). These studies used a continuous presentation of multi-stable stimuli and the analyses were carried out in the time period preceding the button press indicating perceptual reversals. However, reaction times jitter considerably even for very simple reactions, and thus, the precise temporal allocation of the reported variations in frequency power preceding perceptual reversals remains a challenge. Furthermore, these studies remain inconclusive about the respective concomitant neuronal generators of these frequency power changes underlying perceptual reversals.

The use of an intermittent presentation of ambiguous stimuli allows the precise temporal allocation of pre-stimulus EEG events with the onset of perceptual reversals (Pitts and Britz, 2011). The proper titration of the on- and off-times of multi-stable stimuli yields reversal rates comparable to continuous stimulus presentation (Kornmeier et al., 2007; Britz et al., 2009, 2011). We assessed perceptual reversals of the Necker cube (Britz et al., 2009) and during binocular rivalry (Britz et al., 2011) as a function of the pre-stimulus microstate during intermittent stimulus presentation. Both studies found perceptual reversals to arise as a direct consequence of the pre-stimulus EEG microstate by identifying two pre-stimulus microstates that doubly dissociated perceptual reversals from non-reversals. We further assessed their concomitant intracranial sources on the one hand and source differences on the other hand.

The reversals and non-reversals of the Necker cube were preceded by activity in bilateral prefrontal areas implicated in working memory maintenance and manipulation. Analyses of source differences showed that perceptual reversals were preceded by an increased activity in the right inferior parietal lobe. Both frontal and parietal areas have been identified in fMRI studies on multi-stable perception and been interpreted to reflect the appraisal of perceptual reversals (Kleinschmidt et al., 1998). However, the temporal precision of EEG source imaging could show that these higher cortical areas come into play before the occurrence of perceptual reversals (Figures 1A–F).

Figure 1

EEG microstates and perceptual reversals. (A) Ten seconds of spontaneous EEG with eyes open while stimuli are presented. (B) EEG in (A) depicted as a series of scalp maps. (C) Momentary pre-stimulus microstates that discern perceptual reversals from perceptual non-reversals for a complex Necker cube (from Britz et al., 2009). (D) The perceptual awareness of a complex Necker cube alternates stochastically as a function of the pre-stimulus microstate depicted in (B). (E) Source estimates of the concomitant neuronal generators of the reversal-related pre-stimulus microstates reveal activity in bilateral prefrontal areas implicated in working memory maintenance and manipulation. (F) Statistical assessment of their differences reveals activity in right inferior parietal cortex. (G) Momentary pre-stimulus microstates that discern perceptual reversals from perceptual non-reversals during binocular rivalry (from Britz et al., 2011). (H) The perceptual awareness of a binocular rivalry stimulus alternates stochastically as a function of the pre-stimulus microstate depicted in (G). (I) Source estimates of the concomitant neuronal generators of the reversal-related pre-stimulus microstates reveal activity in bilateral occipital and temporal areas. (J) Statistical assessment of their differences reveals increased activity in right inferior parietal cortex preceding perceptual reversals and in bilateral inferior temporal areas preceding perceptual stability.

The perceptual reversals during binocular rivalry likewise arise from differences in pre-stimulus EEG microstates. Unlike for the Necker cube, reversals and non-reversals during binocular rivalry were preceded by activity in primary occipital and lateral and inferior temporal areas. And like for the Necker cube, perceptual reversals were preceded by increased activity in similar areas in right inferior parietal cortex. Perceptually stable trials on the other hand were preceded by increased activity in bilateral inferior and right lateral temporal areas. The parietal areas involved in perceptual reversals have been identified by fMRI in humans (Lumer et al., 1998), and the temporal areas involved in percept stabilization have been identified both by single-cell recordings in monkeys during continuous stimulus presentation (Leopold and Logothetis, 1996) and by fMRI in humans (Sterzer and Rees, 2008). Like for the Necker cube, EEG source imaging can show the temporal precedence of these higher-level areas to come into play before stimulus onset (Figures 1G–J). The evidence from these two studies support the importance of right inferior parietal cortex for the generation of perceptual reversals – it is more active preceding perceptual reversals irrespective of the physical nature of the stimuli whether they are mutually exclusive interpretations of one stimulus or the dominance/suppression of one of two monocular stimuli. These studies show that the pre-stimulus microstates reflect activity in different functional networks which can directly determine the perceptual fate of multi-stable stimuli. Similar microstate-dependent processing differences have been described for the alternations of illusory multi-stable motion perception (Müller et al., 2005). Also non-purely visual processes could be related to the pre-stimulus EEG microstate: the hemispheric specialization for the processing of emotional words was the first process that has been found to depend on the class of pre-stimulus EEG microstates (Mohr et al., 2005).

Brain State-Dependent Differences in Response Accuracy and Response-Speed

Behavioral differences such as accuracy and reaction times have been found to vary as a function of pre-stimulus brain activity. We have shown that the commission of errors in a Stroop task which requires cognitive control to overcome the conflict induced by the stimuli varies with pre-stimulus levels of cognitive control: the averaged evoked topography and its concomitant sources differed between errors and correct trials only before stimulus onset. Pre-stimulus source differences were found in a network comprising dorsal lateral prefrontal areas implicated in cognitive control, and source differences after response execution were found in the network implicated in error detection and conflict monitoring. ERP topographies and their concomitant sources did not differ after stimulus onset and before the response execution. This indicates that the error commission is due to pre-stimulus differences in a network implicated in cognitive control (Britz and Michel, 2010). Even quantitative differences such as reaction times have been shown to depend on pre-stimulus differences in estimated intracranial local field potentials. Faster reaction times in a simple visuo-motor reaction task were predicted by higher intracranial gamma power in frontal and parietal cortex, whereas slower reaction times were predicted by higher intracranial alpha power in extrastriate visual and motor areas. This provides evidence for the role of pre-stimulus neural oscillations as a top-down mechanism that can mediate the response-speed on a trial-to-trial basis (Gonzalez Andino et al., 2005).

Summary

Although visual information processing is generally assessed as a function of stimulus properties, much of the variability in the processing of visually presented stimuli can be accounted for by both trial-to-trial and inter-individual differences in pre-stimulus brain activity. Physically identical stimuli can undergo very different perceptual fates as a function of the pre-stimulus state of both “early” visual and “higher” non-visual brain areas. Very basic sensory processes such as detection or discrimination of simple stimuli have been primarily found to vary with spontaneously fluctuating momentary level of excitability of primary visual cortex as indexed by the magnitude of alpha power. On the other hand, higher-order perceptual processes such as the perceptual awareness for reversals of multi-stable stimuli depend on pre-stimulus activity differences in higher-level non-visual brain areas in parietal cortex, most adequately represented by the pre-stimulus microstate. Similarly, behavioral accuracy can arise as a consequence of reduced pre-stimulus levels of cognitive control in frontal cortex. Thus, the actual sensory processing depends on the momentary state of the visual system whereas perceptual and behavioral differences arise from pre-stimulus activity in frontal and parietal cortices. These studies can furthermore show that functional hierarchy does not imply temporal hierarchy: higher-level non-visual brain areas in frontal and parietal cortex come into play before stimulus onset and largely determine how upcoming stimuli are processed.

The studies reviewed in this article furthermore show that pre-stimulus activity in the baseline period which is commonly dismissed as noise and eliminated by averaging is actually not mere noise, but very well organized activity that carries important functional significance for subsequent stimulus processing.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.