Measuring away an attentional confound?

A recent fMRI study by Webb et al. (Cortical networks involved in visual awareness independent of visual attention, Proc Natl Acad Sci U S A 2016;113:13923-28) proposes a new method for finding the neural correlates of awareness by matching attention across awareness conditions. The experimental design, however, seems at odds with known features of attention. We highlight logical and methodological points that are critical when trying to disentangle attention and awareness.

One challenge in finding the neural correlates of awareness is separating awareness from attention. Webb ) recently claimed to have achieved this feat in a study using functional magnetic resonance imaging (fMRI). As in many previous studies, they compared a visible stimulus with a stimulus rendered nearly invisible by masking. They assessed how much spatial attention was drawn to these stimuli by using them as cues in a cueing paradigm. Specifically, they measured the reaction time difference (Δt) of discriminating the orientation of a subsequent target when it appeared on the same versus the opposite side as the visible/invisible cue. Importantly, this cueing effect was tested only at a single cue-target onset asynchrony (CTOA) of 180 ms. Webb ) argue that attention was “balanced” between the visible and invisible cues, since the cueing effect (Δt) at this time point was non-significantly different between visible (25 ms) and invisible (17 ms) cues.

However, in another study (Webb ), the same authors have shown that with the same protocol, probing the cueing effects of visible and invisible stimuli at other time points leads to clear differences in Δt. In other words, the time course of the attentional effects evoked by the visible versus invisible stimuli used in their fMRI study (Webb ) is known to be very different. This behavioral observation suggests that, against the authors’ claim, attention is unlikely to be drawn in the same way by visible and invisible stimuli while subjects’ brains were being scanned, despite the lack of a statistical difference when measuring the cueing effects (Δt) at one selected time point. Instead, the visible versus invisible fMRI contrast probably includes activity related to the different attentional effects of the two types of stimuli. Just as we cannot actually conclude that a sports car and a bicycle are equally fast by measuring their speeds at a single point in time, we cannot actually match the attentional neural effects of different types of cue by just measuring them at a time known least likely to reveal a behavioral difference.

Perhaps the authors could argue that even though the cueing effects differ between visible and invisible stimuli when measured at different times (say, within a 100–600 ms range), over a longer period (say, 2 seconds or the approximate temporal resolution of fMRI), these effects average out in the brain to be similar. This could be the case because there were crossovers of the cueing effects of visible and invisible stimuli throughout the measured time points in (Webb ). However, if that was the rationale, it was not explained as such (Webb ), and it would contradict their explicit goal of focusing on a particular time point when behavioral effects were equalized across awareness conditions. Moreover, attentional effects are multifaceted. There are positive and negative cueing effects, easily assessed by comparing spatially congruent cueing and incongruent cueing separately against a no-cue baseline (Posner ). There is the phenomenon of inhibition-of-return (Posner and Cohen 1984), which affects performance after aware and unaware cues differently (Ivanoff and Klein 2003; Webb ). Furthermore, attentional effects can also be measured not in terms of reaction times but by target discrimination accuracy (which was not reported in [Webb ]). Lumping all these effects together in one incomplete measurement (Δt) does not mean they will all average out in the brain and thus that attention was matched. This point is of particular importance considering this was an fMRI experiment. The 180 ms CTOA used by the authors is substantially shorter than fMRI’s temporal resolution. This makes rather unlikely that the acquired functional data exclusively reveals neural activity pertaining to attention from the probed time point when one specific cueing effect might have been matched behaviorally.

Previous literature has shown that under many circumstances visible stimuli attract attention more effectively (McCormick 1997; Ivanoff and Klein 2003; Webb ). Thus, in order to equalize attention across the visible and invisible conditions, some kind of experimental manipulation is presumably needed. Previous studies achieved this by directing subjects’ attention away from the stimulus (Tse ; Kouider ). Another approach, also not discussed by the authors, is to factorially manipulate attention and awareness independently (Wyart and Tallon-Baudry 2008). In contrast, in the study under discussion, masked and unmasked stimuli were compared without any further manipulation.

Webb and colleagues may disagree that their experiment lacked any manipulation of attention. The authors may argue that having subjects discriminate the orientation of the cued/uncued target was in fact a way of manipulating attention. For instance, by presenting the target, the time course of attention could have been interrupted, thereby nullifying in the brain the commonly observed difference in cueing effects of visible and invisible stimuli when these are independently probed at later time points (Webb ). Again, if that was the logic, no evidence to support it was offered. On the contrary, attentional effects are known to continue with different time courses for visible and invisible cues well after 180 ms (McCormick 1997; Webb ). Importantly, previous studies (McCormick 1997; Ivanoff and Klein 2003) have established that there are differences in the cueing effects across awareness conditions already at earlier times too. A relevant point is that their task captured attention exogenously, and not through cues that probabilistically predicted the location of the target thus putting attention under subjects’ control. This further suggests that the different neural effects of masked and unmasked cueing continued automatically right after cue presentation and remained uninterrupted by target presentation.

Let us further illustrate our main logical point that the neural effects of attention cannot be balanced by simply matching a behavioral measure (Δt) at a single time point. Consider an alternative study where no target was presented after the visible/invisible cues. If subjects were not asked to discriminate the orientation of the target, by necessity there would not be a difference in cueing effects (Δt) because there would not be anything to be measured. Could one argue that because there was no behaviorally measured cueing effect, attention in the brain was balanced for both the visible and invisible cues? If the answer were affirmative, there would be no need to conduct this alternative study because it would be equivalent to numerous studies conducted since the 1990s that just compared visible versus invisible stimuli [for a review, see Dehaene and Changeux (2011)]. If the answer were negative, which seems more plausible, the upshot is that the cueing effect is just a behavioral measure of how effectively the target is processed at the time of presentation. This is so because exogenous spatial attention is drawn to the cues before target presentation and regardless of whether we measure behaviorally such cueing effects with a target or not. Thus, neural activity pertaining to attention drawn to the visible and invisible cues is unlikely to be balanced in their study (Webb ).

We note that here we do not object per se to Webb and colleagues’ neuroimaging data analyses involving local independent component analysis (Webb ). However, the authors’ conclusion that the temporoparietal junction (TPJ) plays a specific role in consciousness may be unwarranted by the present behavioral design in which, as we have been arguing, attention is unlikely to be balanced across awareness conditions. The authors could have analyzed the neuroimaging data of validly and invalidly cued trials separately to substantiate their claim that cueing effects in the brain were truly matched. In the future, it would be interesting to test the authors’ neuroimaging data analysis on a paradigm with several CTOAs, as in (Webb ). Thus, they could test for an interaction between CTOA and cueing effects, and determine whether the latter are truly matched in terms of neural activity at the selected time point of 180 ms in (Webb ). Alternatively, they could apply their analysis to data collected from a task where attention was directly manipulated and, thus, the likelihood of having matched neural effects in aware and unaware trials was higher.

Finding the neural correlates of awareness requires overcoming the difficult challenge of distilling attention from awareness. This can be achieved by testing neuropsychological population, by reducing attention to a minimum or by manipulating attention and awareness independently. Other methods, some of them perhaps yet untested, might be able to distinguish them too. Unfortunately, attention and awareness are entangled in such a way that only a robust attentional manipulation can be expected to single out conscious-specific neural processes.