The Significance of Memory in Sensory Cortex.

Early sensory cortex is typically investigated in response to sensory stimulation, masking the contribution of internal signals. Recently, van Kerkoerle and colleagues reported that attention and memory signals segregate from sensory signals within specific layers of primary visual cortex, providing insight into the role of internal signals in sensory processing.

The processing of sensory signals comprises only 1–2% of energy changes in the brain; we assume that this processing in sensory areas is driven by the physical properties of our environment. By contrast, internal brain processing (i.e., signalling which does not directly derive from feedforward sensory processing) uses up to 90% of energy consumption, but we have little understanding of its content [1]. One of many candidate processes contributing to these internal signals is memory. For example, visual long-term memory has substantial storage capacity, encoding thousands of images in detail everyday [2]. However, even in a massively parallel processing brain it remains a mystery how such a depth of information can be stored so efficiently. A key limitation to such understanding is the fact that the cerebral cortex remains a mystery. Specifically, how information about the external world is maintained in cortical representations, and how these representations determine future processing. Neurocomputational models of brain function posit that cognitive functions such as memory could interact with processing even in sensory cortex 3, 4. Such frameworks challenge feedforward models of sensory processing, yet forward models, lacking a contribution of internal signals, dominate experimental designs. For example, primary visual cortex is the first cortical projection of the retinal input; feedforward models posit that it accurately reflects the outside world (in a manner analogous to the projection screen of a pinhole camera). Overlaid onto this cortical projection of the retinal inputs are top-down internal signals; the challenge to understand these internal signals in V1 is enormous because we lack intuitive ways to understand their representation. Until recently, no micro-electrophysiological studies have examined the contribution of memory to neuronal responses in non-feedforward stimulated layers of primary visual cortex. In a recent article in Nature Communications, van Kerkoerle et al. [5] found layer-specific neural correlates of working memory and visual attention in monkey primary visual cortex (V1).

Vision does not start from a blank slate. That is, perception incorporates knowledge that we have acquired previously (the literal translation of the German word for perception, ‘Wahrnehmung’ means ‘truth taking’). Yet the brain has largely been conceived of and studied as a feedforward-driven entity; thus, how top-down cortical feedback pathways carrying internal brain signals determine sensory processing is less well understood [6]. Empirical challenges in studying feedback have exacerbated this knowledge gap. One solution is to measure top-down feedback signals in the absence of bottom-up information, in essence allowing the experimenter to eavesdrop on the private life of the brain [7]. van Kerkoerle et al. [5] harnessed the advantage of this approach to characterize properties of cortical feedback in V1.

van Kerkoerle et al. [5] reported evidence for the influence of working memory on the activity of V1 neurons. The authors examined current-source density responses in monkey V1 during both attention and working memory tasks in which the monkeys mentally traced curved lines. During the memory task the stimulus momentarily disappeared requiring the monkey to maintain a representation of the sensory input in order to complete the task. They observed neural signatures of attention and working memory in specific layers of V1 cortex; attentional effects in sensory cortex have been studied previously but the finding is particularly novel for our understanding of memory in sensory cortex. With regards to memory, the activity was outside the temporal window for iconic memory; it was found in the superficial and deep layers of V1, that is, those that receive cortical feedback inputs; it was present in the complete absence of a visual stimulus; and it reappeared after the presentation of a visual mask. These findings have several noteworthy implications. First, few have recorded non-sensory signals in primary sensory areas so convincingly [8] and none with the criteria necessary to ascribe these signals to memory. Here, we broadly define memory as a brain state that exists for longer than the input that triggered the state. In the data of van Kerkoerle et al. [5], the memory trace is more than just maintained, it is reinstated after a visual mask. The presentation of the visual mask has the effect of separating the memory trace from the sensory signal that caused it, providing strong evidence of an internally-generated brain signal in sensory cortex. Moreover, this internal representation is informative in the service of the task; that is, it is relevant to the ongoing cognitive requirement. Lastly, this memory-related activity is seemingly transferred to V1 via cortical feedback. Not only does this finding challenge feedforward models of brain function, but it reveals specific cognitive influences on early sensory processing.

It is possible that these cortical memories of sensory signals contribute to forming higher level internal representations of the environment which are then used for predicting future inputs, a hypothesis which requires feedback [9] and permits the (re)appearance of representations [10]. Memory in this sense is not for processing the past, but for predicting the future. In this context, the data of van Kerkoerle and colleagues open up challenging questions, including: When is working memory encoded and retrieved and what is the temporal capacity of the neuronal mechanisms? How could this activity support a complex stimulus that is represented invariantly in higher cortex but represented retinotopically in early visual cortex? How does this activity shape the processing of new sensory signals? There are multiple memory systems in the brain; what function does cortical memory support? Which mechanisms underlie working memory and which ones allow for longer-term storage? Do the observed activity traces contribute to the divergent reconstruction of memory [6] or are they part of storage? Cortical areas engaged in perception may retain the information that they initially represent in a feedforward manner. However, the data of van Kerkoerle et al. [5] imply something more complex than a mere persistence of sensory representation because the working memory activity they observed was both fed back and reappeared after a mask. Internal models could use memory to reinstate a sensory representation, as if drawing lines in the sand after a wave of sensory inputs. Ultimately, these lines drawn into the sand might hold clues to understanding one’s mind.