Hemispheric Differences within the Fronto-Parietal Network Dynamics Underlying Spatial Imagery.

Spatial imagery refers to the inspection and evaluation of spatial features (e.g., distance, relative position, configuration) and/or the spatial manipulation (e.g., rotation, shifting, reorienting) of mentally generated visual images. In the past few decades, psychophysical as well as functional brain imaging studies have indicated that any such processing of spatially coded information and/or manipulation based on mental images (i) is subject to similar behavioral demands and limitations as in the case of spatial processing based on real visual images, and (ii) consistently activates several nodes of widely distributed cortical networks in the brain. These nodes include areas within both, the dorsal fronto-parietal as well as ventral occipito-temporal visual processing pathway, representing the "what" versus "where" aspects of spatial imagery. We here describe evidence from functional brain imaging and brain interference studies indicating systematic hemispheric differences within the dorsal fronto-parietal networks during the execution of spatial imagery. Importantly, such hemispheric differences and functional lateralization principles are also found in the effective brain network connectivity within and across these networks, with a direction of information flow from anterior frontal/premotor regions to posterior parietal cortices. In an attempt to integrate these findings of hemispheric lateralization and fronto-to-parietal interactions, we argue that spatial imagery constitutes a multifaceted cognitive construct that can be segregated in several distinct mental sub processes, each associated with activity within specific lateralized fronto-parietal (sub) networks, forming the basis of the here proposed dynamic network model of spatial imagery.

Spatial Imagery – A Multifaceted Cognitive-Psychological Construct

Humans are capable of performing a variety of higher order cognitive abilities such as problem solving, reasoning, contemplating, but also language comprehension, object recognition, spatial orientation, or the vivid re-experience of previously perceived or processed information stored in memory. All of these cognitive functions require, and are to a large extent based on, our ability to generate, inspect, and manipulate inner mental representations of objects, events, and scenes that are not physically present. This ability of mental imagery thus describes a multi-facetted set of cognitive processes that are at the heart of most forms of abstract reasoning or contemplating (Kosslyn et al., 1995; Cohen et al., 1997; Kanwisher and Wojciulik, 2000; Riesenhuber and Poggio, 2000).

While mental imagery by itself is a multifaceted psychological construct that shows conceptual and neurobiological overlap with related cognitive processes such as attention and memory, it is useful to also subdivide mental imagery according to the sensory modality based on which the mental representation is generated. In this sense, the processes that are involved in generating, inspecting, and manipulating visual images in the absence of visual input are referred to as visual mental imagery (Finke, 1989). Objects in visual imagery can be manipulated much like actual objects. Hence, a mentally generated inner image can easily also be mentally transformed, distorted, or rotated in our mind. This can help to reason about the consequences of a potential corresponding physical manipulation (Kosslyn et al., 1998). (Visuo)Spatial imagery particularly refers to the inspection and evaluation of spatial features (e.g., distance, relative position, configuration) and/or the spatial manipulation (e.g., rotation, shifting, reorienting) of mentally generated visual images. When we speak about spatial imagery in the remainder of this article, we thus refer to the mental representation of visual objects, events, or scenes which are either mainly defined by spatial characteristics (e.g., the visual imagination of a spatial configuration) and/or which require in addition to the mere generation of the mental representation, a spatial analysis or manipulation to be mentally performed upon this mental visual image.

Spatial imagery, just like all forms of imagery, is by definition a subjective, private experience that cannot be measured directly, but has to be empirically inferred by indirect measures. These measures vary from subjective self reports on the vividness or size of the mental image, to more objective tasks such as mentally rotating a visually presented object to assess whether it matches, or is mirrored to, a second visual object (Shepard and Metzler, 1971). It is believed that the completion of such mental rotations rely, at least partly, on spatial mental imagery. In line with this rational, some studies have shown that mental rotation tasks are indeed performed by mentally rotating an object as if it were moving through the intermediate positions along a trajectory, as would occur if the object was physically rotated (Kosslyn et al., 1998; Carpenter et al., 1999; Richter et al., 2000). Since accuracy and response latency of these mental spatial rotations can be objectively measured and compared with other experimental conditions of, e.g., real manual rotation (Sack et al., 2007), such psychophysical experiments offer a means for assessing spatial imagery performance in a behaviorally more controlled manner.

Although it appeared that the question of which exact brain areas are activated during spatial imagery largely depends on the specific features of the imagery task being investigated, e.g., which spatial operation has to be performed based on which mental object, the emerging picture of brain imaging studies is that our capability to mentally visualize, inspect, and manipulate objects is subserved by distributed cortical networks that include regions that are similarly activated when performing comparable perceptual operations (Thompson et al., 2009; Cichy et al., 2011; but see also Lee et al., 2011, nicely showing that although imagery and perception have similar neural substrates, they may involve different network dynamics; Seurinck et al., 2011). Another important and converging finding of these previous imaging studies is that both conceptually and in terms of underlying neural mechanisms it seems important to distinguish cortical regions and neural mechanisms involved in tasks that require participants to mentally represent specific object categories (Ishai et al., 2000) or specific features of objects (e.g., color, size, shape), from those cortical regions and neural mechanisms involved in tasks that explicitly require processing of spatially coded information or spatial manipulation (Trojano et al., 2000). This distinction of object versus spatial imagery can be regarded as analogs to the dichotomy between ventral (what) versus dorsal (where) information processing during visual perception (Mishkin and Ungerleider, 1982; Mishkin et al., 1983;Haxby et al., 1991, 1994).

The Neurobiological Segregation of what and where during Spatial Imagery

The neurobiological mechanisms underlying spatial imagery are characterized by widely distributed cortical networks with a multitude of nodes and interaction-patterns in the brain. Numerous neuropsychological (Levine et al., 1985; Farah et al., 1988) and neuroimaging studies (Cohen et al., 1996;Mellet et al., 1996, 1998; D’Esposito et al., 1997; Richter et al., 1997; Knauff et al., 2000; Trojano et al., 2000) have aimed at unraveling the neural foundations of mental imagery using a wide variety of imagery tasks (for a review see Kosslyn et al., 2001). These imaging studies have consistently revealed that the pure imagination and mental representation of a specific mental object results in neural activity within category-specific occipital-temporal regions of the ventral visual processing pathway (Ishai et al., 2000, 2002; O’Craven and Kanwisher, 2000), including superior occipital areas (Mellet et al., 1995, 1996; D’Esposito et al., 1997; de Borst et al., 2011), inferior temporal regions (Carpenter et al., 1999; Mechelli et al., 2004; de Borst et al., 2012), parahippocampal cortex (de Borst et al., 2012), and in some tasks early visual cortex (EVC; Stokes et al., 2011) and/or even primary visual cortex (Kosslyn et al., 1999; Slotnick et al., 2005; de Borst et al., 2012). Likewise, brain regions within the dorsal visual processing pathway are recruited during the spatial processing or manipulation of these mental representations (Kawashima et al., 1995;Mellet et al., 1995, 1996; Cohen et al., 1996; Tagaris et al., 1997; Kosslyn et al., 1998;Sack et al., 2002, 2005, 2008). These cortical regions within the dorsal pathway that in this sense are maybe more strictly related to the spatial aspect of spatial imagery are the bilateral inferior and superior parietal lobule (SPL; Richter et al., 1997; Knauff et al., 2000;Trojano et al., 2000, 2002;Sack et al., 2002, 2005, 2008), bilateral intraparietal sulcus (IPS); precuneus; (Mellet et al., 1996; Trojano et al., 2000;Sack et al., 2002, 2005, 2008), middle forntal gyrus (MFG), supplementary motor area (SMA), frontal eye fields (FEF), and premotor cortex (PMC; Kawashima et al., 1995;Mellet et al., 1995, 1996; Cohen et al., 1996; Tagaris et al., 1997; Kosslyn et al., 1998; Richter et al., 2000; Trojano et al., 2000; Lamm et al., 2001;Sack et al., 2002, 2005, 2008; Sack, 2009; de Borst et al., 2012). Regarding this spatial aspect of spatial imagery, Thompson et al. (2009) suggested differentiating between visualizing spatial locations versus mentally transforming locations, both relying on distinct neural sub networks within the dorsal pathway. Concretely, whereas the visualization of spatial locations recruited mainly areas within occipito-parietal sulcus, medial posterior cingulate, and precuneus, mental spatial transformations were correlated more with activation in superior portions of the parietal lobe and in the postcentral gyrus. Still, since any spatial inspection or manipulation during spatial imagery requires some sort of mental (object) representation upon which the spatial operation can be based and performed on, the core neural network of spatial imagery typically includes brain areas of both the dorsal fronto-parietal as well as ventral occipito-temporal visual processing pathway. Figure 1 depicts this core network of spatial imagery in the brain, segregated and color-coded in order to distinguish the spatial dorsal (red-colored) from the content ventral (rose-colored) network in the brain activated during spatial imagery.

Figure 1

The core neural network of spatial imagery. This figure depicts the neural network of spatial imagery including brain areas of both the dorsal fronto-parietal (red-colored) as well as ventral occipito-temporal (rose-colored) visual processing pathway. It summarizes in one figure the different regions identified in various imagery studies as described in the Section “The neurobiological segregation of what and where during spatial imagery” of the current manuscript. Most prominent regions within the dorsal fronto-parietal network include bilateral SPL, superior parietal lobe; IPS, Intraparietal sulcus; MFG, middle forntal gyrus; DLPFC, dorsolateral prefrontal cortex; FEF, frontal eye fields; PMC, premotor cortex; Precuneus; and SMA, supplementary motor area. Most prominent regions within the ventral occipito-temporal network include bilateral EVC, early visual cortex; IT, inferior temporal cortex; IO, inferior occipital cortex; PHG, parahippocampal gyrus. The mesial superior frontal gyrus (mSFG) plays a special integrative role in the context of spatial imagery and is therefore color-coded separately.

Hemispheric Lateralization during Spatial Imagery

As can be seen in Figure 1, most functional imaging studies show bilateral fronto-parietal networks to be activated during the execution of spatial imagery. The fact that both, left and right posterior parietal cortex (PPC; mostly SPL and IPS) is recruited during spatial imagery, is, at first glance, in contrast to most neuropsychological studies on patients with focal brain lesions which generally propose a dominant role of the left hemisphere in visual imagery (Farah et al., 1985; D’Esposito et al., 1997). In a critical clinical review on visual mental imagery, Trojano and Grossi (1994) presented a number of single cases as well as group studies which demonstrate a dominant role of left posterior parietal areas for mental imagery. Nonetheless, the authors also reported evidence of the role of the right hemisphere in visuospatial imagery as well as in perceptual visuospatial processing. Right brain damaged patients with neglect also show neglect symptoms in imagery tasks, and non-neglect right hemisphere patients show visuospatial deficits in perceptual visuospatial processing and during imagery tasks. While the left hemisphere seems to have a specific role for mental imagery, the right hemisphere seems to be of a more general relevance for visuospatial functions (Trojano and Grossi, 1994).

Our group has contributed to the question of hemispheric lateralization within bilateral parietal cortex (PC) during spatial imagery by using conventional functional magnetic resonance imaging (Trojano et al., 2000), fMRI mental chronometry (Formisano et al., 2002), repetitive (Sack et al., 2002), and time-resolved (Sack et al., 2005) transcranial magnetic stimulation (TMS) experiments. By using a spatial imagery task that involves the generation as well as spatial comparison of mental images, we demonstrated, using event-related fMRI, that the bilateral parietal activity associated with this task includes a temporal activation sequence from left to right PC. When relating and modeling different features of the fMRI responses to the behavioral measures, we found that the duration of activation of the early left parietal activation and the onset of the (late) right parietal activation correlate with reaction time during spatial imagery performance. These results support the involvement of both parietal lobes in mental imagery, but suggest that each parietal lobe might have a distinct functional role at different moments in time. The sequential activation from left to right suggests that the early left and late right parietal activation during spatial imagery support different components of the cognitive process, for example the generation and subsequent analysis of the visual image. We therefore concluded that within the bilateral PPC activity during spatial imagery, the left PPC underlies the generation of mental images, while the right PPC subserves the spatial processing upon these images.

Such modular models of spatial imagery that propose a division of labor between hemispheres in which the generation of mental representation from memory rely primarily on structures in the posterior left hemisphere, while spatial operations upon these mental representation rely primarily on structures in the posterior right hemisphere, also provide a solution to the aforementioned apparent discrepancy between lesion and imaging studies with regard to the hemispheric lateralization of visual imagery, and are in this sense in agreement with both neuropsychological lesion as well as brain imaging findings of spatial imagery (for review see Sack, 2009). However, from fMRI measurements alone one cannot assess the exact functional necessity or behavioral contribution of a given brain area for a specific mental sub process, such as mental image generation versus spatial analysis. We therefore used non-invasive functional brain stimulation to focally and transiently disrupt neural processing in either left or right PC during spatial imagery, and assessed the respective behavioral effect of this unilateral functional lesion within PPC on spatial imagery performance (Sack et al., 2002). This brain interference study revealed a hemisphere-specific effect of parietal stimulation with only the right parietal disruption leading to spatial imagery impairments. These results contribute new constraints to the modular model of bilateral activation in spatial imagery and are at first glance not in accordance with the aforementioned hemispheric lateralization and division of labor between hemispheres during spatial imagery. Indeed, if left PPC underlies the generation of mental representations and right PPC reflects the spatial operations upon these mental representations, a suppression of either of these brain regions should result in impaired spatial imagery performance. Fortunately, based on a combined fMRI and transcranial magnetic stimulation study (Sack et al., 2002), we were able to further fractionate specialized processing components in the right PPC and revealed the existence of highly dynamic compensatory mechanisms between the left and right hemisphere during the execution of spatial imagery (Sack et al., 2005). This study suggested that although the left PPC is predominantly specialized in mental image generation and the right PPC in spatial comparisons of imagined content, the right hemisphere is also able to immediately compensate for (virtual) lesions of the left hemisphere by taking over its specific function, but not vice versa. Hence, in case of left parietal functional lesion, the right PC will now subserve both functions, mental image generation, and spatial analysis of the mental image. Discrepancies across studies concerning the hemispheric lateralization during mental imagery likely arise because different aspects of imagery are carried out by different parts of a bi-hemispheric neural network. The fact that an isolated deficit of the ability to generate inner visual images following unilateral lesion is clinically hardly reported could also be explained on the basis of the compensatory processes revealed in our study (Sack, 2009, 2010). Interestingly, such hemispheric asymmetries between left and right PPC apply to both, the direct functional relevance (only right parietal disruption leads to behavioral impairments) as well as ability of inter-hemispheric compensation (right PC can compensate for left PC, but not nice versa).

Dynamic Anterior-to-Posterior Brain Network Connectivity during Spatial Imagery

The execution of various spatial imagery paradigms consistently activates core areas of the dorsal fronto-parietal visual pathway, including bilateral parietal, prefrontal, and premotor areas (Kawashima et al., 1995;Mellet et al., 1995, 1996; Cohen et al., 1996; Richter et al., 1997; Tagaris et al., 1997; Kosslyn et al., 1998; Carpenter et al., 1999; Knauff et al., 2000;Trojano et al., 2000, 2002; Lamm et al., 2001;Sack et al., 2002, 2005, 2008; de Borst et al., 2012). With regard to the spatial processing component of spatial imagery, a strong focus has been put on the bilateral PPC activation due to the prominent role of PPC within the dorsal spatial processing stream. Carpenter et al. (1999) correlated the increment of reaction time during mental rotation of cubes with changes in regional cerebral activation. With higher angular disparity, activation increased in the parietal lobes bilaterally, but not in the temporal lobe. This discrepancy indicates that the PC plays a central role in the visuospatial transformations of mental rotation (Goebel et al., 1998; Formisano et al., 2002; Sack et al., 2002) whereas the ventral (temporal) pathway, which is essential for identifying a figure, does not specifically support this operation. This again seems to strengthen and justify the prominent role of PPC in spatial imagery research. However, importantly, this study also revealed that the activity in the motor areas of the frontal lobe was significantly higher during the mental rotation paradigm as compared to a motor control condition, suggesting that “the so-called motor areas are not simply involved in motor planning and execution” (Carpenter et al., 1999), but play a crucial role in the computation of imagined motion of objects as well. In a similar vein, Richter et al. (2000) used fMRI to investigate the participation of the neocortical motor areas in the Shepard and Metzler’s (1971) mental rotation task. Seven regions of interest (ROIs) were analyzed separately: Left and right SPL, SMA, and left and right premotor areas. The results showed that the observed activation in premotor areas was likely related to the very execution of the mental rotation task (Richter et al., 2000).

These studies thus indicate the potential functional contribution of prefrontal and premotor brain areas during spatial imagery. The question remained, however, whether these prefrontal and premotor activities during mental rotation tasks are more related to the potential involvement of visual working memory rather than being critical neural structures for the visual imagery process per se. Or in other words, what would happen to the here identified premotor and prefrontal activations in case of pure visual imagery, i.e., when generating mental representations of objects that have never been perceived before? In such cases, the generation of visual images does not result from the reactivation of previously stored memories but does result from an online construction of images based on the processing of, e.g., verbal instructions and their encoding in a visual format. Mellet et al. (1996, 1998, 2000) used PET to monitor regional cerebral blood flow variations while participants were constructing mental images of objects made of three-dimensional cube assemblies from acoustically presented instructions. Compared to a control condition, the mental construction task specifically activated a bilateral occipito-parietal-frontal network, including the superior occipital cortex, the inferior PC, and again also the PMC. These studies thus suggest that in addition to the well-established functional role of posterior parietal cortices during spatial imagery, also the prefrontal and premotor activations revealed during imagery task are of direct functional relevance for the imagery performance and likely also sub serve specific cognitive sub functions within the multifaceted cognitive-psychological construct of imagery. However, as described above, while some of the previous functional imaging studies on spatial imagery tried to subscribe different cognitive (sub) functions to, e.g., left versus right PPC (Formisano et al., 2002;Sack et al., 2002, 2005), a systematic investigation of the specific functional contribution and/or the exact spatio-temporal interactions with the always co-activated prefrontal and premotor brain regions during spatial imagery is missing. All studies discussed so far either focused exclusively on the PPC and neglected all additionally activated brain regions (Trojano et al., 2000; Formisano et al., 2002;Sack et al., 2002, 2005) or simply descriptively reported the co-activation of anterior premotor and prefrontal brain regions within large fronto-parietal networks (Mellet et al., 1998, 2000; Carpenter et al., 1999; Richter et al., 2000), without analyzing the network dynamics between these anterior premotor/prefrontal and posterior parietal activations during spatial imagery. These shortcomings were mainly due to respective boundaries in spatial and temporal resolution of the functional imaging techniques being used, and due to the unavailability of more advanced analyses tools for functionally segregating the acquired brain imaging data into separate networks of effective brain connectivity.

In our recent work, our group aimed to address this shortcoming (Sack et al., 2008). Participants were asked to mentally construct an inner image of either an entirely new object or a new configuration of objects, simply based on either visually or verbally presented instructions. Importantly, participants were required to construct and spatially rotate these abstract mental images generated from sequentially presented instructions, meaning that only pieces of the final mental object were provided in successive steps, sequentially building up the final mental object that had never been perceptually encountered as a whole. This behaviorally controlled spatial imagery paradigm was investigated using time-resolved event-related functional magnetic resonance imaging and analyzed based on data-driven and multivariate fMRI analysis tools. By accounting for the full spatial pattern of brain activity measured simultaneously at many locations, we functionally segregated an early from a late premotor-parietal-occipito-temporal-cortex (OTC) and late premotor-prefrontal activation network. We revealed, using effective brain connectivity analyses, that the information coming from sensory brain regions was first sent to bilateral PMC and then to bilateral medial dorsal PC. The early left PC received additional input from bilateral occipito-parietal regions. One might thus speculate that this early bilateral premotor-(medial dorsal) parietal activation network underlies the online processing of the sequentially presented modality-independent spatial instructions (Corbetta et al., 1993; Jonides et al., 1993; Haxby et al., 1994; Courtney et al., 1996; Prabhakaran et al., 2000). In contrast, the identified late premotor-parietal network showed a clear hemispheric difference with only the late left PMC projecting back to bilateral parietal regions while at the same time sending neural signals to bilateral OTC. This specific effective connectivity network might thus represent the juxtaposing of the sequentially presented stimuli and thus the successive construction of the slowly emerging final mental object representation during imagery (Roland and Gulyas, 1995;Mellet et al., 1996, 1998; D’Esposito et al., 1997). During this late premotor-parietal activity network, neural input was also sent to bilateral prefrontal cortex (PFC). This late bilateral premotor-prefrontal activation network might sub serve the necessary maintenance of the spatially rotated visual mental object in spatial short-term working memory (Tulving et al., 1994; Buckner et al., 1995; Moscovitch et al., 1995; Roland and Gulyas, 1995; Smith et al., 1995; Courtney et al., 1996; Cohen et al., 1997). These findings suggest that the activation flow underlying the construction and spatial transformation of visual mental images first recruits premotor regions which then project to, or receive information on demand from, parietal regions in a top-down manner, putting prefrontal, and PMC into a new and central focus also during higher cognitive functions (Sack et al., 2008). Such functionally coupled activations of the parietal and premotor cortices have also been described for other cognitive functions (Abe and Hanakawa, 2009) and visuospatial tasks, such as spatial localization (Haxby et al., 1994) or shifting of spatial attention (Corbetta et al., 1993), and in situations explicitly involving the spatial working memory (Jonides et al., 1993; Courtney et al., 1996). The exchange of information between the premotor regions and the dorsal route thus appears to be a general feature during spatial processing, whatever the nature of the initial input. It is thus likely that the parietal “perceptual” pole and the frontal “motor” pole systematically exchange spatial information, whether a motor action is envisioned or not, thus executing the encoding of a spatial environment in its descriptive and behavioral aspects.

Integration between Dorsal and Ventral Pathways during Spatial Imagery

In the previous sections, we have described and shown that within the dorsal fronto-parietal network activated during spatial imagery, a direction of information flow seems to exist from anterior frontal/premotor to posterior parietal cortices. However, in addition to the dorsal fronto-parietal network, all imagery paradigms also activate several nodes of the ventral occipito-temporal visual processing pathway (Ishai et al., 2000, 2002; O’Craven and Kanwisher, 2000), including superior occipital areas (Mellet et al., 1995, 1996; D’Esposito et al., 1997; de Borst et al., 2012), inferior temporal regions (Carpenter et al., 1999; Mechelli et al., 2004; de Borst et al., 2012), parahippocampal cortex (de Borst et al., 2012), and in some tasks even primary visual cortex (Kosslyn et al., 1999; de Borst et al., 2012). This seems to make perfect sense because any spatial operation or spatial processing in spatial imagery requires some sort of mental (object) representation upon which the spatial inspection or manipulation can be based and performed on. Subsequently, the core neural network of spatial imagery includes brain areas of both the dorsal fronto-parietal as well as ventral occipito-temporal processing pathway, representing the “what” and “where” aspects of spatial imagery (see Figure 1).

An open question, however, then is how the necessary dynamic interaction between the “what” and “where” aspects of spatial imagery is integrated in the brain. de Borst et al. (2011) directly addressed this question by investigating the functional role of the frontal regions and their interaction with the “what,” “where” and early visual regions during complex visuospatial scene imagery. Participants were required to perform detailed scene imagery that captured both object and spatial mental imagery aspects, while measuring all imagery-relevant network nodes in the brain and their relative temporal onset of activation using event-related fMRI. In addition, electroencephalography (EEG) was recorded to validate the fMRI latency results and to derive more information on the underlying functional roles from the involved frequency bands. This study could nicely demonstrate that the “what” and “where” aspects of spatial imagery are integrated into one visually imagined scene by the frontal regions, including PMC, right MFG, and mesial SFG (mSFG). The mSFG seemed to be most crucial for this integration process because it was activated earliest and predicted later imagery performance. The early and behaviorally relevant involvement of the frontal regions suggest that these frontal regions indeed “orchestrate” the ventral occipital-temporal “what” and dorsal parietal “where” regions during spatial imagery. Hence, the frontal regions and in particular the mSFG seem to integrate those areas encoding the detailed mental representation with those areas encoding the spatial layout, in order to form and maintain the subjective experience of one coherent mental picture. In line with this interpretation, several working memory studies have also suggested that mSFG plays a role in the integration of visual and spatial features during visual short-term memory (VSTM; Mitchell et al., 2000; Prabhakaran et al., 2000; Munk et al., 2002). In accordance with the already described anterior-to-posterior information flow within the dorsal fronto-parietal network during spatial imagery (Sack et al., 2008) this study also showed a rather late involvement of bilateral PPC, suggesting again that these PPC activations seem to be drawn on by the frontal regions during the later stages of image construction, rather than being a dorsal starting point of spatial imagery. In line with this integrative role of the anterior premotor and prefrontal regions in spatial imagery, Abe and Hanakawa (2009) also suggested that the functional interplay through the prefrontal-premotor connections may mediate the integration of specific sub-operations for multi-step cognitive manipulation.

Dynamic Network Model of Spatial Imagery

In an attempt to summarize, converge, and integrate the brain imaging findings of our own and other groups on spatial imagery over the past 10 years, Figure 2 depicts the here proposed new dynamic network model of spatial imagery.

Figure 2

Dynamic network model of spatial imagery. This figure depicts the here proposed dynamic network model of spatial imagery. Spatial imagery consistently activates several nodes within both, the dorsal fronto-parietal as well as ventral occipito-temporal visual processing pathway (silver-shaded areas). Most prominent regions within the dorsal pathway during spatial imagery include bilateral PC, Parietal Cortex; PMC, Premotor Cortex; and PFC, Prefrontal Cortex. Likewise, most prominent regions of the ventral pathway activated during spatial imagery are located along the Occipito-Temporal-Cortex (OTC), and include inferior temporal regions and parahippocampal cortex, but also superior occipital areas and in some conditions even primary visual cortex (silver-shaded areas). During spatial imagery, these two pathways can be labeled as representing the CONTENT (curve color-coded in light blue) versus SPATIAL (curve color-coded in dark blue) aspects of spatial imagery. The areas within the fronto-parietal dorsal network dynamically exchange information during spatial imagery with a direction of information flow from anterior frontal/premotor regions to posterior parietal cortices. An early bilateral PMC-PC (solid curve color-coded in dark red) can be segregated from a later left-lateralized PMC-PC-OTC activation network (solid curve color-coded in red). Spatial imagery thus first recruits bilateral anterior premotor cortices, which then send neural information to, or receive on demand neural information from, bilateral parietal cortices. This model thus proposes that the well-established PPC activation during spatial imagery seems to be drawn on by the frontal regions at later stages in the course of imagery, rather than being a dorsal starting point of spatial imagery, as previously suggested. Moreover, in order to form one coherent mental visuospatial picture, all these content and spatial aspects and segregated processing stages of spatial imagery need to be integrated at a brain system level. This integration process is done by the mesial SFG. Mesial SFG orchestrates remote ventral occipital-temporal “what” regions and dorsal parietal “where” regions in order to integrate areas encoding the detailed mental visual representation with those areas encoding the spatial layout or manipulation. Finally, the late neural dynamic information flow between bilateral PMC and PFC (curve color-coded in light red) represents the necessary maintenance of the now spatially processed or manipulated and thus integrated mental object in (spatial) visual short-term working memory.

Spatial imagery consistently activates several nodes within widely distributed cortical networks in the brain. Importantly, these nodes include areas within both, the dorsal fronto-parietal as well as ventral occipito-temporal visual processing pathway (all silver-shaded areas in Figure 2). Most prominent regions within the dorsal pathway during spatial imagery include bilateral PC, PMC, and PFC. Likewise, most prominent regions of the ventral pathway activated during spatial imagery are located along the OTC, and include inferior temporal regions and parahippocampal cortex, but also superior occipital areas and in some conditions even primary visual cortex (see Figure 2, silver-shaded areas).

The parallel processing within dorsal and ventral networks makes perfect sense in case of spatial imagery. It shows that indeed the imagination and mental representation of any specific mental object or scene always results in neural activity within category-specific occipital-temporal regions of the ventral visual processing pathway. This is true for all forms and aspects of visual imagery, whether or not it includes a spatial processing component. In case of spatial imagery, however, additional brain regions within the dorsal visual processing pathway are recruited because of the here required processing of spatially coded information and/or spatial manipulation of these mental representations. In this sense, the two pathways can be labeled as representing the CONTENT (Figure 2; curve color-coded in light blue) versus SPATIAL (Figure 2; curve color-coded in dark blue) aspects of spatial imagery (Figure 2).

Importantly, the areas within the fronto-parietal dorsal network dynamically exchange information during spatial imagery with a direction of information flow from anterior frontal/premotor regions to posterior parietal cortices. A spatio-temporal segregation has been suggested, dissociating an early bilateral PMC-PC (Figure 2; curve color-coded in dark red) from a later left-lateralized PMC-PC-OTC activation network (Figure 2; curve color-coded in red). Importantly, spatial imagery thus first recruits bilateral anterior premotor cortices, which then send neural information to, or receive on demand neural information from, bilateral parietal cortices. This model thus proposes that the well-established PPC activation during spatial imagery seems to be drawn on by the frontal regions at later stages in the course of imagery, rather than being a dorsal starting point of spatial imagery, as previously suggested.

So far, the model describes the parallel processing within the ventral CONTENT and dorsal SPATIAL network of spatial imagery, as well as the dynamic anterior-to-posterior information flow within the dorsal fronto-parietal activation network. However, in order to form one coherent mental visuospatial picture, all these aspects and segregated processing stages need to be integrated at a brain system level. We could recently suggest that this integration process is likely done by the co-activated frontal regions, in particular the mSFG. mSFG seems to literally orchestrate remote ventral occipital-temporal “what” regions and dorsal parietal “where” regions in order to integrate areas encoding the detailed mental visual representation with those areas encoding the spatial layout or manipulation. In line with this idea, neural dynamic information flow during spatial imagery was revealed also late between bilateral PMC and PFC, likely representing the necessary maintenance of the spatially processed or manipulated and thus integrated mental object in (spatial) visual short-term working memory (Figure 2; curve color-coded in light red).

Quo Vadis Spatial Imagery?

Besides the specific questions on the neurobiology, lateralization, and/or spatio-temporal dynamics within and across the described fronto-parietal and occipito-temporal network activity during spatial imagery, a striking conceptual uncertainty occurs on how to differentiate the concept of spatial imagery to related phenomena such as spatial attention or spatial visual working memory. While these related processes are often treated as separate cognitive-psychological constructs, it is undisputed that the ability to maintain visual information online in working memory largely depends on mental imagery, and likewise, performing spatial manipulations upon mentally generated visual images shares large conceptual overlap with the concept of (covert) visuospatial attention and memory. In an attempt to combine aspects of the so-called analog (James, 1890; Paivio, 1971; Kosslyn, 1980; Kosslyn and Ochsner, 1994) versus propositional (Anderson and Bower, 1973; Pylyshyn, 1973) theory of mental imagery, Kosslyn (1980) proposed a computational hybrid imagery model consisting of three basic components: a visual buffer, long-term stored representations, and image-processing operations. Visual images are generated by retrieving information from long-term visual memory (LTM) and constructing them in a spatial format in the visual buffer. Although the LTM contains both analog and propositional components, the final visual image is an analog representation. Images in the buffer can be mentally manipulated and transformed (e.g., by rotation or scanning) and then be inspected for new information (Palmer, 1999); a process likely largely mediated by spatial attention mechanisms. In line with this depictive view of visual mental imagery, Slotnick et al. (2005) demonstrated that visual mental imagery can evoke cortical activity with precise visual field topography, i.e., imagery-induced retinotopic maps that are similar to the perception maps.

As evident from the attempts to segregate and subscribe different cognitive sub functions to the revealed sub networks during spatial imagery using functional imaging, these psychological labels or cognitive-psychological constructs of the revealed brain activation pattern often have to fall back on other cognitive-psychological constructs such as attention and memory. In fact, attention and memory processes are consistently used to explain and describe the various mental sub processes and their underlying neural activations during the execution of (spatial) imagery. This also holds true the other way around: attentional modulation occurs in the absence of any phenomenal experience, e.g., in form of expectations. Kastner et al. (1999) described attention-modulated activity in fronto-parietal areas and in primary visual cortex during a visuospatial attention task without visual stimulation. Regions in parietal and frontal cortex responded when observers covertly pay attention to a peripheral location in expectation of the stimulus occurrence. Therefore, attention is not necessarily associated with visual perception. However, although several neuroimaging studies have shown that attention can modulate extrastriate cortical regions within the dorsal and ventral processing streams (Kanwisher and Wojciulik, 2000; Martinez et al., 2001; Yantis et al., 2002), the role of the primary visual cortex remains controversial (Kastner et al., 1999; Posner and Gilbert, 1999; Sengpiel and Hubener, 1999). But what psychological and neural mechanisms are responsible for the maintenance of spatial attention in the absence of visual stimuli? And what are the conceptual differences to spatial imagery and spatial working memory processes? Mental imagery is defined as a perceptual experience in the absence of an appropriate physical stimulus (Finke, 1989). Conceptually, attention and imagery can both be characterized by specific top-down processes that modulate extrastriate cortex activity in the absence of visual stimulation. On a neuronal level both, spatial imagery and spatial attention show immense functional overlapping: mental imagery is generally, like attention, accompanied by an activation of a fronto-parietal network (Trojano et al., 2000; Formisano et al., 2002; Sack et al., 2002). Several imagery studies have shown that mental imagery, like attention, can modulate extrastriate cortices (Le Bihan et al., 1993; Roland and Gulyas, 1995; D’Esposito et al., 1997). Finally, some imagery studies have even, like attention, revealed primary visual cortex activity during visual imagery (Kosslyn et al., 1999; de Borst et al., 2011; Slotnick et al., 2005). However, the involvement of occipital areas during imagery is, like in attention, still a matter of debate (Kosslyn and Ochsner, 1994; Roland and Gulyas, 1995; Mellet et al., 2000). The ability to maintain spatial information online in memory depends on spatial attention and might be mediated by spatial imagery. Imagery might in general aid any cognitive functions in the respective sensory domain. It has, for example, been claimed that visual imagery ability correlates with visuospatial memory span (Kail, 1997). Keogh and Pearson (2011) showed that performance in visual working memory – but not iconic visual memory – can be predicted by the strength of mental imagery.

This raises the question whether spatial imagery, spatial attention, and memory in the end represent very similar or even identical neuronal processes that can only be differentiated on a psychological level as servants for a thinking framework among neuroscientists. On a neuronal level, these two or three so far rather independently investigated constructs might prove to recruit similar or even identical neural structures. Hence, what might be a very crucial and important semantic and conceptual difference for psychology and neuroscience might for our brain in the end simply require the recruitment of identical networks of in this sense identical processes.

However, while this might in the end indeed show to be the case, such a conclusion would at this point be premature. One problem of most paradigms investigated so far in the context of visual imagery and/or VSTM is the fact that they are usually conceptually confounded by not representing pure operationalizations of the cognitive-psychological construct under investigation. Most cognitive and neuroscientific studies of mental imagery required participants to learn and memorize visual images, or recall images from past autobiographical experiences. Such “imagery” paradigms are by definition confounded by processes involving visual working or episodic memory.

Hence, it might very well be that despite all the conceptual and apparent neurobiological overlap between imagery, attention, and memory, clear differences and neurobiological segregations can be found and described on the basis of, e.g., cleaner experimental designs directly comparing the different processes, and/or more elaborated multivariate brain imaging analysis tools. Another elegant approach might be to use functional brain interference techniques such as transcranial magnetic stimulation (TMS) to chart the time point at which the identified similar neural structures activated by imagery and memory are functionally relevant for either or both of these processes. Using this approach, Cattaneo et al. (2009), e.g., compared the functional contribution of EVC in short-term memory retention and visual mental imagery at different moments in time using chronometric TMS. They could show that short pulses of functional interference applied to EVC at the start of the retention interval increased reaction times for the memory trials, but not for the imagery trials; while later TMS pulses over EVC affected both processes, memory and imagery. In other words, while both visual imagery and VSTM recruited identical neural structures within EVC, the time point of functional recruitment was significantly different between these two processes. This enabled the authors to segregate between imagery and memory in the temporal rather than spatial neural domain. More studies like these, directly comparing and segregating the neurobiology underlying imagery, attention, and memory within one experimental session are needed to clearly separate and differentiate the different mental sub processes involved in spatial imagery from those involved in attention and memory. In the end, we should be able to explain the individual cognitive, phenomenological, psychological, and behavioral differences within spatial imagery (see, e.g., Borst and Kosslyn, 2010), as well as the differences between imagery, attention, and memory, by linking each of these processes to different underlying neurobiological processes and spatio-temporal network dynamics in the brain.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.