Synchronised neural signature of creative mental imagery in reality and augmented reality.

Creativity, transforming imaginative thinking into reality, is a mental imagery simulation in essence. It can be incorporeal, concerns sophisticated and/or substantial thinking, and involves objects. In the present study, a mental imagery task consisting of creating a scene using familiar (FA) or abstract (AB) physical or virtual objects in real (RMI) and augmented reality (VMI) environments, and an execution task involving effectively creating a scene in augmented reality (VE), were utilised. The beta and gamma neural oscillations of healthy participants were recorded via a 32 channel wireless 10/20 international EGG system. In real and augmented environments and for both the mental imagery and execution tasks, the participants displayed a similar cortico-cortical neural signature essentially based on synchronous vs asynchronous beta and gamma oscillatory activities between anterior (i.e. frontal) and posterior (i.e. parietal, occipito-parietal and occipito-temporal) areas bilaterally. The findings revealed a transient synchronised neural architecture that appears to be consistent with the hypothesis according to which, creativity, because of its inherent complexity, cannot be confined to a single brain area but engages various interconnected networks.

Introduction

The ability of people to find coherences in what occurs to be different is intimately correlated with creativity [1, 2]. Embodying inspiration, sophisticated thinking and exploration, creativity can be incorporeal, concerns sophisticated and/or substantial thinking, and involves objects in isolation or combination. Creativity is the essence of civilisation. It is not only fundamental to music, dance and poetry i.e. arts, but also to sciences and everyday life. Immersed in real and/or artificial environments, the brain constantly integrates information reflected by familiar and/or abstract objects that can be real, imagined and/or virtual, and constructs the corresponding representations [3]. Creativity is the highest-order mental process whose apotheosis is originality in that it assists the emergence of singular ideas. Creativity is an enigma considering that the neural activity that allows one to raise original ideas is insufficiently discerned. Nevertheless, it is continually reported that creativity cannot be reduced into a single neural design. For instance, the exquisite talent of an Archimedes might be better understood in the framework of a multimodal interlinked dynamic neural synchronisation. The question of how the neural synchronisation contributes to creative ideation via physical and virtual objects incorporated in real and augmented atmospheres is central in the study. To answer this question, the current research seeks to investigate the anterior frontal and posterior occipito-temporo-parietal brain area interconnections of beta and gamma oscillations during mental imagery and execution of a scene using familiar and abstract objects in real and augmented environments.

Theoretical background

Associated with mental processes, the conceptual architecture that supports the creative ideation takes place in an organised five-stage model which is expressed on a continuum. These five stages can be explained as follows: preparation, incubation, intimation, illumination, and verification [4]. During preparation a person develops the knowledge needed to design and flourish a creative idea [5]. Incubation represents the mental intention which seeks to find coherence between information [6]. Delicate and ephemeral phenomenon, intimation, i.e. the feeling of knowing [7], resembles creative intuition [8]. Intimation constitutes the link between incubation and illumination and might lead directly to illumination. If intimation can be considered a manifestation of a "rising train of association", illumination, which is equivalent to insight [9] mirrors the culmination of a "train of association" whose realisation expresses the peculiarity of the creative thinking [4]. Specifically, intimations appear to be similar to intuitions and precede insights. However, like intimation, illumination is an elusive phenomenon. On the other hand, verification allows for the structuring and revelation of intimations [10]. The eclectic aforementioned five-stage model of creativity is expressed in terms of "proximity of consciousness". While preparation and verification are accomplished voluntarily through a conscious process, incubation, intimation and illumination are performed smoothly over unconscious or conscious interpenetration and are open to introspection [11]. Potential sources for creative minds, intimations and illuminations, i.e. intuition and insight, are mental representational phenomena involving both cortical and sub-cortical activities that can occur in continuity in real and hybrid environments, i.e. real and virtual environments [12]. Creative ideation naturally relies on mental and cognitive simulation [13], which requires the intervention of internal imagery [14] and allows the representations of visual, motor and/or verbal actions to arise [15, 16]. Such an approach is assisted by abundant scientific literature that describes and validates the neurocognitive overlapping between the mental simulation of an action and the action itself [17]. Intrinsically, creative ideation transcends representations of past, current and future [18, 19].

Considering that the neural oscillations are probably the best anchor of creativity, an interest in using EEG for analysing the dynamic neural correlates of creative ideation has blossomed in recent years [20]. Despite the noticeable heterogeneity across EEG findings, the general consensus is that the variations of alpha oscillations play a major role in creativity [21, 22, 23]. Electrophysiological findings noticed that an increase in alpha rhythm synchronisation mirrors the activation of top-down processes, while an alpha activity decrease in synchronisation depicts the involvement of bottom-up mechanisms [24, 25, 26, 27]. Directly inspired by the classic approach, neuroscientific studies analysed and reported physiological modifications of alpha oscillations in specific brain areas as, for instance, frontal and/or temporal areas [21], rarely occipital and parietal [28, 29]. But, they did not fully consider the occipito-parietal and occipito-temporal pathways and reported that creativity is confined to specific brain areas [30]. This approach is incoherent with the multidimensional nature of creativity which naturally requires neural interconnections among various distributed neural networks. Recent references consistently showed evidence that creative thinking is not related to alpha oscillations or one specific brain area and is not lateralised, i.e. does not specifically involve left or right hemisphere [20, 31, 32, 33].

In current years, the involvement of gamma and beta oscillating neural activities during verbal and/or nonverbal visual creative tasks were investigated by multi-channel EEG measuring. Synchronised beta oscillations were considered symptomatic of increased alertness and active thinking and concentration during creativity [34, 35], whereas synchronised gamma oscillations were emblematic of ideas and/or objects perception and associations that arise in consciousness [36]. Some studies reported desynchronization of beta oscillations or synchronisation [37] in anterior and posterior cortical areas. Others simply contrasted gamma and/or beta oscillations during visual or verbal tasks against rest condition, a baseline in general, and between individuals expressing a distant degree of creativity (i.e. high vs low). Using regular visual and verbal creativity association tests, for instance, greater beta oscillations were recorded into frontal areas [38]. When task-related coordination was analysed along with visual creativity, studies reported synchronisation in beta and gamma oscillations in frontal, fronto-parietal, and occipital brain areas bilaterally compared to the baseline [39]. Whereas in a task involving drawing a picture of arbitrary objects, Nagornova [40] observed primarily an increase in right gamma temporal and parietal synchronisation, and incidentally an increase in left beta frontal and right temporal synchronisation. A decrease in synchronisation in beta and gamma oscillations of the right frontal areas and an increase in synchronisation within the left temporal areas during nonverbal creative processes were also observed [41]. Task-related coherence modifications were often demonstrated by right intrahemispheric synchronisation, i.e. increased functional connectivity, and between frontal and occipital areas for beta oscillations only [42].

Relative to control condition, the above-mentioned findings showing modifications in beta and gamma oscillations are still, imprecise and inconsistent with the neural architecture associated with creativity (visual and/or verbal creativity). One possible reason for this is that neural modifications (i.e. synchronisation or desynchronisation) are habitually observed in any effort necessitating additional cognitive involvement. This is likely because of the broad variety of creativity tasks engaged, and specifically of the involvement of tasks such as reading, visual image examination, or production of words [43] whose creativity value is controversial [33]. In many studies, the analysis of visual creativity-related coherence modifications, beta synchronisation in particular, were associated more with frontal [41, 42, 44, 45, 46] than occipito-temporal brain areas. Even though some studies exhibited right hemisphere influence, recent meta-analysis data did not provide consistent support for the left or the right hemisphere strength in regard to visual creativity [33]. However, as an internal process, creativity inspiration and ideation might be directly and/or indirectly associated with mental imagery activities whose neural expression would be systemic and include at least anterior frontal and posterior occipito-temporo-parietal areas of the left and the right hemispheres [20, 21, 30]. In essence, mental imagery involves all senses, but in creativity research, visual mental imagery is the one usually explored.

Visual imagery appears to engage highly similar regions to visual perception does, and more specifically occipito-temporal areas [47]. EEG findings of visual imagery creativity revealed a decrease in synchronisation over occipital and parietal areas for alpha oscillations only [39] and frontal beta and gamma oscillations of both hemispheres [41]. It was therefore suggested that this posterior occipito-parietal interconnections might reflect top-down modulation and regulation of visual mental processing [48] and frontal regions would have a leading role in imaginative thinking [33, 41, 49, 50, 51]. Nevertheless, it is uncertain to what degree the visual mental processes are affected by bottom-up and/or top-down mechanisms. Creativity inspiration can be involved with recognition and location of visual, haptic, and acoustic objects which can be real, mentally simulated and/or artificial, for instance, virtual objects [52]. Object recognition and location attributes are known to be highly dependent on the neural activity of occipito-temporal and occipito-parietal pathways respectively [53]. Occipito-parietal pathways are not only involved in the perception of object location, but also in the visuomotor coordination tasks associated with objects, including the analysis of different spatial attributes in a visual scene [54]. These pathways transform visual information into cognitive representations specific to object location. Occipito-temporal pathways are assumed to be involved in the visual recognition of the particular features and forms that make up the objects including their details. These pathways are considered to transform visual information into cognitive representations essential for the recognition and identification of objects. Both occipito-temporal and occipito-parietal pathways work in a highly integrated mode and as such, the appearance of objects in a scene would quasi-simultaneously activate neurons in many cortical areas [3]. As far as familiar objects degraded perception are concerned, recent EEG studies concluded that the occipito-temporal and occipito-parietal pathways display graduated, but not completed, separation in the human visual cortex and suggested independence between them [55].

In the present investigation, creativity is appraised of as a sophisticated thinking that transcends representations, and it is associated with mental imagery. In other words, creativity is a mental simulation process in essence. When a multimodal and multidimensional active alertness and conscious phenomenon such as creativity is concerned [56], it is undeniable that there is much to explore in the interplay between both left and right hemispheres [20, 21, 30] and brain regions, and the repercussions of neighbouring brain areas and frontal, parietal, occipito-parietal and occipito-temporal at both beta and gamma oscillations level. From a methodological viewpoint, there are several ways to investigate brain correlates of creativity, one of which is to analyse brain areas interconnections during the accomplishment of a visual imagery creativity task, i.e. imagine creating a scene using physical pictures of familiar (FA) or abstract (AB) objects in real environment; and during the mental imagery and execution of the creativity task using the virtual (disembodied) versions of the same FA and AB objects in an augmented reality environment. The use of augmented reality ambience was initiated by methodological reasons. Through incorporating virtual objects into a real environment, the resulted augmented atmosphere guarantees comparability between the designed experimental conditions. A sample consisting of young adults from the general population, that is, without any specific training on how create a scene via objects and use augmented reality, participated in the study. It was expected that the interconnections between anterior frontal and posterior (i.e. parietal and occipito-parieto-temporal) areas for both beta and gamma oscillations would be high within each condition, i.e. real mental imagery, virtual mental imagery and virtual execution, and that the patterns of neural connectivity would afford similarities between conditions.

Methods

Participants

The feasibility of the study was assessed via G∗Power 3.1. The results have shown that the minimum number required was 31 participants in order to achieve an adequate statistical power of .85 with a medium effect size (d = .50), and alpha level of .05. However, 30 participants (15 males and 15 females) were recruited. The age of participants was 22.13 years old on average (SD = 2.27). All participants were from two Universities. Three of the participants were postgraduates and 27 participants were undergraduates. 21 of the participants were from social sciences, 7 from professional and applied sciences, and 2 from humanities. None of the participants had specific training in arts and augmented reality techniques. They were all from a middle to high socioeconomic background. All participants declared that they had normal or corrected-to-normal vision and no history of vestibular, cardiac or neurological disorders. They all had average visuo-spatial perception and orientation capacities and adequate level of creative mental imagery as demonstrated by the Visual Object and Space Perception Test Battery (VOSP) [57] and the Test of Creative Imagery Abilities (TCIA) [58] respectively. Each participant signed the study consent form. Due to technical issues only 27 of the 30 participants were included in the analysis. The study was approved by the local ethics committee (Bond University Human Research Ethics Committee, i.e. BUHREC 16140) and conformed to the declaration of Helsinki 2.0. Anonymity was guaranteed.

Experimental devices

EEG device

A Mobita 32-Channel Wireless EEG System (Biopac Systems Inc.) was used to record the electrical activity of the brain. Thirty-two electrodes were placed in accordance with the international 10/20 extended system in the following locations: Fp1, Fpz, Fp2, F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6, T7, C3, Cz, C4, T8, TP9, CP5, CP1, CP2, CP6, TP10, P7, P3, Pz, P4, P8, PO, O1, Oz, and O2 equally distributed over the scalp. The reference electrode was placed on the right clavicle. Data were recorded via AcqKnowledge 5.0 software, the sampling rate was set at 1000 Hz. Details with regards to Mobita's quality and reliability are given by Bateson et al. [59].

Augmented reality mounted head device

A self-contained holographic computer, the HolonLens 1st generation Head Mounted Device (HMD) that allows the user to interact with holograms in the 3D augmented reality environment, was used [60]. It is operated through hand gestures picked up by the device's motion sensors and front camera. The device displayed virtual simulations of familiar (FA) and abstract (AB) objects.

Visual stimulations

The stimulations consisted of objects presented either physically, on A4 paper or as virtual incorporated in the 3D augmented reality environment via Hololens 1st generation. There were six Familiar (FA) and six Abstract (AB) physical and virtual objects. All objects were built as follows: the FA were selected from a pre-existing application (HoloLens) but only the six most frequently occurring objects in the English corpus as assessed by Google Ngram Viewer [61] were included. All FAs were namable objects, and existed in the real world. There was a plant, a key, a star, a chair, a painting, and a framework. The ABs were developed using Unity (Version 2018) and were similar to the FA in terms of shape, colour and size. All ABs were not nameable, did not exist in real world, and were unreal in nature. The FA and AB objects are presented in Figure 1.

Figure 1

A: Familiar objets (FA) were selected from a pre-existing application (Hololens) but only the six most frequently occurring objects in the English corpus as by Google Ngram Viewer were considered. There were all namable: a key, a star, a painting, a frame, a chair and a plant. B: Abstract objects (AB) were created using software Unity (version 2018), there were similar to the FA in terms of shape, colour and size. All ABs objects were not namable. The objects (FA and AB) were presented either physically, on a A4 paper or as virtual incorporated in the 3D augmented reality environment via HolonLens 1st generation.

Procedure

The procedure comprised one baseline and three phases (i.e. recognition and training and experimental). A baseline EEG of 1 min was recorded while participants faced a blank wall in the experimental room remaining motionless and non-cognisant. Participants were requested to stay on the same upright position [62, 63] marked by a white ‘X’ during the whole study. Recognition and training phases: While standing, participants completed the recognition phase in which they were presented with the familiar (FA) and the abstract (AB) physical or virtual objects one at a time. Then the participants were trained to imagine pinching and grasping each object (FA or AB) one-by-one independently to create a scene (2–3 min approximately). The experimental phase (Figure 2) consisted of three conditions: Real Mental Imagery (RMI), Virtual Mental Imagery (VMI) and Virtual Execution (VE). During the RMI condition, the participants were instructed to imagine creating a scene in their mind by pinching and grasping all six FA or AB objects one-by-one independently and putting them together. During the VMI condition, the participants were immersed in AR via the HolonLens and they were asked to imagine pinching and grasping each FA and AB virtual object and create a scene within the augmented reality environment. In the VE condition, the participants were immersed in the same AR environment as previously and were instructed pinching and grasping each of the FA and AB incorporated virtual objects to create a scene in the 3D augmented environment. In each condition (RMI, VMI and VE), participants disposed of 2 min and 30 s to create the scene with FA objects, and another 2 min 30 s to create the scene with AB objects. The inter-condition duration time was 2 min approximately. At the end of each time period and for both FA and AB objects within each experimental condition, the participants were asked to give a title to their scenes. Continuous EEG recording was obtained for the creation of the scenes with FA and AB separately, i.e. 2 minutes and 30 s for FA and 2 min and 30 s for AB in each experimental condition: RMI, VMI, and VE.

Figure 2

The experiment phase comprises three conditions: RMI (Real Mental Imagery), VMI (Virtual Mental Imagery) and VE (Virtual Execution). In the RMI participants were equipped with the mobita EEG, in the VMI and VE conditions participants were outfitted with EEG and the AR (Hololens) device. In all experimental conditions participants standing in the experimental room (i.e. on a white "X"). During the RMI, the participants were instructed to imagine creating a scene in their mind by pinching and grasping all six FA or AB objects one-by-one independently and putting them together. During VMI condition, the participants were immersed in AR via the HolonLens. As previously, participants were asked to imagine pinching and grasping each FA and AB virtual object and create a scene within the augmented reality environment. In the Virtual Execution (VE) condition, the participants were instructed to use the FA and AB incorporated virtual objects to create a scene in the 3D augmented environment. In each condition, the participants were told to remain in the same position (i.e. indicated by a white X), not to move their head and were not allowed to resize, rotate, or break apart the object and use all the objects. In each condition (RMI, VMI and VE), participants disposed of 2 minutes and 30 seconds to create the scene with FA objects, and another 2 minutes 30 seconds to create the scene with AB objects. The inter-condition duration time was 2 minutes approximately. At the end of each time period and for both FA and AB objects within each experimental condition, the participants were asked to give a title to their scenes. Continuous EEG recording was obtained for the creation of the scenes with FA and AB separately, i.e. 2 minutes and 30 seconds for FA and 2 minutes and 30 seconds for AB in each experimental condition: RMI, VMI, and VE.

Data analyses of EEG signal processing

To preprocess and process the EEG data, MATLAB (Version R2020b) was used along with FieldTrip toolbox [64]. The preprocessing script contained a high-pass filter of 1 Hz and a low-pass filter of 80 Hz. Bad channels and high-amplitude EEG artefacts were automatically removed. Supplementary artefacts particularly associated with electromyogram activity, electrooculogram activity and electrocardiogram activity, were removed manually after visual inspection by experts and using independent component analysis (ICA) methods. Additionally, all data were visually inspected by two independent experts and all remaining artefacts were manually eliminated. The total percentage of retained trials was about 92%. All trials with EOG artefacts (3.9%) and EMG artefacts (4.1%) were eliminated. The processing script performed a beta (13.5–30 Hz) and gamma (55–80 Hz) frequency analysis. To investigate all cortical areas bilaterally the 32 electrodes were organised into 4 regions of interest (ROIs). The correspondence between each ROI and electrodes resulted from the combination of both left and right brain hemispheres as follows: left and right frontal (F7, F3, Fz, F4, F8, FC5, FC1, FC2, FC6), left and right parietal (CP5, CP1, CP2, CP6, Pz), left and right occipito-parietal (PO, P7, P8, P3, P4) and left and right occipito-temporal (O1, Oz, O2, T7, T8). The frequency analysis resulted in an average power spectral density measured in microvolts per Hertz (mV2/Hz) in each of the aforementioned bilateral regions of interest for both beta and gamma oscillations.

Statistical analysis

Statistical analysis was performed in SPSS software package version 27.0. All variables were normally distributed according to Shapiro-Wilk statistics (p > .05) and Levene's test was non-significant, thus equal variance could be assumed. Consequently, a series of parametrical tests was performed. Specifically, paired sample t-tests and repeated measured ANOVA were used to compare the average beta (12.5–30 Hz) and gamma (55–80 Hz) brain oscillations between familiar (FA) and abstract (AB) objects in each experimental condition (RMI, VMI and VE) for each region of interest (frontal, parietal, occipito-parietal and occipito-temporal) bilaterally (i.e. both hemispheres). For both hemispheres, EEG functional connectivity was inferred on the basis of statistical dependencies expressed by cross-correlations (i.e. nodes) between distant neural activity's computations [65, 66]. For all three conditions the dynamic functional connectivity between brain areas within each oscillation band (beta and gamma) was explored using pairwise Pearson correlation coefficient (r) [65].

The multivariate approach Principal Component Analysis (PCA) was performed to identify unknown but dynamic bilateral structured brain patterns present during the creativity task, i.e. correlograms [67, 68]. Each correlogram expresses the temporal correlation of multiunit recordings of separable neuronal spike trains recorded simultaneously from the brain areas (frontal, parietal, occipito-parietal and occipito-temporal), and oscillatory bands of interest beta (12.5–30 Hz) and gamma (55–80 Hz) and experimental condition (RMI, VMI and VE). Based on the correlograms, functional connectivity organises computations of the distributed brain activity. Prior to performing PCA, the suitability of data for a Factor Analysis was assessed. Despite the sample size (N = 27), it was deemed acceptable as no more than 5 factors were expected and that the communalities were all in the range of 0.50, confirming that each item shared some common variance with others [69] and stability [70]. The inspection of the correlation revealed the presence of many coefficients greater than 0.50 suggesting reasonable factorability. The Bartlett's test was significant (p < .001), further supporting the factorability of the correlation matrix.

Results

Familiar (FA) and abstract (AB) objects

Beta and gamma anterior frontal and posterior (occipito-parieto-temporal) oscillations were compared between familiar (FA) and abstract (AB) objects within the three experimental conditions (RMI, VMI, and VE). Both the repeated measure ANOVA and paired sample t-test provided similar results (see Table 1 and Table 2), the t-test results will be reported here. The significant level for the ANOVA and t-test analysis was fixed at α = .10 to reduce the risk of type I and II errors [71]. Each of the paired sample t-tests run for this analysis revealed no significant differences in brain activity between FA and AB objects across the three conditions (p > .10) for each brain frontal anterior and posterior (parietal, occipito-parietal and occipito-temporal) areas oscillations (beta and gamma) (p > .10), implying that whatever the characteristics (FA or AB) and the constitution (physical or virtual) of the objects were, all participants processed them similarly. Given the above, FA and AB were pooled for each brain area and oscillation within each experimental condition.

Table 1

Comparisons between familiar (FA) and abstract (AB) objects within each experimental condition (RMI, VMI and VE) for each beta neural oscillations area (anterior frontal and posterior occipito-parietal and occipito-temporal). Both paired sample t-test and repeated measure ANOVA provided non significant results. The significant level for both statistical tests was fixed at =.10 to reduce the risk of type I and II errors.

		Familiar (FA) vs Abstract (AB) objects	T	F	p
Beta	RMI	Frontal	0.07	0.05	.945
		Parietal	-0.62	0.38	.543
		Occipito-Parietal	-0.49	0.24	.630
		Occipito-Temporal	-0.85	0.71	.406
	VMI	Frontal	-1.17	1.37	.257
		Parietal	0.81	0.66	.429
		Occipito-Parietal	1.58	2.49	.131
		Occipito-Temporal	1.42	2.01	.171
	VE	Frontal	1.15	1.31	.269
		Parietal	-0.26	0.07	.796
		Occipito-Parietal	-0.61	0.37	.550
		Occipito-Temporal	-0.75	0.56	.464

Note. Comparison between Familiar (FA) and Abstract (AB) objects. T = paired sample t-test; F = ANOVA; p = significance value.

Table 2

Comparisons between familiar (FA) and abstract (AB) objects within each experimental condition (RMI, VMI and VE) for each gamma neural oscillations area (anterior frontal and posterior occipito-parietal and occipito-temporal). Both paired sample t-test and repeated measure ANOVA provided non significant results. The significant level for both statistical tests was fixed at =.10 to reduce the risk of type I and II errors.

		Familiar (FA) vs Abstract (AB) objects	T	F	p
Gamma	RMI	Frontal	-1.48	2.19	.152
		Parietal	0.38	0.15	.704
		Occipito-Parietal	-0.09	0.01	.931
		Occipito-Temporal	-0.09	0.01	.927
	VMI	Frontal	-0.28	0.08	.782
		Parietal	0.32	0.10	.753
		Occipito-Parietal	0.06	0.00	.950
		Occipito-Temporal	0.87	0.75	.396
	VE	Frontal	0.78	0.61	.447
		Parietal	-0.57	0.32	.577
		Occipito-Parietal	0.34	0.11	.740
		Occipito-Temporal	-0.09	0.01	.927

Note. Comparison between Familiar (FA) and Abstract (AB) objects. T = paired sample t-test; F = ANOVA; p = significance value.

Functional connectivity within each experimental condition

Pearson correlations

Given the complexity of creativity, it was expected that high cortico-cortical connectivity would characterise each experimental condition (RMI, VMI and VE) in each oscillation (beta and gamma). Figures 3 and 4 illustrates the measuring pairwise Pearson's correlation coefficient (r) of all nodes activity time-series between the brain areas of interest (frontal, parietal, occipito-parietal, and occipito-temporal) bilaterally for each oscillation (beta and gamma) and within each experimental condition (RMI, VMI and VE). To control for Type I and II error a False Discovery Rate (FDR) adjustment was calculated at p < .006 [72]. There was clear evidence of medium to strong positive and negative correlations between the selected brain areas, indicating that these areas were functionally connected. More particularly, in the RMI condition, beta oscillations (Figure 3) appeared positively correlated between parietal and occipito-parietal (r = .71, p = .001, r = .50) areas, occipito-parietal and occipito-temporal (r = .61, p = .005, r = .37) areas and negatively correlated between frontal and occipito-parietal (r = - .79, p < .001, r = .62) and frontal and occipito-temporal (r = - .66, p = .002, r = .44) areas. Positively correlated beta oscillations were also observed between parietal and occipito-parietal (r = .67, p = .002, r = .45), and occipito-parietal and occipito-temporal (r = .65, p = .002, r = .42) areas, and negatively correlated beta oscillations were recorded between frontal and parietal (r = -.60, p = .005, r = .36), frontal and occipito-parietal (r = -.69, p = .001, r = .48), and frontal and occipito-temporal (r = -.83, p < .001, r = .69) areas in the VMI condition. When the VE condition was considered, positive beta oscillations were noticed for occipito-parietal and occipito-temporal (r = .76, p < .001, r = .58) and negative correlations for frontal and parietal (r = -.73, p < .001), and frontal and occipito-parietal (r = -.69, p = .001, r = .48) areas. Medium to strong effect sizes for all corrected significant correlations were observed [73]. In particular, within each experimental condition (RMI, VMI and VE), positive correlations of beta oscillations were observed between bilateral posterior areas, and negative correlations between bilateral anterior and posterior areas.

Figure 3

Neuroanatomical connection network for beta oscillations (12.5 to 30 Hz). Top: The network includes the frontal, parietal cortices and the occipito-temporal and occipito-parietal pathways of the occipital cortex in RMI, VMI and VE experimental conditions. Bottom: Functional connectivity network represents cross-correlations of the regional electrical signal, as estimated from simulated mathematical model dynamics. The colour bar gives the Pierson correlations values. Red colours represent positive correlations, while blue colours represent negative correlations. F: frontal, P: Parietal, OP: Occipito-parietal, OT: Occipito-temporal. In summary, within each experimental condition (RMI, VMI and VE), positive correlations of beta oscillations were observed between bilateral posterior areas, and negative correlations between bilateral anterior and posterior areas.

Figure 4

Neuroanatomical connection network for gamma oscillations (55 to 80 Hz). Top: The network includes the frontal, parietal cortices and the occipito-temporal and occipito-parietal pathways of the occipital cortex in RMI, VMI and VE experimental conditions. Bottom: Functional connectivity network represents cross-correlations of the regional electrical signal, as estimated from simulated mathematical model dynamics. The colour bar gives the Pierson correlations values. Red colours represent positive correlations, while blue colours represent negative correlations. F: frontal, P: Parietal, OP: Occipito-parietal, OT: Occipito-temporal. In summary, when gamma oscillations were analysed positive correlations characterised the bilateral posterior areas in VMI and VE conditions, and negative correlations the bilateral posterior and anterior areas in all experimental conditions (RMI, VMI and VE).

Gamma negative correlations were reported between frontal and occipito-parietal (r = - 0.75, p < .001, r = .56) and between parietal and occipito-temporal (r = -.74, p < .001, r = .55) areas in RMI condition. Similarly, gamma positive correlations were present between parietal and occipito-parietal (r = .76, p < .001, r = .58) and occipito-parietal and occipito-temporal (r = .66, p = .002, r = .44) areas, and negative correlations between frontal and parietal (r = - .75, p < .001, r = .56) and frontal and occipito-parietal (r = -.59, p = .006, r = .35) areas in VMI condition. A positive correlation was revealed between occipito-parietal and occipito-temporal (r = .93, p < .001, r = .86) areas, and negative correlation between frontal and parietal (r = -.82, p < .001, r = .67) areas in VE condition. All corrected significant correlations reported from medium to strong effect sizes [73]. In summary, when gamma oscillations were analysed positive correlations characterised the bilateral posterior areas in VMI and VE conditions, and negative correlations the bilateral posterior and anterior areas in all experimental conditions (RMI, VMI and VE) (Figure 4).

Principal Component Analysis

The dynamic and predominant patterns of brain connectivity during the creativity task were explored via PCA [74]. For each beta and gamma oscillations, the PCA analysis revealed four factors that had eigenvalues greater than one. The visual inspection of each scree plot displayed a clear break after the fourth factor, indicating that only three factors should be retained for all analyses [65, 68, 75]. The three factors corresponded to the experimental conditions: RMI, VMI and VE. To facilitate the interpretability of these three factors, a Varimax rotation was performed, as the correlations between factors produced by the Factor Correlation Matrix (FCM) were smaller than 0.30 [76]. With a cut-off of 0.50 for inclusion of variables in interpretation of factors, all variables loaded in at least one factor. Factor loadings of the rotated solution for the beta and gamma oscillations separately are presented in Figure 5. Overall, large positive and negative loadings were revealed indicating strong effects on each of the PCA factors.

Figure 5

Loadings derives from PCA (Principal Component Analysis) for beta oscillations. The PCA revealed three factors accounting for 63.94% of the total variance and all four areas of interest, i.e. frontal, parietal, occipito-parietal and occipito-temporal were included in the model. There was more beta oscillations neural activation in parietal, occipito-temporal and ocipito-parietal areas, and less activation in frontal areas during the creative task in RMI (Real Mental Imagery), VMI (Virtual Mental Imagery) and VE (Virtual Execution) conditions.

When beta oscillations were analysed (Figure 5), the PCA revealed three factors accounting for 63.94% of the total variance and all four areas of interest, i.e. frontal, parietal, occipito-parietal and occipito-temporal were included in the model. More particularly, large positive loadings were observed for the occipito-parietal (0.947), occipito-temporal (0.753), and parietal (0.695) areas, but negative loadings for frontal areas (−0.865) predicted 10.67% of the brain activity in RMI. Similar connectivity patterns characterised brain activity in the VMI condition (29.21% of the total variance), with large negative loadings for the frontal areas (−0.919) and large positive loadings for the occipito-parietal (0.870), occipito-temporal (0.829) and parietal (0.764) areas. Finally, once again, large negative loadings were revealed for the frontal regions (−0.897), and positive loadings for the occipito-parietal (0.885), parietal (0.755), and occipito-temporal pathways (0.579) in the VE condition (24.06% of the total variance).

With gamma oscillations (Figure 6), the PCA reported three factors which explained 71.35% of the total variance. The patterns of correlated brain activity showed large positive loadings in occipito-temporal (0.830), and occipito-parietal (0.794) areas but large negative loadings in frontal (−0.824) and parietal areas (−0.757) in the RMI condition (17.97% of total variance). Large positive loadings were also shown in parietal (0.905), occipito-parietal (0.895) and occipito-temporal (0.532) areas, and large negative loadings in frontal areas (−0.800) during the VMI condition (22.04% of total variance). Large positive loading was reported for occipito-temporal pathways (0.882), occipito-parietal (0.815) and frontal areas (0.650); negative for parietal (−0.776) in VE condition (31.40% of total variance).

Figure 6

Loadings derives from PCA (Principal Component Analysis) for gamma oscillations. Three factors which explained 71.35% of the total variance were reported with the all four regions of interest included in the model. There was more gamma oscillations neural activation in occipito-temporal and occipito-parietal areas during the creative task in all experimental conditions (RMI-Real Mental Imagery, VMI-Virtual Mental Imagery and VE-Virtual Execution). Similarly, more neural activity in parietal and frontal areas was observed in VMI (Virtual Mental Imagery) and VE (Virtual Execution) condition respectively. However, there was less gamma oscillations in a frontal and parietal areas in RMI (Real Mental Imagery) condition, in frontal areas in VMI (Virtual Mental Imagery) condition and in parietal areas in VE (Virtual Execution) condition.

Discussion

Using mental imagery and execution tasks, the beta and gamma brain oscillations associated with the creation of a scene using familiar (FA) and abstract (AB) physical pictures or virtual objects were analysed. To that end, the interconnections between the bilateral anterior frontal and posterior occipito-temporo-parietal areas in real and augmented environments were scrutinised. As per construction, augmented reality immerses an individual within a hybrid environment, i.e. both real and artificial, and brings this new atmosphere into a person's perception. In the present situation, all participants were immersed in a constant real environment (experimental room) but perceived the same physical or virtual objects depending on the experimental condition. In the Real Mental Imagery (RMI) condition, participants were immersed in a real environment incorporating physical pictures of FA or AB objects and were explicitly instructed to imagine pinching and grasping each object independently to create a scene. They were also unambiguously invited to do the same using the same objects virtually presented when immersed in augmented reality (Virtual Mental Imagery), and to effectively create the scene by pinching and grasping using the virtual FA or AB objects in augmented reality (i.e. Virtual Execution). The participants were instructed to respect specific rules, always the same, in all experimental conditions: not to move their head, not to resize, rotate or break the objects apart, and to use all the objects. Based on scientific evidence according to which the motor system blocks motor commands for muscle contractions during mental imagery tasks [77, 78], the electromyographic (EMG) activity was not recorded in real mental imagery or virtual mental imagery conditions. Instead, all data containing EMG and EOG artefacts were marked and eliminated after visual inspection made by two different experts and via ICA methods. The dynamic neural activity associated with the creation of a scene (mental imagery and execution) was explored via parametric tests. Instead of comparing participants' neural states between the experimental conditions and against a baseline, which is usually the case in all creativity studies, in the present study, functional connectivity between cortical areas was assessed via Pearson correlation coefficient (r Pearson) [65, 79]. Due to the fact that electroencephalographic (EEG) recordings because of their high temporal resolution allow a unique opportunity of linking changes in networks interconnections approaches [80] the cross-correlation between the frontal, parietal, and occipito-parietal and occipito-dorsal regions for beta and gamma oscillations within each experimental condition. The Principal Component Analysis (PCA) was additionally performed to identify the most important neurodynamic data dimensions for each brain oscillations bilaterally [65, 68, 75]. The study was motivated by the idea that creativity, as the act of transforming thinking into reality, transcends representations of the past and present, and is a complex multidimensional process associated with mental simulation, which in essence cannot be explained by brain localisation but might be discerned by neural network connectivity of both the left and right hemispheres [20, 21, 30, 49]. In detail, and when beta and gamma brain oscillations of anterior (frontal) and posterior (parietal, occipito-parietal and occipito-temporal) areas were considered, the results revealed a similar neural signature in the brain between FA and AB objects scenes, in all three experimental conditions. Taking object recognition (associated with FA) and location (associated with AB objects) pathways into account, this suggests the possibility that at neural level the objects (physical or virtual) would be similarly (but not identically) analysed. Interestingly, once participants completed the scenes, they suggested only meaningful titles. In other words, they did not differentiate the scenes based on the characteristics of the objects (FA or AB vs physical or virtual), but mentally manipulated objects in order to create meaningful scenes in both real and augmented environments and, effectively created them in the virtual executive condition. The participants appear to have assembled and integrated information based on cooccurring individual object representations, that is, multi object representations.

The assembling neural mechanism might have reduced inter object competition within and between object category (FA vs AB) regardless of the objects constitution (physical vs virtual). The integrative neural processing seems to be similar between real, virtual imagery and virtual execution conditions, which might signify that mentally simulated actions associated with posterior (parietal, occipito-parietal and occipito-temporal) and anterior (frontal) areas would coherently represent multi object scenes. Multi object scenes might engender neural assemblies between representations through neural synchrony [81, 82]. The above mentioned findings are consistent with existing data revealing that multiple objects are represented as an arrangement not as individual items [83]. However, these findings also extend existing data as it was demonstrated for the first time, to our knowledge, that such multi object representational mechanisms would be associated with imagery in real and augmented environments and execution in augmented reality environments. Such observations might be coherent with the assumption that objects, physical or virtual, would share similar neural construal representations. The findings can also be associated with constant electrophysiological data observed in humans when immersed in real environments and the "simulation hypothesis" according to which mental imagery of actions and real action execution would enlist analogous neural representations [84, 85]. The present data add to the existing findings as it was shown that analogous neural representations might exist during mental imagery in reality, and mental imagery and execution in augmented reality with physical (2D) and virtual (3D) objects. This would signify that the neural assemblies associated with multi object representations might be interconnected in similar ways regardless of the environment and the objects’ characteristics (familiar or abstract) and constitution (physical or virtual).

The findings also showed an increase in beta oscillation synchronisation in occipito-temporo-parietal and parieto-occipito-parietal, and a decrease in synchronisation in fronto-occipito-parietal and fronto-occipito-temporal network at both left and right hemispheres during the task. This neural configuration was quite similar in real mental imagery, virtual mental imagery experimental conditions. However, beta oscillation synchronisation increased only in occipito-temporo-parietal network and decreased in fronto-occipito-parietal and fronto-parietal networks in virtual execution condition. As such, regardless of the objects constitution (physical or virtual) a common minimal connectivity pattern was observed in real and virtual experimental conditions following which at beta oscillation level and bilaterally, the coactivation of posterior (parietal, temporal and occipital) areas was synchronised. Conversely, the coactivation of anterior bilateral frontal and posterior areas was desynchronised. By contrasting on connectivity, the findings suggest that there is a mutual opposing adjustment between posterior and anterior brain areas in both real and augmented environments for mental imagery and execution tasks. Such neural configuration might indicate simultaneously spontaneous (posterior network) and contained (antero-posterior network) mental imagery during the creativity task. This configuration was confirmed by the Principal Component Analysis (PCA). Specifically, the results showed more activation in all posterior areas (parietal, occipito-parietal and occipito-temporal) and less activation within bilateral anterior frontal areas in all three experimental conditions. Interestingly, the virtual mental imagery accounted for the most of the data variance. Bearing in mind that beta oscillations are associated with active thinking, and coherent cognitive interactions with the external world and also that the frontal areas are involved in enabling voluntary behaviours according to the environment, developing mental representations of external and internal worlds by organising the coordination of spatial and sensory areas; and finally considering that the occipito-temporal and occipito-parietal areas are implicated in properties and spatial location and transformation of objects respectively [3, 86, 87], the present findings suggest that creativity as a mental imagery and execution engagement might implicate a delicate balance between anterior frontal areas and posterior sensory networks at both left and right hemispheres (i.e. bilaterally). The decrease of beta synchronisation in bilateral frontal areas in all three conditions might signify weakening of the functional connections of the frontal regions which would facilitate a better analysis for performing adequate associations. Such findings are consistent with recent models describing a posterior-anterior gradient in the abstract expression of the mental representations that can be formed [22, 51, 88, 89]. More anterior region activity supports more abstract thinking; less anterior regions activity implicates less abstract thinking. Coherent with this is the observation that only meaningful titles (e.g "A peaceful place", "Space", "Person", "Christmas tree", "Flower", "The animal" etc) were given to the scenes regardless of the objects’ characteristics (FA or AB) and constitution (physical or virtual); that is, participants abstract thinking was minimised or quasi-absent in all experimental conditions.

When analysing the participants' neural activity, an increase in synchronisation was shown in bilateral occipito-temporal and occipito-parietal gamma oscillation networks in VMI and VE experimental conditions, and in bilateral parieto-occipito-parietal network in VMI condition but a decrease in synchronisation was observed in the bilateral fronto-occipito-parietal network in real and virtual imagery conditions. Similarly, a decrease in synchronisation in bilateral fronto-parietal network in virtual mental imagery and virtual execution conditions was revealed. In other words, in VMI and VE experimental conditions, bilateral gamma sensory networks (i.e. posterior networks) involved in identification, location and visuomotor objects coordination were synchronised. Such neural network engagement is consistent with constant data reported that when the visual cortex (visuo-parietal and visuo-temporal) is stimulated by appropriate information, objects for instance, gamma activity is typically engaged [90] and gamma cortical signature increases [91] in relationship to the sensory incitement including perceptual grouping [26]. This also suggests that the creative thinking associated with the virtual objects might be first analysed in bilateral posterior brain areas when participants had to imagine creating a scene with the objects in augmented reality, and when they have to effectively execute the scene in augmented reality. However, there was no statistically significant increase in synchronisation at posterior network in RMI condition. Note that in the current situation, the difference between the real and augmented environments was, essentially, materialised by the objects’ constitution. The familiar and abstract objects were image entities in a real environment but virtual (disembodied) in augmented reality. Note also that in the augmented reality atmosphere, objects, by construction, are virtual and incorporated into a real environment [3, 15, 16]. This suggest that the bilateral gamma posterior network would be more involved in imagery and execution tasks involving virtual objects, that is, better synchronised for tasks associated with virtual objects.

Importantly, a decrease in synchronisation was induced in bilateral gamma frontal-occipito-parietal network when participants imagined creating a scene in real (with physical objects) and augmented reality (with virtual objects) environments. In addition, a decrease in synchronisation was also shown in fronto-parietal gamma oscillations when participants imagined create a scene with virtual objects, and when they effectively created a scene (with virtual objects) in augmented reality. Gamma PCA outcome reported more activation in both occipito-parietal and occipito-temporal networks bilaterally in all experimental conditions, less bilateral frontal activation in real and virtual imagery conditions and less bilateral parietal activity in real imagery and virtual executive conditions. PCA results also showed bilateral frontal areas enhancement in the virtual execution condition only. This latter condition explained the most of the variance. One can expect that such modifications on the gamma oscillations’ sensitivity would be understood as their activity has been found to increase with the stimulation size [92]. However, such speculation cannot account for the present study as all features of the physical and virtual objects, including their size, were identical. Nor can the findings be understood by any memory intervention, as memory effects were neutralised by the presence of the objects in each experimental condition. Nonetheless, the findings might be associated with the intrinsic properties of gamma activity for object representations. Gamma activity is considered to be relevant for high-level cognitive processes [93] and increased during complex representational tasks. Given the current results, it seems that imagine create and execute a scene with virtual objects necessitate a more complex representational state that imagine create a scene via real objects. Gamma activity is also considered to reflect a binding mechanism of sensorimotor integration and increase in relationship with perceptual (bottom-up) analysis of external stimulations. The strong enhancement of gamma oscillations in bilateral occipito-parietal and occipito-temporal networks observed in the present study is consistent with data according to which gamma activity is significantly involved in posterior areas of the visual cortex when coherent percepts are concerned [94]. However, our data also demonstrated that gamma activity was present (i.e. desynchronised and synchronised) in bilateral frontal and parietal areas in real and augmented environments when participants were instructed to imagine creating or effectively creating a scene using physical or virtual objects. Gamma oscillations are significantly involved in bottom-up and top-down matching [95] and provide a unique signature of task related brain activity and creativity. Visuomotor and sensorimotor coordinations associated with frontal and parietal areas seem to contribute differentially to the creative task. Both frontal and parietal areas were less activated during the mental imagery task in real environment, frontal areas during the mental imagery in augmented reality and parietal areas during the creative execution in the augmented reality environment. Given that frontal and parietal areas are associated with motor planning and regulation, a fronto-parietal oversight of imagery creativity in the real environment might emerge suggesting top-down gamma oscillations originating from occipito-parietal and occipito-temporal cortices to disrupt sensory activity processing from irrelevant information. Similarly, an increase in occipito-temporal and occipito-parietal gamma oscillations might result in a decrease in frontal gamma oscillations during real and virtual creative imagery and parietal gamma oscillations during virtual execution. On the other hand, frontal gamma oscillations were increased in synchronisation during the virtual execution task likely because of the direct involvement of frontal areas in motor actions, that is, the direct use of the hands to picking and grasping the virtual objects. Hence, the current results account solely for the cortico-cortical interactions in creative mental imagery in reality and augmented reality with physical and virtual objects (familiar or abstract) and execution in augmented reality. Given the involvement of gamma oscillations in high level cognitive processes and objects representations [36], we suggest that the neurotopography of bilateral anterior and posterior gamma oscillations might reflect the internal representations of physical objects on the one hand but likewise virtual (disembodied) objects, on the other hand, when are assembled together in the mind to create a meaningful scene.

As far as the significant level for the statistical analysis was fixed at .10 to minimise both type errors I and II, the beta and gamma oscillations aforementioned findings cannot be explained neither by failing of rejecting a true null hypothesis nor by failing or rejecting a false null hypothesis [71]. The sample of the study is limited but very similar to recent published studies on creativity as for instance, Jia and Zeng (2021) (29 participants with 4 females and 25 males), Zhou et al (2018) (23 healthy adults), and Beaty et al. (2015) (25 young adults). Likewise, the findings cannot be due to the statistical approach (t-test comparisons) given that both t-test and ANOVA test gave analogous results. Finally, the data cannot be explained by an order effect given that the objects were the same in the three experimental conditions and this because not only they differ in terms of constitution (physical vs virtual) but that their order of presentation was counterbalanced across the subjects, the conditions and within they own category (FA vs AB).

Since the precise neural architecture responsible for creativity is not fully acknowledged, the creative mental imagery process described in the present paper might be associated with the Wallas' five stage model of creativity [4, 11]. Expressed on a continuum going from preparation to intimation and from intimation to verification, Wallas' model, also illustrated in terms of proximity of consciousness, might be endorsed by the interconnected bilateral cortico-cortical neural network described in the present paper. Preparation and incubation that lead to intimation, i.e. "rising train of association" might be associated with bottom-up processes and more particularly with the beta and gamma occipito-temporal and occipito-parietal oscillation networks. Illumination, i.e. culmination of a "train of association" that expresses the creative thinking and verification (production) that allows for the revealing of intimation, as involved in action and decision making processes, might be connected with the bilateral beta and gamma frontal neural oscillations networks. This beta, but specifically gamma fronto-parieto-temporo-occipital cortico-cortical network appears to delineate the conscious interpenetration of the Wallas’ model in mental imagery and execution of creative tasks rendered possible via physical and virtual (disembodied) familiar (FA) or abstract (AB) objects in real but also in augmented reality environments. Future studies should examine beta and gamma interactions during mental imagery via real and virtual object manipulation in reality and virtuality (virtual and augmented reality) which might provide further insights into the role of posterior and anterior networks in mental creative imagery and creative thinking. In addition, new studies could also provide significant information with regards to sub-cortical feedback and cortico-subcortical interconnections in mental imagery creative thinking.

Even though the total amount of data was significantly high as each participant performed three experiment conditions of 2.30 min each where the electrical activity was recorded via a 32 EEG of 1000 Hz, limitations might be associated in particular with the experimental device and data analysis procedures, and occasionally, the data may include irregular features for both beta and gamma oscillations even though all precautions were taken to eliminate them. We do acknowledge that functional connectivity per se is purely correlative and can be assessed via different approaches [81]. However, given the high temporal resolution of EEG, functional connectivity analyses for beta and gamma oscillations do indicate neural mass activity, i.e. synchronisation, between posterior and anterior areas (note that gamma oscillations are usually recorded with more than 62 electrodes and the device used in the study has 32 electrodes), and that gamma oscillations are of high transmission rate and spatial precision [96], future studies might opt for a multimodal neuroimaging based on the combination of 62 EEG electrodes and fNIRS neurotechniques, for instance, to improve data quality recording.

Notwithstanding, it can be concluded that in a general sample of healthy adults and for both beta and gamma oscillations, the present findings suggest the existence of a bilateral cortico-cortical distributed network assisting synchronisation (or desynchronisation) during a creativity ideation task in real and augmented environments. Using familiar and abstract physical or virtual (disembodied) objects in real and augmented environments to imagine creating a scene, and augmented reality to effectively creating a scene, the findings seem to display an analogous neural architecture essentially represented by the bilateral fronto-parieto-temporo-occipital neural network. These findings are consistent with the hypothesis according to which creativity, because of its inherent complexity, cannot be confined to a single brain area but would engage various interconnected areas. The results are also consistent with the assumption that in essence, creativity is a mental imagery simulation process.

Declarations

Author contribution statement

Giannopulu, I: Conceived and designed the experiment; Analyzed and interpreted the data; Wrote the paper.

Brotto, G: Analyzed and interpreted the data.

Lee, T.J: Performed the experiments; Contributed reagents, materials, analysis tools or data.

Frangos, A, To, D: Performed the experiments.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

Datasets used for the current study may be made available on reasonable request.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.