Distributed affective space represents multiple emotion categories across the human brain.

The functional organization of human emotion systems as well as their neuroanatomical basis and segregation in the brain remains unresolved. Here, we used pattern classification and hierarchical clustering to characterize the organization of a wide array of emotion categories in the human brain. We induced 14 emotions (6 'basic', e.g. fear and anger; and 8 'non-basic', e.g. shame and gratitude) and a neutral state using guided mental imagery while participants' brain activity was measured with functional magnetic resonance imaging (fMRI). Twelve out of 14 emotions could be reliably classified from the haemodynamic signals. All emotions engaged a multitude of brain areas, primarily in midline cortices including anterior and posterior cingulate gyri and precuneus, in subcortical regions, and in motor regions including cerebellum and premotor cortex. Similarity of subjective emotional experiences was associated with similarity of the corresponding neural activation patterns. We conclude that different basic and non-basic emotions have distinguishable neural bases characterized by specific, distributed activation patterns in widespread cortical and subcortical circuits. Regionally differentiated engagement of these circuits defines the unique neural activity pattern and the corresponding subjective feeling associated with each emotion.

Introduction

The organization of human emotion systems is currently a topic of lively debate (Hamann, 2012; Lindquist ; Kragel and LaBar, 2014; Meaux and Vuilleumier, 2015; Saarimäki ; Adolphs, 2017; Barrett, 2017). This discussion revolves around the number of distinct emotion systems and the organization of human emotion circuits in the brain. Most research on specific emotion categories has focused on ‘primary’ or ‘basic’ emotions (usually anger, fear, disgust, happiness, sadness and surprise). According to the basic emotion theories, emotions have been shaped during the evolution to serve distinct survival functions via distinct neural circuits and physiological systems (Panksepp, 1982; Ekman, 1992, 1999; Damasio, 1999). In turn, constructivist theories consider specific emotions emerging from the interaction of fundamental processes shared across psychological domains and, thus, not unique to emotions alone (see, e.g. Russell, 2003; Lewis and Liu, 2011; Hamann, 2012; Cunningham, 2013). Basic emotion theories emphasize similarities in emotion mechanisms across individuals, while constructivist theories emphasize inter- and within-individual variability. Several human neuroimaging studies support the view that at least the canonical basic emotions have distinguishable neural bases, as they are associated with discernible neural activity patterns as measured by BOLD-functional magnetic resonance imaging (fMRI) (e.g. Kragel and LaBar 2014; Saarimäki ; for a review, see Kragel and LaBar, 2016; Nummenmaa & Saarimäki, 2017).

Yet, a wide array of other emotions, including ‘secondary’ or ‘social’ emotions (see reviews and proposed taxonomies in Damasio, 1999; Adolphs, 2002a), also serve adaptive survival functions and are characterized by distinctive facial expressions (Baron-Cohen ; Shaw ), bodily sensations (Nummenmaa ), and neural activity patterns (Kassam ; Kragel and LaBar, 2015). Nevertheless, the psychological and neural mechanisms of these non-basic emotions, as well as their commonalities or differences relative to basic emotions, remain largely unresolved (Ekman, 1999; Ekman and Cordaro, 2011; Adolphs, 2002b). These emotions may involve more elaborate cognitive representations acquired through experience, education, and social norms (Panksepp and Watt, 2011), and hence, recruit brain systems partly distinct from those implicated in more ‘primitive’ and possibly innate basic emotions. It is thus possible that also non-basic emotions may have distinguishable neural bases which would, however, be discernible from that of basic emotions. Accordingly, the specific number of basic emotions has been debated and the distinction between basic and non-basic emotions has been questioned (e.g. Russell, 2003). For example, Adolphs (2002b) proposed that affective processes could be understood at various taxonomies depending on the level of analysis. The basic emotion conceptualization would be particularly relevant for primary-process core affects governed by phylogenetically old emotion circuits especially within the subcortical structures (Panksepp and Watt, 2011).

A set of core emotion processing regions is consistently engaged during multiple emotions. These include cortical midline regions (Peelen ; Chikazoe ; Trost ), somatomotor regions (Adolphs ; de Gelder ; Nummenmaa , 2012; Pichon ), as well as amygdala, brainstem and thalamus (Adolphs, 2010; Damasio and Carvalho, 2013; Kragel and LaBar, 2014). These regions serve as candidate areas containing distinct neural signatures for different basic emotions (Peelen ; Saarimäki ). Yet, it is currently unclear whether these regions also code for other, non-basic emotions. Prior studies using univariate analyses have quantified neural responses to emotions such as regret (Coricelli ; Eryilmaz ), guilt (Wagner ), pride (Takahashi ; Zahn ; Simon-Thomas ), rejoice (Chandrasekhar ) and maternal love (Bartels and Zeki, 2004), as well as aesthetic feelings such as wonder or nostalgia (Vuilleumier and Trost, 2015). Yet, these studies (except Kassam ; Kragel and LaBar 2014) have usually compared brain activation differences between two emotions in a univariate fashion. Thus, it remains unclear whether similar, distinct circuits as previously observed for basic emotions would also support these types of emotions.

Here, we investigated the neural underpinnings of multiple basic and non-basic emotion categories. We induced 14 emotions in participants by guided mental imagery while their brain activity was measured with fMRI. First, to examine whether different emotions have distinguishable brain bases, we employed pattern classification. Second, using hierarchical clustering, we examined the similarity of neural substrates of different emotions and tested how this similarity was related to how similarly these emotions are experienced. Third, we mapped representation of different emotions in the core emotion processing regions of the brain, by characterizing the emotion-dependent neural response profiles using univariate analyses and cumulative activation mapping.

Materials and methods

Participants

Twenty-five female volunteers (aged 19–38, mean age 23.6 years) participated in the experiment. All were right-handed, neurologically healthy and with normal or corrected-to-normal vision, and gave written informed consent according to the Declaration of Helsinki. The Institutional Review Board of Aalto University approved the experimental protocol. Female participants were chosen to maximize the power of the experiment, as when compared to males, females typically experience and portray more intensive emotional reactions (see review in Fischer and LaFrance, 2015), and show greater brain responses during emotions (e.g., Hofer ) and stronger facial mimicry as indexed by EMG (Grossman and Wood, 1993).

Stimuli

The stimuli for the guided affective imagery were sixty 5–20-s long narratives describing an antecedent event triggering prominently one emotional state. Narrative-based guided imagery is known to be an efficient emotion induction technique that engages reliably the brain's emotion circuits (Costa ; Nummenmaa ) and results in strong subjective emotional feelings. Each narrative elicited primarily one out of possible 14 emotions, or a neutral emotional state. Targeted emotions included six basic or primary emotions (anger, fear, disgust, happiness, sadness and surprise) and eight social or secondary emotions (shame, pride, longing, guilt, love, contempt, gratitude and despair). The narratives included a short description of a situation that participants were instructed to imagine would happen to them, for instance, ‘It is a late night on a dimly-lit parking lot. Your car stands alone in a dark corner. Suddenly you hear a gun shot behind you.’ (fear), or ‘Your lover is lying next to you on a bed. You look into his eyes when he gently touches your hair and bends to kiss your lips.’ (love). Based on an online pilot experiment (see Supplementary Material), we selected four narratives per category (total of 60 narratives; see Supplementary Table S1 for all narrative stimuli) to be included in the fMRI experiment.

The selected narratives were spoken by a female speaker using neutral prosody without cues for the affective content of the narrative. The background noise in the recording room was recorded and equalized (bandpass filter 50–10 000 Hz) with Apple Logic Pro 9 (Apple Inc.), and gate and compressor were used to attenuate the background noise during moments of silence and slightly compress the voice dynamic to limit the variance of the sound power. The loudness of each narrative was normalized according to the peak value.

The recorded narratives were divided into 4 runs of 15 narratives, with one narrative per category in each run. The runs contained the same narratives for all participants but the presentation order was randomized within each run and for each participant. During fMRI, the four sets of recorded narratives were all presented twice thus resulting in altogether eight runs. Each run lasted 7–8 min and consisted of 15 trials. A trial started with a fixation cross shown for 0.5 s, followed by a 2-s presentation of a word describing the target emotion (anger, fear, disgust and so forth) to prepare participants for the forthcoming emotional imagery task and thus to make the induction more powerful. Next, the narrative was spoken out, followed by a 10-s imagery period. The trial ended with a 10-s wash-out period to counter for possible carryover effects. Participants were instructed to try to get involved in the narratives by imagining the described events as happening to themselves and to experience the corresponding feeling as vividly as possible.

Auditory stimuli were delivered through Sensimetrics S14 insert earphones (Sensimetrics Corporation, Malden, MA, USA). Sound was adjusted for each participant to be loud enough to be heard over the scanner noise. The visual stimuli were delivered using Presentation software (Neurobehavioral Systems Inc., Albany, CA, USA), and they were back-projected on a semitransparent screen using a 3-micromirror data projector (Christie X3, Christie Digital Systems Ltd., Mönchengladbach, Germany), and from there via a mirror to the participant.

After the scanning, participants were presented with the narratives again. For each narrative, the participants rated the felt intensity of each of the 14 emotions (plus neutral state) using a continuos scale arbitrarily ranging from 0 to 1. (for details, see Supplementary Material). We ran k-means clustering on the resulting intensity profiles (i.e. intensity ratings per category for each narrative) to test whether the 15 target categories could be identified from the emotion-wise intensity ratings, thus revealing whether or not the narratives elicited distinct categorical emotions. Also, to test the similarity of narratives belonging to the same category, we calculated the Euclidean distance between the intensity profiles of each pair of narratives.

Finally, participants rated the similarity of all possible pairs of the emotions induced during the experiment (a total of 105 pairs) using a direct comparison method. The participants were shown one pair of emotion words at the time and asked to rate how subjectively similar they experience the emotions (ranging from no similarity [0] to full similarity [1]). Based on the ratings, we extracted the recollected experiential similarity matrix for each participant individually and averaged these across all participants.

MRI data acquisition and preprocessing

MRI data were collected on a 3-T Siemens Magnetom Skyra scanner at the Advanced Magnetic Imaging center, Aalto University, using a 20-channel Siemens volume coil. Whole-brain functional scans were collected using a whole brain T2*-weighted EPI sequence with the following parameters: 33 axial slices, TR = 1.7 s, TE = 24 ms, flip angle = 70°, voxel size = 3.1×3.1×4.0 mm³, matrix size = 64×64×33, FOV = 256×256 mm. A custom-modified bipolar water excitation radio frequency (RF) pulse was used to avoid signal from fat. High-resolution anatomical images with isotropic 1×1×1 mm³ voxel size were collected using a T1-weighted MP-RAGE sequence.

Data were preprocessed using FSL 5.0 (Jenkinson ). Motion correction was performed using MCFLIRT (Jenkinson ) and non-brain matter was removed using BET (Smith, 2002). High-pass temporal filtering was applied using Gaussian-weighted least-squares straight line fitting with sigma of 100 s. Participant-wise gray matter masks were generated by segmenting the T1-weighted images into gray and white matter and cerebrospinal fluid using the FAST segmentation tool (Zhang ) and transforming the masks to the native space using FLIRT (Jenkinson ) with nine degrees of freedom. The gray matter maps were subsequently thresholded using intensity threshold > 0.5 to create participant-specific masks. This threshold was chosen to include those voxels with a higher probability of belonging to the gray matter and, subsequently, the masks were visually inspected to make sure most gray matter was included. On average, the gray matter mask included 16 000 voxels.

For univariate general linear model (GLM) analysis, the preprocessed functional data were registered to 2-mm Montreal Neurological Institute (MNI) 152 standard space template using FLIRT (Jenkinson ) and 9 degrees of freedom, and smoothed using a Gaussian kernel with FMWH 8.

Multivariate pattern classification within participants

The classification of emotion categories was performed using the Princeton multi-voxel pattern analysis (MVPA) toolbox (https://pni.princeton.edu/pni-software-tools/mvpa-toolbox) in Matlab 2012b using each participant’s data in native space. A separate classifier was trained for each participant and, after all steps, the classification results were averaged across the participants. After preprocessing, voxels outside gray matter were masked out using the participant-specific gray matter masks and the functional data from each run were standardized to have a mean of zero and variance of one. Next, each participant’s data were divided into training (N−1 runs) and testing sets (the remaining run). Feature selection was performed using one-way ANOVA (testing for the main effect of emotion category) for the training set to select the voxels with a significant (P < 0.05) main effect for emotion, i.e. to select the voxels whose mean activation differed between at least some of the 15 possible emotion conditions. The feature selection preserved on average (across cross-validation folds and participants) 41% of the voxels. Hemodynamic lag was corrected for by convolving the boxcar category regressors with the canonical double gamma HRF function and thresholding the convolved regressors using a sigmoid function to return the regressors to the binary form. The classification was performed on all the standardized, HRF-convolved fMRI volumes from the 10 s imagery period following the narrative (treating all single time points per category as samples for that category; median 6 volumes per one emotion category in one run) to extract only emotion-related brain activity, and to minimize activity related to the acoustic and semantic features of the stimuli. Thus, each of the eight runs included on average 90 TRs (5–6 TRs per category) that were used in the classification. A linear neural network classifier without hidden layers was trained to recognize the correct emotion category out of 15 possible ones (multiclass classification, see Polyn for details). Naïve chance level, derived as a ratio of 1 over the number of categories, was 6.7%. A leave-one-run-out cross-validation was performed, thus, dividing the data into all possible N−1 run combinations and repeating the classification pipeline for each such cross-validation fold, and the participant-wise classification accuracy was calculated as an average percentage of correct guesses across all the cross-validation folds. To test whether classification accuracy exceeded chance level, we used permutation tests to simulate the probability distribution of the classification by shuffling the category labels of the training set (across training set runs) and re-running the whole classification pipeline, repeated 1000 times for each participant. FDR correction at P < 0.05 was used for multiple comparisons.

Hierarchical clustering

We next investigated the similarities between emotions using the category confusions from the whole-brain classification. Clustering was performed for exploratory purposes to characterize the similarities in neural and recollected experiential data, this however does not provide statistical evidence for the similarity structures. Nevertheless, similarity between neural and experiential data would suggest similar representational segregation in the neural code and subjective feelings. From the group-averaged confusion matrix, we calculated a distance matrix by taking the category confusion vectors for each pair of emotions and by calculating the Euclidean distance between these vectors (see Reyes-Vargas ). We then employed hierarchical cluster analysis in Matlab to investigate how different emotions cluster together based on their neural similarities. The agglomerative hierarchical cluster tree was calculated on the distance matrix using linkage function with ‘complete’ option (i.e. the furthest distance method). Finally, we constructed the clusters from the cluster tree (cluster function) and chose the solution that minimized the number of categories while keeping at least two emotions per category. Note that the clustering was selected solely for data visualization rather than for statistical inference. To visualize the similarities in subjective and neural organization of emotions, we extracted the clusters in both neural and recollected experiential data, and subsequently plotted the cluster solutions using alluvial diagrams (Rosvall and Bergstrom, 2010; www.mapequation.org).

We then investigated to which extent the neural similarities between different emotional states correspond to their recollected experiential (subjectively felt) differences. Experiential similarity matrices were calculated based on pairwise similarity ratings of emotions and averaged over the participants. Subsequently, the mean neural and experiential similarity matrices were correlated using Spearman’s rank correlation coefficient. The P level for the Spearman test was obtained with a permutation test by shuffling the neural matrix and re-calculating the correlation for 5000 times.

Finally, we tested whether basic and non-basic emotions generally differ in how categorical their experience is recognized. To do so, we calculated prototypical experience scores for each emotion, defined as the sum of off-diagonal elements in the experiential and neural similarity matrices separately. This analysis was based on the assumption that the off-diagonal elements represent confusions across emotions, thus indicating a lack of sharp categorical representation within a group of emotions. In other words, we tested whether emotion confusions were systematically different for basic and non-basic types, in both subjective and neural data. The resulting ‘prototypicality’ scores (i.e. off-diagonal scores) for basic and non-basic emotions were then compared using Mann–Whitney U test.

Regional effects in GLM

To investigate the overall effect of any emotion on brain activity, we first ran GLM to compare all emotions together against the neutral baseline, and then ran separate GLMs to compare each of the 14 emotions against the neutral baseline. First level analysis was performed in SPM 12 (wwwl.fil.ion.ucl.ac.uk/spm/) to obtain individual contrast maps and second level analysis was then performed with FSL randomize with the threshold free cluster enhancement option as it implements the least biased way to control for multiple comparisons (Eklund ; N = 5000 permutations). Emotion-wise t maps were then qualitatively summarized across emotions as a cumulative map where each voxel shows the number of statistically significant emotions, at the cluster corrected level of P < 0.05.

Visualization of emotion clusters in the brain

Finally, to summarize and visualize where emotion-wise activation patterns were located, we mapped the three principal clusters obtained with hierarchical clustering on the cortical and subcortical maps using R, G, B color space. The last cluster containing surprise and neutral was plotted separately (see Supplementary Figure S6) given it contained neutral-like states only. For each emotion, we took the unthresholded second level t maps obtained from the GLM analysis, summed them for emotions belonging to the same cluster, and assigned the summed values to the corresponding R, G, B channels. The color channels were subsequently visualized in MNI space. Consequently, the RGB color at each voxel reflects the cluster distribution of that voxel, and can be used for localizing brain regions contributing to different emotions.

Results

Behavioral results

Behavioral ratings showed that the narratives successfully elicited reliable and strong target emotions (intensity profiles in Figure 1A; mean intensities per target category: pride 0.80, longing for 0.77, happiness 0.84, gratitude 0.84, love 0.90, surprise 0.76, neutral 0.88, disgust 0.85, sadness 0.96, fear 0.89, shame 0.75, anger 0.68, guilt 0.91, contempt 0.66, despair 0.87). In k-means clustering, the accuracy to assign a narrative to the correct target category based on its intensity profile was 97% (against the chance level 6.7%). Also, the narratives within each category had highly similar intensity profiles (Figure 1B); i.e. narratives belonging to the same category elicited similar emotions.

Fig. 1.

The stimuli consisted of 60 brief (5–20 s) narratives that induced 14 emotional states and a neutral state. (A) Participants rated on a scale from 0 to 1 how strongly each emotion was elicited by the narrative. The coloring indexes the mean intensity for experiencing each emotion for each narrative. (B) Based on the emotion intensity ratings, we calculated the similarity of emotional experience between narratives by using Euclidean distances.

Classification of basic and non-basic emotions

Mean classification accuracy across the 14 emotions and the neutral state was 17% (against naïve chance level of 6.7%; 95th percentile of the permuted classification accuracy distribution was 8.4%). After correcting for multiple comparisons, the classification performance was above a permutation-based significance level for all emotions except shame and longing (P < 0.05, Figure 2; see Supplementary Table S2 for effect sizes and Supplementary Figure S1 for a confusion matrix). On average, basic emotions (anger, disgust, fear, happiness, sadness, surprise) could be classified more accurately than the non-basic emotions (26% vs 15%, respectively, t(24) = 7.39, P < 0.0001, Cohen’s h = 0.16), while we found no significant differences in classifier accuracies between positive and negative emotions (17% vs 14%, respectively, t(24) = 1.52, P = 0.075). We also trained separate classifiers for each a priori selected region-of-interest (see Supplementary Methods) which showed that classification accuracies were above chance level in frontal ROIs, especially in frontal pole, and in somatomotor ROIs, especially for pre- and post-central gyri, yet did not exceed that of the whole-brain classification (Supplementary Figure S2).

Fig. 2.

Means and standard errors for emotion-wise whole-brain classification accuracy. Dashed line represents chance level (6.7%). Colors reflect the clusters formed on the basis of experienced similarity of emotions (see Figure 3).

Fig. 3.

(A) Left: Neural similarity matrix extracted from the classifier confusion matrix. The similarity matrix was created by calculating the Euclidean distance between each pair of emotions based on their category confusion vectors. Right: Experiential similarity matrix based on pairwise similarity ratings for emotions elicited by the narratives. (B) Alluvial diagram showing the similarity of hierarchical cluster structure of the experiential and neural similarities. Coloring of the emotion categories is based on the clusters in the neural similarity matrix.

We also ran separate classifiers across positively and negatively valenced emotion categories to test whether classification within a similarly valenced superordinate category is comparable to that of classification with all categories. Average classifier accuracy for positive emotions was 30.5% (naïve chance level 20%) and all emotions could be classified with above-chance level accuracy: love (36.9%), happiness (31.5%), gratitude (30.8%), pride (28.5%), and longing (24.9%). Average classifier accuracy for negative emotions was 22.4% (naïve chance level 12.5%) and above chance level accuracy was observed for each emotion: sadness (28.8%), disgust (27.8%), despair (25.7%), fear (25.1%), anger (20.5%), contempt (19.0%), and shame (15.9%).

Similarity in neural basis corresponds to experienced similarity

Organization of recollected experiential similarity matrices derived from behavioral ratings was significantly associated with the neural similarity matrices derived from confusion matrices from whole-brain classification (r = 0.37, P = 0.0048; Figure 3A). Clustering of confusion matrices divided the emotions into four clusters (Figure 3A): (1) happiness, pride, gratitude, love, and longing; (2) surprise and neutral; (3) disgust, sadness, fear, and shame; and (4) anger, contempt, guilt, and despair, which mostly corresponded to the four main clusters of experiential similarities (Figure 3B): (1) happiness, pride, gratitude, and love; (2) surprise and neutral; (3) longing and sadness; and (4) disgust, anger, contempt, shame, guilt, fear, and despair.

To test whether basic and non-basic emotions generally differ in how prototypical their experience is, we calculated a prototypical experience score for each emotion (i.e. rate of confusions with other emotions) and then compared the average prototypicality scores for basic and non-basic emotions. There were no differences between basic and non-basic in either the neural or experiential data.

Affective space in the brain

To investigate the brain regions generally activated and deactivated by our emotion stimuli, we contrasted the brain activity related to all emotions with the neutral condition (Figure 4 outlines; see also Supplementary Figure S4). The areas activated by emotion in general included pre-motor cortex, thalamus, insula and putamen. Deactivations were observed in visual and auditory cortices, precuneus, PCC, right anterior PFC and right lateral parietal areas.

Fig. 4.

(A) Cumulative activation map showing the cumulative sum of binarized t maps (P < 0.05, cluster-corrected) across each emotion vs neutral condition. Outline shows the GLM results for all emotions contrasted against the neutral condition (P < 0.05, cluster-corrected). (B) Cumulative deactivation map showing the cumulative sum of binarized t maps (P < 0.05, cluster-corrected) across neutral vs each emotion. Outline shows the GLM results for the neutral condition contrasted against all emotions (P < 0.05, cluster-corrected).

To reveal the brain regions contributing most consistently to different emotions, we constructed cumulative activation and deactivation maps of emotion-driven hemodynamic responses (Figure 4; corresponding effect size maps in Supplementary Figure S4; the emotion-wise statistical t maps are available in http://neurovault.org/collections/TWZSVODU). These maps reveal how many of the 14 possible emotional states activated each voxel. The cumulative activation map (Figure 4A) showed that most emotions involved activation of midline regions including anterior cingulate cortex (ACC) and precuneus, as well as subcortical regions including brain stem and hippocampus, motor areas including cerebellum, and visual cortex. The cumulative deactivation map (Figure 4B) showed that most emotions involved deactivation of auditory cortex, frontal areas including superior frontal gyri, right middle frontal gyrus, and left inferior frontal gyrus and parietal areas including supramarginal gyrus.

Finally, to visualize where specific emotions are encoded in the brain, we mapped the clusters resulting from the hierarchical clustering on cortical and subcortical surfaces (Figure 5). All emotions activated areas in the visual cortex, ACC, right temporal pole, supplementary motor area and subcortical regions. In addition, positive emotions belonging to Cluster 1 (happiness, pride, gratitude, love, longing) were more prominent in anterior frontal areas including vmPFC. Negative basic emotions from Cluster 2 (disgust, sadness, fear, shame) activated especially insula, supplementary motor area and specific parts of subcortical structures. Negative social emotions belonging to Cluster 3 (anger, contempt, guilt, despair; see Supplementary Table S1 for the story stimuli targeting different emotions) were most prominent in left insula and the adjacent frontal areas. Finally, surprise (Cluster 4, Supplementary Figure S6) activated especially parts of auditory cortex, supplementary motor areas, and left insula.

Fig. 5.

Activation maps showing the summed uncorrected t maps for each cluster obtained from the hierarchical clustering analysis in (A) cortical regions and (B) subcortical regions. Colors represent the three clusters: positive (red), negative basic (green), and negative social (blue) emotions.

Discussion

Our results reveal that multiple emotion states have distinct and distributed neural bases, as evidenced by the above chance-level classifier performance for all emotions, except longing and shame. Together with the spatial location of emotion-dependent brain activation, this suggests that a multitude of different emotions are represented in the brain in a distinguishable manner, yet in partly overlapping regions: each emotion state likely modulates different functional systems of the brain differently, as shown by distinct patterns measured with BOLD–fMRI, and the overall configuration of the regional activation patterns defines the resulting emotion. While a set of ‘core’ emotion processing areas in cortical midline regions, motor areas, sensory areas, and subcortical regions are engaged during practically all emotions, the relative engagement of these areas varies between emotions. This unique neural signature of each emotion might relate to the corresponding subjective feeling, as evidenced by the correspondence between neural and recollected experiential similarity between emotions.

Different emotions are characterized by distinct neural signatures

Altogether 12 emotions (excluding longing for and shame) out of the 14 included in the study could be reliably classified from the fMRI signals. Our results extend previous studies, which have shown classification of specific emotional states usually focusing on the basic emotions only or a subset of these (Ethofer ; Peelen ; Said ; Kotz ; for a review, see Kragel and LaBar, 2014). While the ‘classic’ basic emotions have attracted most attention in psychological and neurophysiological studies, they constitute only a small portion of the emotions humans universally experience (Edelstein and Shaver, 2007). Furthermore, accumulating behavioral evidence suggests that other, non-basic emotions are also characterized by distinctive features in facial expressions (Baron-Cohen ; Shaw ), bodily changes (Nummenmaa ), and physiology (Kreibig, 2010; Kragel and LaBar, 2013). The present data corroborate these findings by showing that also emotions not considered as ‘basic’ may each have distinctive brain activation patterns. The local brain activity patterns underlying different emotions are most probably to some extend variable across participants and reflect individual responses, as brains are always intrinsically shaped by individual development and experiences.

If we consider discrete emotion systems as wide-spread, distinct neural activation distinct to each emotion state, successful pattern classification of brain states across emotions would provide support for separate neural systems for each emotion. In turn, constructivist emotion theories suggest that all emotions are generated by a shared set of fundamental functional systems which are however not specific to emotional processing per se (Kober ; Lindquist ). The present data show that different emotions are associated with granular activation changes across multiple functional systems, and their spatially distributed configuration ultimately defines the specific emotion at both psychological and behavioral levels (Meaux and Vuilleumier, 2015). For instance, two emotions might share their somatosensory representations, but underlying interoceptive representations could be different. Thus, the general configuration of the central and peripheral nervous system leads to distinct emotion states.

However, we stress that the current data cannot resolve whether these functional signatures of distinct emotions would be necessary for each emotion, or whether the presently observed structure of the emotions is optimal, as the classification solution is contingent on the employed a priori category labels. Further, it must be noted classification analysis does not readily reveal the actual neural organization of each emotion system, as the pattern classification only tells us that, on average and at the level measurable with BOLD–fMRI, the cortical/neuronal pattern underlying each category differ enough to be separated, whereas localizing the actual source of differences is more difficult. Therefore, we have complemented the pattern classification analysis with visualization of different emotion categories using GLM and clustering. Furthermore, the pattern recognition techniques employed in this study cannot provide causal evidence for the existence of basic emotion systems: even if we find emotion-specific activation patterns for specific emotions, it does not prove that these patterns are strictly necessary for the corresponding emotional state (Nummenmaa and Saarimäki, 2017). The causality issues can be resolved, for instance, in lesion studies or with brain stimulation techniques. Moreover, the current analyses do not directly answer whether the data are better described by discrete versus dimensional models of emotion.

If basic emotions were somehow ‘special’ or different from non-basic emotions at the neural level, we should observe (1) distinct neural activation patterns for basic emotions but not for non-basic emotions, or (2) different (or perhaps additional) neural systems underlying basic and non-basic emotions. Our classification results and cumulative maps show that both basic and non-basic emotions could be classified accurately, and they elicited activation in largely overlapping brain areas. On average, classification accuracies were higher for basic emotions than for non-basic emotions. This suggests that while there exists partially exclusive neural codes for a multitude of emotion states, as those investigated here, the canonical basic emotions are more discrete than the complex social emotions included in this study, suggesting that there are both universal and experience-dependent components to emotions. Also, classification accuracies within single regions showed that only basic emotions could be distinguished in some areas, including somatomotor regions (insula and supplementary motor area), midline areas (PCC and ACC), and inferior frontal gyrus (Supplementary Figure S3). Another potential explanation for the differences in classification accuracies between basic and non-basic emotions is that maybe there is a clearer cultural understanding or prototypical experience related with basic emotions, manifested as clearer patterns underlying these emotions. To test this hypothesis, we calculated prototypical experience scores for each emotion, defined as the sum of off-diagonal elements in recollected experiential and neural similarity matrices separately, and compared the average prototypicality scores for basic and non-basic emotions. This allowed us to quantify the specificity vs confusion of recognition for different emotion categories. There were no differences between basic and non-basic in either the neural or recollected experiential data, suggesting that both basic and non-basic emotions have equally distinct neural and experiential underpinnings.

Correspondence between neural and phenomenological organization of emotions

An explorative hierarchical clustering of emotion-specific neural patterns identified four clusters in the neural data that correspond to positive emotions (pride, longing, happiness, gratitude, love), negative basic emotions (disgust, sadness, fear, shame) and negative social emotions (anger, guilt, contempt, despair) and surprise (Figure 3). Clustering is an exploratory and descriptive technique that does not allow strong inferences about underlying causal structure. Rather, clustering of neural similarities was used to reduce the dimensionality of the data for visualization purposes. Comparison of subjectively experienced similarity of emotions and the similarity of the neural patterns suggested a direct link between the whole-brain neural signatures of emotions and the corresponding subjective feelings: the more similar neural signatures two emotions had, the more similar they were experienced. This accords with prior work suggesting that emotion-specific neural activation patterns might explain why each emotion feels subjectively different (Damasio ; Saarimäki ). Emotions might constitute distinct activity patterns in regions processing different emotion-related information, such as somatosensory (bodily sensations), motor (actions), as well as brainstem and thalamocortical loops (physiological arousal). Activation from these areas is then integrated in the cortical midline, such integration then giving rise to the interpretation of the subjective feeling (Northoff and Bermpohl, 2004; Northoff ). Thus, a subjective feeling of a specific emotion stems from the net activation of different sub-processes, rather than solely on the basis of any single component of emotion.

Organization of the affective space in the brain

Our hierarchical clustering analysis and cumulative mapping reveal how different patterns of activity may give rise to different emotions. First, midline regions including ACC, PCC, and precuneus were activated during most emotions. These regions might code emotional valence (Colibazzi ; Chikazoe ), participate in self-relevant and introspective processing (Northoff and Bermpohl, 2004; Northoff ), and integrate information of internal, mental, and bodily states (Northoff ; Buckner and Carroll, 2007). We also found consistent emotion-dependent activity in the brainstem, including periaqueductal grey, pons and medulla, for almost all emotions. This activation might reflect the control of autonomic nervous system’s reactions to different emotions (Critchley ; Linnman ) and/or covert activation of particular motor programs (Blakemore ). Also, subcortical regions including amygdala and thalamus showed distinct activity patterns that differed between clusters. Both of these regions are related to salience processing and emotional arousal (Anders ; Adolphs, 2010; Damasio and Carvalho, 2013; Kragel and LaBar, 2014) and show specific activation patterns for basic emotions (Wang ), findings that we now extent also to non-basic emotions.

Second, somatomotor areas including premotor cortex, cerebellum (including vermis and the anterior lobe), globus pallidus, caudate nucleus, and posterior insula were activated during most emotions, but according to the cluster visualizations especially during the processing of emotions that have a strong impact on action tendencies and avoidance-oriented behaviors (fear, disgust, sadness, shame, surprise; Frijda ). These areas are engaged during emotion perception (Nummenmaa , 2012; Pichon ), emotion regulation (Schutter and van Honk, 2009), and somatomotor processing and action preparation related to emotional processing (Kohler ; Wicker ; Mazzola ).

Third, anterior prefrontal cortex was activated especially during positive emotions (happiness, love, pride, gratitude, longing) according with previous research linking anterior prefrontal cortex with positive affect (Bartels and Zeki, 2004; Zahn ; Vytal and Hamann, 2010). Fourth, negative emotions such as guilt, contempt, anger, and despair clustered together, potentially reflecting their social dimension and interpersonal aspects or their self-conscious nature. Especially, left hemisphere activation in orbitofrontal cortex connected to rewards and punishments (Kringelbach and Rolls, 2004), as well as in inferior frontal cortex and dorsolateral prefrontal cortex, which subserve language and executive control (Poldrack ; Kane and Engle, 2002), and in anterior insula linked to processing of social emotions (Lamm and Singer, 2010) was activated during these emotions. Fifth, surprise did not resemble any of the other emotions included in this study, but was instead closest to the neutral state. This is in line with previous research showing that surprise tends to separate from other emotions in subjective ratings (Toivonen ).

Finally, we also found decreased activation in auditory areas and increased activation in visual areas during the imagery of all emotion categories, likely reflecting the offset of the auditory stimulation followed by mental imagery of the emotion-evoking situation (Ganis ) and emotion-related modulation of this activity (Holmes and Mathews, 2005; Nummenmaa ; Kassam ).

Limitations

Despite the careful selection of the emotional stimuli and k-means clustering suggesting clear categorical structure in the evoked affect (see Figure 1), it is possible that the narratives did not fully capture the target emotion only, and might have elicited also a mixture of emotions. Yet these may (1) arise in different time points during the narratives and (2) be not as strong as the main target emotions (see Figure 1), thus average trial-wise activations most likely pertains to the target emotion. Despite this, the observed MVPA pattern may reflect whether each narrative is dominated by one emotion, or at least show the weighted influence of each emotion on a voxel activity. Therefore, the successful classification per se shows that at least the target emotions were successfully elicited, yet, better classification accuracies could potentially be reached if stimuli could target more selectively one category at the time.

Emotions were elicited using narrative-guided emotional imagery. The narrative stimuli across different categories of emotion differed along dimensions that are not of interest (for instance, different words or different kinds of imagery prompted), thus creating variation both within and between emotion categories, though the latter is unlikely to systematically correlate with particular emotions. Moreover, any within-category variation would probably work against the classifier by lowering the classification accuracy. Despite this, we observed above chance level accuracy for all emotion categories except longing and shame. However, it is possible that there were differences between categories in, for instance, evoked emotional imagery, that vary across emotion categories in some uncontrolled manner, thus potentially in part affecting the classification accuracy. This may constitute a limitation of our study but is also inherently related to variations in scenarios and appraisal dimensions that constitute distinct emotion types.

Conclusions

Our results characterize the distinct and distributed neural signatures of multiple emotional states. Different emotions result from differential activation patterns within a shared neural circuitry, mostly consisting of midline regions, motor areas and subcortical regions. The more similar the neural underpinnings of these emotions, the more similarly they also are experienced. We suggest that the relative engagement of different parts of this system defines the current emotional state.

Supplementary data

Supplementary data are available at SCAN online.

Click here for additional data file.