Effects of left and right medial temporal lobe resections on hemodynamic correlates of negative and neutral scene processing.

Enhanced visual cortex activation by negative compared to neutral stimuli is often attributed to modulating feedback from the amygdala, but evidence from lesion studies is scarce, particularly regarding differential effects of left and right amygdala lesions. Therefore, we compared visual cortex activation by negative and neutral complex scenes in an event-related fMRI study between 40 patients with unilateral temporal lobe resection (TLR; 19 left [lTLR], 21 right [rTLR]), including the amygdala, and 20 healthy controls. We found preserved hemodynamic emotion modulation of visual cortex in rTLR patients and only subtle reductions in lTLR patients. In contrast, rTLR patients showed a significant decrease in visual cortex activation irrespective of picture content. In line with this, healthy controls showed small emotional modulation of the left amygdala only, while their right amygdala was activated equally by negative and neutral pictures. Correlations of activation in amygdala and visual cortex were observed for both negative and neutral pictures in the controls. In both patient groups, this relationship was attenuated ipsilateral to the TLR. Our results support the notion of reentrant mechanisms between amygdala and visual cortex and suggest laterality differences in their emotion-specificity. While right medial temporal lobe structures including the amygdala seem to influence visual processing in general, the left medial temporal lobe appears to contribute specifically to emotion processing. Still, effects of left TLR on visual emotion processing were relatively subtle. Therefore, hemodynamic correlates of visual emotion processing are likely supported by a distributed cerebral network, challenging an amygdalocentric view of emotion processing.

INTRODUCTION

Preferential processing of emotional and especially negative stimuli enhances evolutionary fitness by enabling adaptive behavior. On the neural level, this preferential processing is reflected by enhanced blood oxygenation level dependent (BOLD) response to negative scenes in visual brain areas (Aldhafeeri et al., 2012; Lang et al., 1998; Reisch et al., 2020; Sabatinelli et al., 2005). A possible mechanism underlying emotional modulation of visual cortex is reentrant feedback originating in the amygdala (Vuilleumier, 2005), which receives subcortical (McFadyen et al., 2017; Méndez‐Bértolo et al., 2016) and cortical visual input (Kravitz et al., 2013) and projects back to the ventral visual processing stream (Amaral et al., 2003; Freese & Amaral, 2006).

Causal evidence for this model came from studies showing decreased emotional modulation of BOLD response to fearful faces in visual processing areas following unilateral amygdala sclerosis, especially ipsilateral to the lesion (Vuilleumier et al., 2004), or to emotional body expressions after unilateral temporal lobe resections (TLR) including the amygdala (Van de Vliet et al., 2018). However, for pictures of complex scenes, Edmiston et al. (2013) demonstrated preserved emotional modulation of visual cortex after unilateral TLR. This challenges the causal role of medial temporal lobe structures for emotion‐driven enhancements in visual processing and supports the assumption that other brain regions substantially contribute to visual emotion processing (Pessoa & Adolphs, 2010; Petro et al., 2017). However, the lack of group differences in the study by Edmiston et al. (2013) might also be due to analysis across different sides of TLR. Although patients with left and right TLR did not appear to differ in emotional modulation of BOLD response, given the small and unequal sample sizes of the subgroups (5 left, 13 right), small differences might have gone undetected and deficits in one patient subgroup could have been masked by preserved emotional modulation in the other. Other studies on effects of amygdala lesions on visual emotion processing suggested a higher impact of left TLR on visual emotion processing (Van de Vliet et al., 2018) or examined patients with unilateral left or bilateral but not unilateral right amygdala sclerosis (Vuilleumier et al., 2004). Therefore, the larger number of right TLR patients in the study by Edmiston et al. (2013) could have contributed to the preserved emotional modulation. Furthermore, unlike the other studies (Van de Vliet et al., 2018; Vuilleumier et al., 2004), Edmiston et al. (2013) used a blocked fMRI design, which could enable sustained intrinsic emotional modulation in visual cortices independent of amygdala feedback (Miskovic & Anderson, 2018). An event‐related fMRI study in a reasonably large sample of patients with left and right TLR might be more sensitive in detecting effects of unilateral TLR on visual processing of complex emotional scenes and further specify possible effects of laterality. Moreover, in line with the conceptualization of the amygdala as a general relevance detector (Janak & Tye, 2015; Sander et al., 2003), several fMRI studies showed amygdala activation also by neutral stimuli (Ousdal et al., 2008; Reisch et al., 2020), especially when these were unpredictable (Herry et al., 2007) or novel (Schwartz et al., 2003). This implies that effects of TLR on visual processing could be general and not specific to emotional stimuli, which has not been addressed previously.

Against this background, we investigated visual BOLD response during free viewing of negative and neutral pictures of complex scenes with an event‐related fMRI paradigm in a comparatively large sample of patients with unilateral left and right TLR. Our aim was to test if previously demonstrated preserved emotional enhancement of visual activation by complex scenes (Edmiston et al., 2013) might be explained by diverging effects of left and right TLR or the blocked fMRI design rather than pointing to robust emotion processing from complex scenes. According to previous event‐related fMRI studies (Van de Vliet et al., 2018; Vuilleumier et al., 2004), we expected reduced emotional modulation of BOLD response in visual processing areas following left TLR, which should be particularly evident ipsilateral to the TLR (Vuilleumier et al., 2004). In contrast, we expected preserved emotional modulation in right TLR patients as suggested by results of Edmiston et al. (2013), whose sample comprised many more right than left TLR patients. We further addressed the question, if amygdala response in healthy controls is specific to negative pictures or rather general and to what extent the response‐profile is laterality dependent and may account for alterations seen in patients. Ratings of valence and arousal were examined to further investigate potential effects of unilateral TLR on subjective emotional appraisals.

MATERIALS AND METHODS

Participants

Nineteen patients with left (lTLR) and 21 patients with right (rTLR) temporal lobe resection and 20 healthy controls participated in the study, one additional lTLR patient had to be excluded because of a postoperative cerebral infarction affecting visual areas. Patients had a history of pharmacologically intractable medial temporal lobe epilepsy and were recruited from the Epilepsy Centre Bethel based on clinical criteria (presence of unilateral TLR including the amygdala conducted at least 2 years ago, high level of functioning in terms of self‐reliant living, mostly positive postoperative outcome, no severe other lesions in the brain or psychiatric or neurological diseases other than epilepsy) with the constraint that we aimed at an equal number of females and males in the sample. The resulting sample covered a wide age range (21–58 years) and patients had diverse educational backgrounds. Demographic information for all experimental groups as well as clinical characteristics of the patient groups are detailed in Table 1. All rTLR patients had undergone unilateral anteromedial TLR, comprising amygdala, hippocampus, and surrounding cortical tissue, while in the lTLR patients, resections consisted of anteromedial (n = 14) and apical TLRs (n = 5; Elsharkawy et al., 2011), the latter including amygdala and anterior cortical tissue, while sparing the hippocampus (see Figure 1). Both types of lTLR were combined in one patient group, as the amygdala as our structure of theoretical interest was included in both resection types and the two subgroups did not differ in emotional modulation of BOLD response or emotional appraisals. Healthy controls were recruited by word‐of‐mouth or advertisements in social media and selected to match the patient groups regarding sex (χ 2 (2) = 0.26, p = .880, Cramer's V = .07), age (F (2,57) = 0.09, p = .917, η p 2 <0.01) and highest educational achievement (p = .731, Fisher's exact test). Controls were free from self‐reported current or previous brain injuries or neurological and psychiatric problems and showed lower depression scores than both patient groups (F (2,55) = 3.57, p = .035, η p 2 = 0.12), measured using the Beck Depression Inventory (Hautzinger et al., 2009). There was no group difference in anxiety scores (state anxiety: F (2,56) = 3.02, p = .057, η p 2 = 0.10; trait anxiety: F (2,56) = 2.14, p = .127, η p 2 = 0.07), measured using the State‐Trait Anxiety Inventory (Laux et al., 1981). The two patient groups did not differ regarding language lateralization (p = .307, Fisher's exact test) determined using fMRI or Wada test (Wegrzyn et al., 2019; Woermann et al., 2003), age at epilepsy onset (t (38) = 0.02, p = .982, d = 0.01), age at surgery (t (38) = 0.32, p = .749, d = 0.10), time since surgery (t (38) = 0.39, p = .701, d = 0.13), or postoperative outcome according to Engel class (Engel et al., 1993) judged 2 years after surgery (p = .164, Fisher's exact test). Vision of all participants was normal or corrected to normal. All participants gave written informed consent according to the Declaration of Helsinki and received a financial compensation of 100€ for their participation in the full study, which also included an EEG experiment as well as several behavioral experiments and extensive neuropsychological testing, for which data will be reported elsewhere. The study was approved by the Ethics Commission of the German Psychological Society (DGPs).

TABLE 1

Demographic and clinical information of experimental groups

	Controls (n = 20)	lTLR (n = 19)	rTLR (n = 21)
Sex (female:male)	10:10	11:8	11:10
Handedness (left:ambidextrous:right)	0:0:20	4:1:14	0:0:21
Language lateralization (left:right:inconclusive)	—	15:2:2	20:1:0
Years of age (M [SD])	36.60 (12.39)	37.89 (12.68)	36.33 (12.49)
Depression (BDI‐II; M [SD])	3.50 (4.01)	7.76 a (7.98)	8.38 (6.51)
State anxiety (STAI‐S; M [SD])	33.50 (6.09)	35.89 b (10.86)	40.52 (10.37)
Trait anxiety (STAI‐T; M [SD])	35.00 (9.75)	41.06 b (12.07)	41.57 (11.60)
Years of age at epilepsy onset (M [SD])	—	16.71 (11.61)	16.79 (9.67)
Years of age at surgery (M [SD])	—	33.37 (12.06)	32.14 (11.93)
Mean months since surgery (M [SD])	—	53.79 (29.68)	50.57 (21.71)
Outcome of surgery (Engel class I:II:III:IV)	—	15:1:3:0	19:1:0:1

Note: lTLR: left temporal lobe resection; rTLR: right temporal lobe resection. BDI‐II: Beck Depression Inventory II (Hautzinger et al., 2009). STAI‐S: State‐Trait Anxiety Inventory, State subscale; STAI‐T, State‐Trait Anxiety Inventory, Trait subscale (Laux et al., 1981). Language lateralization was determined using fMRI or Wada test (Wegrzyn et al., 2019; Woermann et al., 2003). Outcome of surgery was assessed in follow‐up examination 2 years after surgery.

Data of two patients were missing.

Data of one patient were missing.

FIGURE 1

Overlap of temporal lobe resections, which consisted of anteromedial (n = 14) and apical (n = 5) temporal lobe resections in the lTLR patients (top row) and of anteromedial temporal lobe resections (n = 21) in all rTLR patients (bottom row). Resections are superimposed on the mean structural T1 image of the respective patient group. Color bar represents the number of patients with overlapping resections. Resected areas were traced manually in the individual structural T1 images of the patients and converted to MNI space by applying the deformation fields derived from normalizing procedure in the preprocessing of the respective patient to the resection mask

7.76

35.89

41.06

Stimuli

Pictures were taken from the International Affective Picture System (Lang et al., 2008) and supplemented by analogously constructed and validated own pictures. The stimulus set consisted of 60 negative and 60 neutral scenes, which were matched for the depiction of persons or animals and visual characteristics like contrast, brightness, or complexity. IAPS numbers or brief content descriptions of our own supplementary pictures are detailed in Table S1 in the Supporting Information. It should be noted that negative and neutral scenes used in our study differed in both valence and arousal (see Table 2 for results of valence and arousal ratings). In the following, the term “negative pictures” refers to highly arousing negative pictures. No positive scenes were chosen for the “emotional” condition to reduce scanning time for the participants and because negative stimuli are assumed to induce stronger amygdala activation than positive ones (Costafreda et al., 2008; Straube et al., 2008).

TABLE 2

Results of valence and arousal ratings (M [SD])

		Controls (n = 20)	lTLR (n = 18)	rTLR (n = 20)
Valence	Negative	1.87 (0.54)	2.04 (1.18)	2.04 (0.71)
	Neutral	4.39 (0.51)	4.53 (0.82)	4.31 (0.77)
Arousal	Negative	5.31 (1.23)	4.48 (1.81)	4.90 (1.70)
	Neutral	2.58 (1.23)	2.67 (1.11)	3.08 (1.18)

Note: The scale for valence ratings was ranging from 1 = negative to 7 = positive and for arousal from 1 = low arousing to 7 = high arousing.

Experimental design

The present fMRI experiment was part of a larger project, which took place on two consecutive days: The first day comprised the fMRI and an analogous EEG experiment that employed the same stimuli as the present fMRI experiment (see Mielke et al., 2022 for description of the EEG experiment and report of EEG data). The order of fMRI and EEG experiments was counterbalanced and did not differ between groups (χ 2 (2) = 1.10, p = .576). On the second day, participants completed several behavioral experiments, including ratings of a subset of the stimuli, as well as extensive neuropsychological testing (to be reported elsewhere). In the event‐related fMRI experiment, participants viewed 40 negative and 40 neutral pictures randomly selected from the stimulus set. There was no specific task in the fMRI experiment, but participants were instructed to attentively view the pictures. Each stimulus was presented once and the sequence of emotion conditions of the presented stimuli was pseudo‐randomized. Stimulus presentation started with a fixation cross for 3,000 ms. Stimuli were presented centered on a black background for 2000 ms with a jittered inter‐stimulus interval (i.e., fixation cross), ranging from 2500 to 23,000 ms. Randomization of conditions (negative and neutral) and jitter of the inter‐stimulus intervals were generated using the fMRI simulator software version 9 by Chris Rorden (https://github.com/neurolabusc/fMRI-Simulator). The fMRI experiment also included two analogously constructed runs, in which faces and words were presented (data to be reported elsewhere; see Reisch et al., 2020 for description of the full fMRI experiment). Position of the picture run was counterbalanced and the number of participants who viewed pictures in the first, second, or third run did not differ between groups (χ 2 (4) = 2.25, p = .691). In the rating task, 10 negative and 10 neutral pictures, randomly selected from the stimuli shown in the fMRI experiment, were re‐presented for rating on a 7‐point scale regarding valence and arousal. All experiments were created using Presentation software (Neurobehavioral Systems Inc., http://www.neurobs.com).

Image acquisition

MRI data were recorded using a 3T Magnetom Verio Scanner (Siemens, Erlangen, Germany) with a 12‐channel head coil. High‐resolution T1‐weighed structural images were acquired in 192 sagittal slices (TR = 1900 ms, TE = 2.5 ms, voxel size = 0.75 × 0.75 × 0.8 mm, matrix size = 320 × 320 × 192). Functional echo‐planar images were collected in 35 coronal slices (TR = 3000 ms, TE = 33 ms, flip angle = 90°, voxel size = 2.4 × 2.4 × 4 mm, matrix size = 80 × 80 × 35). Functional scans were oriented orthogonal to the hippocampus, so that amygdala and hippocampus were included in as many slices as possible. The first three volumes were excluded as dummies, resulting in a total of 207 volumes.

Statistical analysis

Ratings of valence and arousal were analyzed in two separate 2 × 3 analyses of variance (ANOVAs) with emotional content (negative vs. neutral) as within factor and group (lTLR vs. rTLR vs. controls) as between factor using IBM SPSS Statistics 21.

Preprocessing and analysis of fMRI data was conducted using SPM12 (https://www.fil.ion.ucl.ac.uk/spm/software/spm12/) running under MATLAB R2015a (The MathWorks, Inc., Natick, MA). Preprocessing comprised slice timing correction, manual artifact correction using ArtRepair toolbox (Mazaika et al., 2005), realignment, co‐registration to the structural images. Following the approach used by Edmiston et al. (2013), normalization to MNI space was accomplished using the unified segmentation procedure (Ashburner & Friston, 2005) implemented in SPM12, since this approach has been shown to be quite robust to brain lesions, even without applying cost function masking (Crinion et al., 2007). Spatial smoothing was applied with an 8 mm full‐width‐half‐maximum Gaussian kernel. Manual artifact correction resulted in an interpolation of 0.56% of all collected volumes (maximum of 2.58% in a single participant).

Two‐stage mixed effect models were set‐up for statistical analyses. For each subject, first‐level contrasts were created by modeling the negative and the neutral condition as well as the contrast negative > neutral and all pictures (negative + neutral) against baseline (i.e., fixation cross) with the hemodynamic response function. Movement parameters from the realignment were included in the model as covariates of no interest. In the second‐level analysis, individual contrast images for negative and neutral pictures were entered into a 2 (valence: negative, neutral) × 3 (group: lTLR, rTLR, and controls) whole‐brain ANOVA in a full‐factorial design.

The contrast negative > neutral was additionally examined within each group with whole‐brain one‐sample t‐tests and in regions of interest (ROIs) of the ventral visual cortex to specifically compare patterns of emotional modulation between the three groups. For the ROI analysis, two symmetrical functional ROIs in the ventral visual cortex were derived from an independent sample of 34 healthy participants, who underwent the same fMRI paradigm in the pilot phase of the present study (Reisch et al., 2020) and were not part of the actual control group. ROIs were defined as clusters showing emotional modulation in a whole‐brain one‐sample t‐test on the contrast negative > neutral thresholded voxel‐wise at p <.05 family‐wise error rate corrected to keep the ROIs relatively small and limited to ventral visual areas. Mean contrast estimates of the contrast negative > neutral were extracted from the ROIs and tested against zero within each group and hemisphere with Bonferroni‐correction for multiple comparisons.

To analyze the influence of time since surgery, age at surgery, and age at epilepsy onset on emotional modulation, the respective clinical variable was included as covariate in separate whole‐brain one‐sample t‐tests on the contrast negative > neutral in the patients to check for any correlations.

To address the question if amygdala activation is specific to negative pictures or rather general, emotional modulation (negative > neutral) as well as general activation (negative + neutral against baseline) of the amygdala was explored in whole‐brain analyses within groups. Laterality differences in emotion‐specificity were additionally analyzed within the controls in structural ROIs of the amygdala, defined using the Harvard‐Oxford subcortical atlas, in a side (left vs. right) x valence (negative vs. neutral) ANOVA.

We further analyzed correlations between activation in the functional ROIs of the visual cortex and activation in the structural ROIs of the amygdalae within controls and the two patient groups. Contrast estimates in the patients were extracted from ROIs of both left and right amygdala, thus including the resected amygdala. This was intended as a sanity check, since correlations between activation in the resected amygdala (i.e., activation within the cerebrospinal fluid filling the resected area) and visual regions would have pointed to false positive activation. Correlations were analyzed at a one‐tailed threshold of p <.05, as we specifically expected a positive relationship between activation of amygdala and visual cortex. Between groups correlations were compared using Fisher's Z at a threshold of Z = 1.645 according to p <.05 one‐tailed. The MNE‐Python package (Gramfort et al., 2013) was used for visualization of correlations.

In line with Edmiston et al. (2013), whole‐brain analyses were thresholded voxel‐wise at p <.001 (uncorrected) and corrected for multiple comparisons using a topological false discovery rate (FDR) of q <0.05 as cluster‐forming threshold. For analyses of amygdala activation, an arbitrary cluster threshold of k = 10 voxels was applied to account for the small volume of the amygdala. For post hoc tests of the whole‐brain ANOVA and the ROI analyses, mean contrast estimates were extracted from respective contrast images using Marsbar toolbox in SPM12 (Brett et al., 2002) and following analyses were conducted with IBM SPSS Statistics 21.

To rule out influences of language lateralization, fMRI analyses were repeated following exclusion of patients with atypical or inconclusive language lateralization. Since there were no changes in the pattern of results, we report analyses of the full sample. We also tested exclusion of patients with a poor postoperative outcome (Engel class III or IV) but there were no changes in the pattern of results of the whole‐brain ANOVA and only minor changes in the analyses of emotional modulation within the patient groups.

RESULTS

Behavioral results

Results of valence and arousal ratings are displayed in Table 2. A main effect of emotional content confirmed that negative pictures were rated as more negative (F (1,55) = 263.64, p <.001, η p 2 = 0.83) and also more arousing (F (1,55) = 92.34, p <.001, η p 2 = 0.63) than neutral pictures. There was no main effect of group (valence ratings: F (2,55) = 0.43, p = .656, η p 2 = 0.02; arousal ratings: F (2,55) = 0.75, p = .477, η p  = 0.03) and no interaction of emotional content and group in the ratings (valence ratings: F (2,55) = 0.28, p = .757, η p 2 = 0.01; arousal ratings: F (2,55) = 1.91, p = .157, η p 2 = 0.07).

Imaging results

Whole‐brain ANOVA

Results of the whole‐brain ANOVA (Table 3) revealed a significant main effect of valence in bilateral ventral visual pathways, including inferior temporal and fusiform gyri, and lateral occipital cortices, where negative pictures elicited stronger activation than did neutral pictures. Conversely, stronger activation by neutral than by negative pictures was observed in the bilateral lingual gyri, the right superior frontal gyrus, and a cluster spanning rolandic operculum, superior temporal gyrus, and posterior insula. Moreover, there was a main effect of group in a cluster spanning right fusiform and lingual gyrus (Figure 2), in which rTLR patients showed less activation than controls and lTLR patients by pictures in general, irrespective of valence. The lTLR patients and controls did not differ in their overall activation. This main effect of group did not overlap with the resected area but was located directly posterior to the resection. There was no interaction of valence and group, indicating no group differences in emotional modulation.

TABLE 3

Peak voxels of clusters showing significant effects in the whole‐brain ANOVA

			Peak voxel
Side	Region	Cluster size	F‐value	x	y	z	Direction of effect
Main effect of valence
R	Inferior temporal gyrus	1142	28.20	48	−54	−6	neg > ntr
	Fusiform gyrus
	Lateral occipital cortex
L	Inferior temporal gyrus	824	26.86	−46	−68	−4	neg > ntr
	Fusiform gyrus
	Lateral occipital cortex
R	Lingual gyrus	210	26.20	26	−40	−10	ntr > neg
L	Lingual gyrus	246	28.51	−22	−42	−10	ntr > neg
R	Superior frontal gyrus	310	21.29	20	22	52	ntr > neg
R	Rolandic operculum	676	18.90	66	−12	2	ntr > neg
	Superior temporal gyrus
	Posterior insula
Main effect of group
R	Fusiform gyrus	438	16.53	18	−46	−10	con > rTLR
	Lingual gyrus						lTLR > rTLR
Interaction of valence × group
	—

Abbreviations: con, controls; lTLR, left temporal lobe resection; L, left; neg, negative; ntr, neutral; rTLR, right temporal lobe resection; R, right.

FIGURE 2

(a) F‐values of the main effect of group in the whole‐brain ANOVA superimposed on the mean structural T1‐image of the rTLR patients. L, left; R, right. (b) Mean contrast estimates extracted from the cluster showing a significant main effect of group in the right fusiform and lingual gyrus. Error bars denote 95% confidence intervals. Asterisks indicate Bonferroni‐corrected p‐values of post hoc tests. **p <.01; ***p <.001

Emotional modulation within groups

Given our specific interest and the paucity of data in this area, emotional modulation was additionally explored within each group, as a difference could go undetected in a three‐group design, if two groups show very similar responses. Whole‐brain analyses (Table 4, Figure 3a) showed enhanced BOLD response to negative pictures for both controls and rTLR patients in bilateral ventral visual areas, comprising the fusiform and inferior temporal gyrus as well as lateral occipital cortex. In the lTLR patients, no emotional modulation was observed at the applied cluster‐forming threshold according to q <0.05 (FDR‐corrected) but in smaller clusters in bilateral ventral visual cortices.

TABLE 4

Peak voxels of clusters showing emotional modulation in the contrast negative > neutral separately for controls and patients

			Peak voxel
Side	Region	Cluster size	t‐value	x	y	z
Controls
L	Fusiform gyrus	203	4.44	−40	−54	−12
	Lateral occipital cortex
R	Inferior temporal gyrus	397	6.22	52	−58	−8
	Lateral occipital cortex
lTLR
L	Inferior temporal gyrus	71 a	4.80	−46	−66	−4
	Lateral occipital cortex
R	Lateral occipital cortex	80 a	4.32	42	−72	−6
rTLR
L	Inferior temporal gyrus	639	5.87	−38	−54	−12
	Fusiform gyrus
	Lateral occipital cortex
L	Lateral occipital cortex	150	4.73	−30	−90	10
R	Inferior temporal gyrus	523	5.68	34	−92	10
	Fusiform gyrus
	Lateral occipital cortex
L	Superior occipital gyrus	90 b	4.71	−24	−76	22

Abbreviations: L, left; R, right.

Cluster size p <.10 (uncorrected).

Cluster size p <.05 (uncorrected).

FIGURE 3

Emotional modulation within groups. (a) T‐values of whole‐brain t‐tests on the contrast negative>neutral thresholded at p <.001 (uncorrected) and an arbitrary cluster‐forming threshold of k = 50 voxels superimposed on the mean structural T1‐image of the respective group (see Table 4 for information on significance of clusters diverging from the originally applied cluster‐forming threshold according to q <0.05 (FDR‐corrected)). Contours delineate functional regions of interest (ROIs) of the ventral visual cortex derived from the contrast negative > neutral in an independent sample of 34 healthy controls. (b) Location of the functional ROIs of the ventral visual cortex (left panel) and mean contrast estimates of emotional modulation (negative > neutral) within ROIs (right panel). Error bars denote 95% confidence intervals. Asterisks indicate Bonferroni‐corrected p‐values of one‐sample t‐tests of emotional modulation against zero. L, left; R, right. *p <.05, **p <.01

Analysis of emotional modulation in functional ROIs in the ventral visual cortex (Figure 3b), defined in an independent sample of 34 healthy participants who underwent the same experimental paradigm, revealed significant enhancements for negative compared to neutral pictures in both hemispheres in the controls and rTLR patients. In contrast, lTLR patients showed significant emotional modulation in the right hemisphere only.

Age at seizure onset, age at surgery, and time since surgery were not correlated to emotional modulation in the patients. However, there was a sub‐threshold cluster of 50 voxels at the mediodorsal edge of the right, resected amygdala in the rTLR patients showing a positive correlation between time since surgery and emotional modulation, indicating increasing emotional modulation with time since surgery.

Analysis of amygdala activation

Figure 4 displays amygdala activation elicited across negative and neutral pictures in the individual groups. In whole‐brain analyses within groups, emotional modulation of amygdala response was only evident at a very liberal threshold of p <.05 (uncorrected) in the left amygdala of the controls. At this liberal threshold, both patient groups showed emotional modulation in the intact amygdala contralateral to the TLR, which was more pronounced in the rTLR patients' left amygdala. Analysis of general amygdala activation (Figure 4, upper row) showed that all pictures, irrespective of their valence, analyzed together against baseline (i.e., fixation) elicited significant activation of bilateral amygdalae in the controls. In the rTLR patients, activation by pictures in general was observed in bilateral clusters spanning putamen and pallidum and in the left hemisphere also extending into the amygdala. The lTLR patients showed no amygdala activation for pictures in general. Follow‐up analyses within emotion conditions (Figure 4, bottom row) revealed that the left amygdala was activated by negative pictures in controls. Still, even the neutral pictures elicited bilateral amygdala activation, which was particularly evident in the controls' right amygdala.

FIGURE 4

Amygdala activation within groups superimposed on the mean structural T1‐image of the respective group. Top row: t‐values of the contrast all pictures (negative + neutral) against baseline (i.e., fixation cross) thresholded at p <.001 (uncorrected) and a cluster‐forming threshold of k = 10 voxels. Bottom row: amygdala activation separately for negative and neutral pictures against baseline zoomed in on the amygdala in a coronal section. Contours outline the amygdala defined by the Harvard‐Oxford Subcortical Atlas. L, left; R, right; neg, negative; ntr, neutral

ROI analysis of amygdala activation in the controls showed an interaction of side and valence (F (1,19) = 4.78, p = .042, η p 2 = 0.20), indicating stronger, even though not significant, emotional modulation in the left amygdala (diffnegative‐neutral: M = 0.12, SD = 0.39, t (19) = 1.39, p = .181), while the right amygdala was activated similarly by negative and neutral pictures (diffnegative‐neutral: M = 0.03, SD = 0.43, t (19) = .37, p = .719). Main effects of side (F (1,19) = 1.57, p = .225, η p 2 = 0.08) and valence (F (1,19) = 0.76, p = .394, η p 2 = 0.04) were not significant.

Correlations between visual cortex and amygdala

As whole‐brain analysis showed only marginal emotional modulation in the amygdala and both negative and neutral pictures induced substantial activation in the amygdalae in the controls, the relationship between activation in the functional ROIs of the visual cortex and structural ROIs of the amygdala was explored separately within negative and neutral pictures (Figure 5). In the controls, amygdala activation by negative pictures in both hemispheres was correlated with activation of the ipsilateral and contralateral visual cortex by negative pictures. Activation by neutral pictures in the left amygdala was similarly associated with bilateral visual cortex activation by neutral pictures, while for the right amygdala this correlation was only observed ipsilaterally. The rTLR patients showed the same correlation pattern, but only for the intact left amygdala. In the lTLR patients, correlations between activation in the intact right amygdala and ipsilateral and contralateral visual cortices were only observed for negative, but not for neutral pictures. Congruent with the amygdala resections, no correlations with visual cortices were observed for the right amygdala in the rTLR patients and for the left amygdala in the lTLR patients. Accordingly, comparisons of correlation coefficients between groups showed significantly reduced correlations in both patient groups for the resected amygdala compared to healthy controls. Correlations between left amygdala and the visual cortices were higher in the rTLR patients than in the lTLR. Conversely, correlations between right amygdala and visual cortex in the lTLR patients did not differ from rTLR patients.

FIGURE 5

Significant positive correlations (Pearson's r) between mean contrast estimates in the amygdala and ventral visual cortex separately for negative and neutral pictures against baseline (i.e., fixation cross) and Z‐scores of their comparison between groups thresholded at p <.05 (one‐tailed). For clarity purposes, correlations between left and right amygdala as well as left and right visual cortex are omitted. L, left; R, right

DISCUSSION

We investigated effects of unilateral TLRs on hemodynamic response to negative and neutral complex scenes as well as subjective appraisals of valence and arousal. We demonstrated largely preserved emotional modulation of visual cortex activation following unilateral left (lTLR) or right (rTLR) temporal lobe resection, as comprehensive testing across groups revealed no significant group difference between patients and controls regarding emotion‐specific activation. Subjective ratings of valence and arousal were also normal in the patients. In contrast, there was a group difference in visual processing in general, irrespective of valence, with rTLR patients showing reduced activation of the ventral visual cortex ipsilateral to the resection. Furthermore, targeted analyses of emotional modulation within groups indicated subtle reductions regarding emotional modulation of hemodynamic activity in visual cortex in lTLR but not in rTLR patients. In the controls, the amygdalae responded not only to negative, but also to neutral pictures, which was particularly evident in the right amygdala. Furthermore, correlations between activation in amygdala and visual cortex were observed irrespective of valence in both hemispheres in the controls, while significantly reduced in both patient groups ipsilateral to the TLR.

To the best of our knowledge, our study is the first to investigate emotional modulation of hemodynamic response to complex visual scenes in two separate reasonably sized groups of patients with left and right TLR including the amygdala using an event‐related fMRI paradigm. We found no differences in emotional modulation of BOLD response between patients and healthy controls, corroborating findings of a previous smaller study in a combined sample of left and right TLR patients using a blocked fMRI design (Edmiston et al., 2013). Healthy controls showed enhanced visual cortex activation by negative pictures even in the absence of differential amygdala activation, which is in line with results of a previous study using the same experimental paradigm in a different sample of healthy participants (Reisch et al., 2020). These findings argue against the widely held assumption that emotional modulation of visual cortices critically depends on reentrant feedback from the ipsilateral amygdala (Pourtois et al., 2013; Vuilleumier, 2005). This amygdalocentric view of visual emotion processing was particularly supported by studies showing that emotional modulation of visual cortices during processing of faces (Vuilleumier et al., 2004) or body expressions (Van de Vliet et al., 2018) is reduced in patients with amygdala lesions. However, pictures of complex scenes induce stronger and more widespread emotional modulation in visual cortices than facial expressions (Britton et al., 2006; Reisch et al., 2020; Sabatinelli et al., 2011), which could be more robust against lesions of medial temporal lobe structures including the amygdala. In line with this, patients with bilateral amygdala lesions seem to be impaired in recognizing emotions from facial expressions presented in isolation but not from complex scenes (Adolphs & Tranel, 2003).

Recently, it has been suggested that visual emotion processing could occur directly in the visual cortex itself (Miskovic & Anderson, 2018; Todd et al., 2020). In general, emotional modulation of visual cortex across different paradigms seems a robust marker of emotion processing (Villalta‐Gil et al., 2017), whereas emotional modulation in the amygdala appears to be rather variable and dependent on task characteristics (Costafreda et al., 2008). Furthermore, pictures of complex scenes also elicit emotional modulation of BOLD response in frontoparietal regions (Edmiston et al., 2013; Reisch et al., 2020; Sabatinelli et al., 2014; Sabatinelli & Frank, 2019; Sambuco et al., 2020), associated with directing attention to behaviorally relevant stimuli (Corbetta & Shulman, 2002). In the present study, frontoparietal activation by negative compared to neutral pictures was also observed in both patients and controls, even though at a more liberal statistical threshold of p <.005. Particularly frontal regions could contribute to visual emotion processing via cortico‐cortical modulations (Petro et al., 2017). Pessoa and Adolphs (2010), based on extant literature, suggested a multiple pathway model of visual emotion processing that contains numerous connections within the primary and secondary visual cortices next to frontoparietal and visuo‐amygdalar connections. Moreover, recent studies also indicate a dynamic interplay of widespread cortical and subcortical functional networks in affective processing (e.g., Wager et al., 2015). These findings, together with our results, suggest that visual emotion processing from complex scenes is supported by a “degenerate network” (Edelman & Gally, 2001) of brain regions associated with various functions, such as basic visual processing, interoception, attention, or cognition (Barrett & Satpute, 2019; Lindquist et al., 2012; Pessoa, 2008), rather than depending primarily on the amygdala. However, neuroscience has only just begun to move from an amygdalocentric and strongly locationist view to a more constructionist view of emotion. Therefore, instead of mainly focusing on the amygdala's role, future research should address whole‐brain networks underlying visual emotion processing.

Even though emotional modulation did not differ significantly between groups, it was descriptively reduced in the lTLR patients, as reflected by within‐group analyses. The lTLR patients only showed emotional modulation in small ventral visual clusters below the originally applied cluster‐forming threshold and ROI analyses pointed to reduced emotion‐specific activation in the left hemisphere, ipsilateral to the resection. Together with previous fMRI reports of a stronger left‐lateralization of emotion processing in the amygdala (Baas et al., 2004; Canli et al., 2000; Lane et al., 1997; Pessoa et al., 2002; Vuilleumier et al., 2001; Wager et al., 2003) and previous lesion studies (Van de Vliet et al., 2018; Vuilleumier et al., 2004), this finding suggests at least some contribution of left temporal lobe structures to hemodynamic correlates of visual processing of negative pictures. Moreover, the rTLR patients showed significantly reduced general activation, irrespective of emotional content, in right ventral visual areas. This is consistent with the assumption of ipsilateral reentrant feedback from the amygdala modulating visual processing (Vuilleumier et al., 2004) and further suggests that, at least in the right hemisphere, reentrant feedback is not specific for negative stimuli.

Reduced general visual cortex activation in rTLR patients and the subtle decrease of emotional modulation in the lTLR patients suggest at least some contribution of medial temporal lobe structures to visual processing, with different roles of the right and left amygdala regarding emotion specificity. This was also supported by analysis of amygdala activation within the controls, who showed some emotion‐specific response in the left amygdala only. Overall, controls exhibited substantial amygdala activation even by neutral pictures, which was more pronounced in the right amygdala. Correlations between amygdala and visual cortex activation in the controls were also observed for both negative and neutral pictures and these correlations were remarkably reduced in both patient groups ipsilateral to the TLR. In fact, correlations also indicated cross‐hemispheric interactions, which might contribute to relatively intact visual processing following unilateral resections.

Emotion‐specific activation of the left and rather general response of the right amygdala has already been demonstrated during face processing (Reisch et al., 2020; Wright & Liu, 2006). Present results further suggest that this laterality difference in emotion specificity could also apply to complex scenes. In general, the high amygdala activation even by neutral pictures in our study is congruent with the conceptualization of the amygdala as a general relevance detector rather than a “fear module” (Janak & Tye, 2015; Sander et al., 2003). Several other studies also demonstrated amygdala response to neutral stimuli, especially when behaviorally relevant (Ousdal et al., 2008), novel (Schwartz et al., 2003), or unpredictable in timing (Herry et al., 2007). Unpredictable picture onsets due to a variable inter stimulus interval (ranging from 2500 to 23,000 ms) could have contributed to overall relatively high amygdala activation in our study. Since amygdala activation is quite sensitive to experimental characteristics (Costafreda et al., 2008; Villalta‐Gil et al., 2017), future studies should more systematically investigate experimental parameters under which emotion‐specific or emotion‐independent amygdala activation is found, including to what extent lateralization is affected by such manipulations.

Of note, the same rTLR patients, who showed intact emotional modulation of visual cortex activation in the present fMRI experiment, showed reduced emotional modulation of early event‐related potentials (ERPs) and ipsi‐resectional reductions also in mid‐latency ERPs in an EEG experiment conducted on the same day as the present fMRI study and using exactly the same stimuli (Mielke et al., 2022). Following early reduction, emotional modulation of ERPs in the rTLR patients became increasingly normal over time and did not differ from controls in late ERP components. Therefore, at least the right amygdala's influence on visual emotion processing seems confined to specific time‐windows in the processing stream. This had already been suggested by two previous smaller EEG studies on emotional face processing in amygdala‐damaged individuals (Framorando et al., 2021; Rotshtein et al., 2010). The results by Mielke et al. (2022) demonstrate that right medial temporal lobe structures are particularly involved in early visual emotion processing from complex scenes, while effects of right TLR can be compensated in later stages of cerebral processing, probably via recruitment of additional cortical structures and perhaps also due to the influence of the intact contra‐resectional amygdala. In line with this, a recent magnetoencephalography study suggested early face processing to be subserved by a cortico‐subcortical network including the amygdala, whereas purely cortical connectivity seems to underlie late stages of visual face processing (Garvert et al., 2014). Since particularly late ERP components are correlated with emotional modulation of BOLD response in visual regions (Sabatinelli et al., 2007, 2013), the transient changes in early visual emotion processing in the rTLR patients (Mielke et al., 2022) are likely obscured by the fMRI's slow hemodynamic response that integrates activity across several seconds. In the future, a combined analysis of the EEG and fMRI data of the rTLR and also lTLR patients could provide more specific insights into the relationship between emotional modulation of ERP components and BOLD response on the level of the individual participant.

In line with previous findings (Buchanan et al., 2001; LaBar & Phelps, 1998; Phelps et al., 1997), subjective ratings of valence and arousal were intact in TLR patients. Given the fact that we observed preserved emotional modulation of hemodynamic response to the pictures, this might not be surprising. However, intact emotion recognition in behavioral measures has previously been reported even in the face of reduced emotional modulation in visual cortices (Van de Vliet et al., 2018; Vuilleumier et al., 2004). This might reflect an influence of cognitive processes and conceptual knowledge about emotions (Lindquist & Barrett, 2008).

In sum, our results indicate that, at least for negative contents, hemodynamic correlates of visual emotion processing from complex scenes do not heavily depend on the integrity of either right or left medial temporal lobe structures including the amygdala. This finding challenges the still widely accepted amygdalocentric view of emotion processing. Instead, emotion processing could occur in the visual cortex itself and may be further supported by a degenerate network of cortical as well as other subcortical regions, including the contralateral amygdala and the frontoparietal network. Nevertheless, we found some contributions of medial temporal lobe structures on hemodynamic correlates of visual processing with different roles of the left and right amygdala. While left medial temporal lobe structures seem to affect emotional modulation of visual cortex activation, right medial temporal lobe structures seem to have a more general influence on visual processing.

CONFLICT OF INTEREST

The authors declare no potential conflict of interest.

Table S1 List of IAPS pictures (Lang et al., 2008) and short description of our own supplementary pictures used in the fMRI experiment

Click here for additional data file.