Orienting to different dimensions of word meaning alters the representation of word meaning in early processing regions.

Conscious processing of word meaning can be guided by attention. In this event-related functional magnetic resonance imaging study in 22 healthy young volunteers, we examined in which regions orienting attention to two fundamental and generic dimensions of word meaning, concreteness versus valence, alters the semantic representations coded in activity patterns. The stimuli consisted of 120 nouns in written or spoken modality which varied factorially along the concreteness and valence axis. Participants performed a forced-choice judgement of either concreteness or valence. Rostral and subgenual anterior cingulate were strongly activated during valence judgement, and precuneus and the dorsal attention network during concreteness judgement. Task and stimulus type interacted in right posterior fusiform gyrus, left lingual gyrus, precuneus, and insula. In the right posterior fusiform gyrus and the left lingual gyrus, the correlation between the pairwise similarity in activity patterns evoked by words and the pairwise distance in valence and concreteness was modulated by the direction of attention, word valence or concreteness. The data indicate that orienting attention to basic dimensions of word meaning exerts effects on the representation of word meaning in more peripheral nodes, such as the ventral occipital cortex, rather than the core perisylvian language regions.

Introduction

An efficient semantic system must be able to adapt to specific situational demands (Jefferies and Lambon Ralph 2006; Noonan et al. 2013; Schumacher et al. 2019). Activation of a concept most likely is partly automatic and context-independent and partly adaptive depending on context, and the admixture of both components is often difficult to tease apart. For instance, imagining carnival will rely on other conceptual aspects than evaluating its societal role, although both aspects will be inevitably activated in response to the word. This dynamic balance between more automatic and more task-driven activation of meaning has recently been conceptualized as a distinction between semantic representation and semantic control (Hoffman et al. 2010; Davey et al. 2016; Lambon Ralph et al. 2016; Chiou et al. 2018). The mechanisms underlying the flexibility of access to semantic knowledge remain unclear. Here, we investigate how orienting attention to basic dimensions of word meaning (concreteness and valence) alters the information coded in regional activity patterns.

Several previous studies have investigated the flexible access to conceptual knowledge by directing attention to or away from prototypical knowledge (Fairhall 2020), specific mental codes (visual, verbal, or semantic; Lewis-Peacock et al. 2015), a conceptual feature of interest (e.g., emotion (Williams et al. 2005; Straube et al. 2011), thematic, taxonomic, or behavioral similarities (Nastase et al. 2017; Wang et al. 2018)). Some of these studies employ tasks that are computationally very dissimilar: for instance, in Lewis-Peacock et al. (2015), subjects are asked to judge whether concepts sometimes occur together, contain a specific string of letters, or have the same shape. These judgments rely on the semantic, verbal, and visual systems, respectively. Effects of selective attention have been observed on both the voxelwise BOLD amplitude (Williams et al. 2005; Straube et al. 2011) and the multivariate representations (Lewis-Peacock et al. 2015; Nastase et al. 2017; Wang et al. 2018; Fairhall 2020). In the current study, we aim to investigate shifts in concept representation along two fundamental dimensions of word meaning: concreteness (concrete vs. abstract words) and valence (positive vs. negative words) and their interaction. We hypothesize that semantic concepts (and the activity patterns they elicit) can be represented in a multidimensional space, where the pairwise distance reflects their semantic similarity. This configuration will shift when attending to concreteness or valence, placing concepts that share the attended dimension closer together (Fig. 1; Nosofsky 1986). When attention is oriented towards valence, regional activity patterns for positive words would become more similar to one another (idem for negative words). This similarity structure would be best modeled by a valence similarity matrix. When attention is oriented towards concreteness, the activity patterns for concrete words would become more similar (idem for abstract words), which would be better modeled using a concreteness similarity matrix (Fig. 1).

Figure 1

Overview of the hypothesis based on Nosofsky (1986).

The selection of concreteness and valence as dimensions of interest is motivated by long-standing evidence of their importance in the conceptual system at the behavioral and neurobiological level (Binder et al. 2005; Cousins et al. 2016, 2017; Hoffman, 2016; Roxbury et al. 2016). Concrete words are acquired and represented based on sensorimotor experience and elicit activation in widespread bilaterally distributed regions, while abstract word processing predominantly activates the left perisylvian regions (Binder et al. 2005; Meersmans et al. 2020). Neural activation patterns for emotion words are associated with orbitofrontal regions, the amygdala, the insula, and the anterior and posterior cingulate cortex (Posner et al. 2009; Citron 2012). Furthermore, valence and concreteness show an interaction effect at the behavioral level: words at the extremes of the valence scale (i.e., positive or negative) tend to be more abstract. At the neurobiological level, an interactive effect between valence and concreteness on voxelwise BOLD responses has been observed in left inferior frontal gyrus and left middle temporal gyrus (Pauligk et al. 2019). We selected words factorially along the concreteness and valence axis and asked subjects to judge stimulus word concreteness or valence. This factorial design is well suited to probing flexibility in semantic activation, as it allows one to probe combinations of the different levels.

Materials and Methods

Participants

A total of 22 subjects (16 women, 6 men; mean age = 22.9 years [sd = 3.5], range: 19–30) were recruited from a population of university students. They completed questionnaires assessing medical history, MRI safety, handedness (Edinburgh Handeness Inventory). Subjects had to be native speakers of Dutch, right-handed and free from neurological disorders or abnormalities in language development. All subjects provided written informed consent prior to participation. The study was approved by the Ethics Committee of University Hospitals Leuven.

Stimulus selection

Stimulus words were selected from the Dutch Small World of Words dataset (SWOW-NL; (De Deyne et al. 2013, 2019) along the valence and concreteness dimensions in a semiautomated manner. SWOW is a large-scale dataset of semantic associations, collected from over 70 000 participants. Subjects were asked to provide the first three responses (i.e., semantic associates) when presented with a cue. We selected 30 concrete positive nouns, 30 concrete negative nouns, 30 abstract positive nouns, and 30 abstract negative nouns, resulting in a total stimulus set size of 120 words (Fig. 2; Supplementary Table 1). On a 1 to 7 scale, negative words were defined as words with valence rating below 3.6, while positive words had valence ratings above 4.6. On a 1 to 5 scale, abstract words were defined as words with concreteness rating below 3, while concrete words had concreteness ratings above 3.5. Ratings for valence and concreteness were taken from Van Rensbergen et al. (2016) and Brysbaert et al. (2014), respectively. Words were selected to maximize the range in pairwise semantic similarities (from closely similar to widely dissimilar), assuming this would increase sensitivity for the correlational Representational Similarity Analysis. The pairwise semantic similarity was calculated from the SWOW dataset by representing the cue-associate pairs as a graph, with edges weighted by associative strength (i.e., the probability of a response given a cue). Pointwise mutual information weighting was applied to disfavor general associations that are elicited by a large number of cues. Semantic similarity is calculated using a random walk algorithm to extract word embeddings from the graph and taking the pairwise cosine similarity. Words were matched on frequency, orthographic neighborhood density, prevalence, and word length. We aimed to match the stimulus groups as closely as possible on age of acquisition, dominance, and arousal, but these variables are closely intertwined with concreteness and valence. Abstract words are generally acquired later in life, while valence, dominance, and arousal are all affective in nature. In the original selection, three words were replaced due to their synonymy with other stimuli (e.g., feest [party] and feestje [Dutch diminutive of party] were judged too similar and feestje was replaced by bruiloft [wedding]) and one word was replaced due to its questionable common/proper and loan word noun status (bazooka was replaced by tornado). These replacements were automatically selected so that matching between subgroups and the distribution of the semantic similarity values was maximally maintained.

Figure 2

A) Stimulus set. A total of 120 nouns were selected from the Small World of Words dataset, evenly spread over each quadrant. Valence was rated on a scale of 1 to 7 (negative to positive), while concreteness was rated on a scale of 1 to 5 (abstract to concrete). The words were presented in Dutch but here we provide their English translation. The colored lines indicate the exact position of words that were shifted slightly to improve readability. B) Visualization of the symmetrical semantic similarity matrix derived from the Small World of Words dataset. A+ abstract positive; A− abstract negative; C+ concrete positive; C− concrete negative stimuli. C) Experiment design, showing an auditory, visual, and control trial. Note that there are two options for probe phase of the experiment: abstract versus concrete as depicted or pleasant versus unpleasant.

Experiment Design

Subject performed a concreteness or valence judgment task during functional magnetic resonance imaging (fMRI) (Fig. 2). They were instructed to either categorize the stimuli as concrete or abstract, or as pleasant or unpleasant. The experiment had a 2 × 2 × 2 × 2 factorial design with stimulus modality (visual/auditory), concreteness (abstract/concrete), valence (positive/negative), and task (valence judgment/concreteness judgment) as factors. A white fixation dot turned green 350 ms before stimulus presentation. Then, words were presented for 1500 ms in auditory or visual modality, accompanied by a visual/auditory control stimulus. Visual controls were derived from the stimulus words by replacing vowels with consonants and shuffling the letter order. Auditory stimuli were delivered through a set of headphones equipped with active noise cancellation technology to eliminate scanner noise (OptoAcoutics Ltd). Auditory controls were generated by rotating the spectrogram around half of the maximal frequency and applying a low-pass filter at 95% maximal frequency. The stimulus was immediately followed by the visual probe question (in Dutch). Two words appeared on the screen simultaneously, either the two words “abstract” and “concrete” or the two words “pleasant” and “unpleasant”, with one word on the left side of the screen and one word on the right side. Subjects were instructed to categorize the stimulus words as abstract/concrete or positive/negative. We did not provide a specific definition for these labels. Subjects held the response box in their right hand and had to press a left- or right-sided button depending on the position of the correct response on the screen. The probe question remained on the screen for 1500 ms and was followed by a white fixation dot (3900 ms). The intertrial interval was 7.25 s. During control trials, a visual and auditory control stimulus of the type described above were presented simultaneously. In control trials, the probe question consisted of consonant strings. Subjects were instructed to press the button on the side where the probe question was capitalized. In order to match the control trials, we also capitalized one of the responses at random during experimental trials. The full experiment consisted of eight runs of 135 trials (120 stimulus +15 control trials), which were divided over two scan sessions, one on each day (average number of days between scans = 7.9; SD = 8.3).

Image acquisition and processing

Structural and functional images were acquired using a 3 T Philips Achieva equipped with a 32-channel head coil and noise cancellation technology (Optoactive II™ active noise canceling headphones; Optoacoustics Ltd). Structural images were acquired using a T1-weighted 3D turbo-field-echo sequence (repetition time = 9.6 ms echo time = 4.6 ms, in-plane resolution = 0.97 mm, slice thickness = 1.2 mm). Functional images were acquired using a T2* sequence with 60 slices (multiband acceleration factor = 2; repetition time = 2 s; TE = 30 ms; voxel size = 2 × 2 × 2.2 mm3). Images were submitted to a preprocessing pipeline in SPM12, comprising of realignment and slice timing correction and coregistration to the anatomical image. A mean functional image was created. The structural image was coregistered with the mean functional image and segmented into gray matter, white matter, and cerebrospinal fluid. Functional and structural images were normalized to MNI space based on the warping parameters obtained during segmentation. For univariate purposes, smoothing with a 5 × 5 × 7 mm3 FWHM Gaussian kernel was applied. For multivariate purposes, no smoothing was applied at this stage.

Univariate Factorial Analysis of Task and Stimulus Type

Using SPM12, we created a general linear model with 17 conditions, i.e., one condition per cell of the factorial design plus the control condition. Motion regressors were included as nuisance variables. At the group-level, we performed a random effects analysis using a one-sample t-test with cluster-level inference at a threshold of whole-brain Family Wise Error (FWE)-corrected P < 0.05 (with voxel-level set at uncorrected P < 0.001). The main effects of task (two levels: concreteness versus valence judgment) and stimulus type (two factors, concreteness and valence, with two levels each: high versus low) were determined as well as the interaction effects between task and concreteness, and between task and valence. We will also report the interaction between concreteness and valence. To help the interpretation of the interaction effects, we calculated time-activity curves in the significant clusters as percentage signal change per condition over time.

Representational Similarity Analysis

To investigate how the neural patterns of the stimulus set depend on and are altered by task, we applied Representational Similarity Analysis. To test the main hypothesis, we used two models: the concreteness similarity matrix (calculated as 1 minus the absolute value of the pairwise difference in concreteness) and the valence similarity matrix (calculated as 1 minus the absolute value of the pairwise difference in valence). The use of different behavioral models allows for testing different hypotheses on the shift that occurs in the representational space depending on the task (Kriegeskorte et al. 2008; Bruffaerts et al. 2019; Fig. 1). To evaluate the effect of stimulus modality, the RSA was repeated for written and spoken stimuli separately.

The correlation between the concreteness and valence model equalled −0.005, the correlation between the concreteness and semantic similarity matrix was 0.16, and the correlation between the valence and semantic similarity matrix was 0.29. The correlation between the valence similarity matrix and semantic similarity matrix was significantly higher than the correlation between the concreteness similarity matrix and the semantic similarity matrix (z = −8.1; P < 10−15).

A region-of-interest-based Representational Similarity Analysis was applied to examine task-dependent changes in representation. We applied this analysis in the clusters showing an interaction effect between task and concreteness, or task and valence. Every cluster was overlayed with a subject-specific gray matter mask before RSA (GM probability > 0.3). Neural patterns per trial were estimated by calculating the integral of the BOLD response from 2–7.25 s post stimulus onset. Per cluster, an fMRI pattern similarity matrix was calculated by taking the cosine similarity between trial-specific vectorized patterns, which were then averaged within and across subjects. This averaged matrix was correlated with concreteness similarity matrix and valence similarity matrix (Spearman’s rho). The significance of this correlation was determined via random permutation labelling of the word labels of the semantic similarity matrix and correlation of this randomized semantic similarity matrix with the fMRI similarity matrix.

Significance was set at a one-tailed P < 0.05 and Bonferroni correction was applied to correct for the number of regions. Confidence intervals (95%) were calculated using bootstrapping on 100 000 samples. The significance of the between-task differences in correlation strength was evaluated using Pearson and Filon’s z as implemented in the cocor package for R (Diedenhofen and Musch 2015; version 1.1.3; R version 3.5.2).

To obtain a wider view of the functional-anatomical context of the effects examined, we also performed a whole-brain RSA with the concreteness similarity and the valence similarity matrix. A subject-level whole-brain Representational Similarity Analysis was performed for valence and concreteness judgments pooled, using a searchlight of 150 voxels (voxel size 2 × 2 × 2.2 mm3; Oosterhof et al. 2016). The resulting correlation maps are R-to-Z transformed and smoothed (FWHM 5 × 5× 7 mm3) before being submitted to a group-level t-test (FWE-corrected threshold P < 0.05 with uncorrected voxel-level threshold P < 0.001). In the significant clusters, task-dependent effects were examined using a regional RSA for valence and concreteness judgments separately.

As a secondary analysis we also performed a whole-brain RSA with the SWOW similarity matrix in order to determine the broader functional-anatomical context in which the valence and concreteness effects occurred. This also allowed for an evaluation whether the effect of valence or concreteness similarities were neuroanatomically separate or overlapping with the encoding of higher level semantic relationships.

Results

Behavioral analysis

Reaction times (Fig. 3) were significantly different depending on task (One-way analysis of variance; F(1, 20 887) = 790.2, P < 2 × 10−16), and stimulus type F(1, 20 886), P < 2 × 10−16). Post hoc Tukey HDS testing revealed that subjects were significantly faster when judging valence (mean RT = 1264.9 ms ± 444.7) than when judging concreteness (mean RT = 1435.7ms ± 85.2; P < 10−14). They were significantly faster for concrete words than for abstract words, while word valence did not have an effect on reaction times (Concrete positive words: mean RT = 1279.8 ms ± 447.2; Concrete negative words: mean RT = 1295.6 ms ± 449.2; Abstract positive words: mean RT = 1349.8 ms ± 454.9; Abstract negative words: mean RT = 1349.7 ms ± 471.7). In addition, we calculated intersubject agreement (i.e., consistency of concreteness and valence judgment across subjects), which was significantly above zero for both concreteness judgments (Fleiss Kappa = 0.05; P < 0.0001) and valence judgments (kappa = 0.31; P < 0.0001; Fig. 2). The low kappa for concreteness judgments was mainly driven by lower intersubject agreement for abstract words, relative to concrete words (t(111,17) = −8.8; P = 1.8 × 10−14). Concreteness ratings are known to vary across subjects (Pollock 2018) depending on the type of words presented (Fig. 2), which can explain the lower interrater agreement during concreteness judgments.

Figure 3

A) Distribution of reaction times according to stimulus type. The black line indicates the median value. B) Distribution of reaction times per task. The black line indicates the median value. C) Intersubject agreement on concreteness judgments and valence judgments. Both are significantly above zero. Data were pooled over subjects and trials for both reaction times and intersubject agreement.

Univariate analysis: task-related main effects and interactions

First, we examined the main effect of task on response amplitude in a univariate analysis. This yielded significant effects for valence judgment > concreteness judgment in a distributed pattern shown in Figure 4 and listed in Supplementary Table 2. The most extensive clusters were the left subgenual and rostral anterior cingulate cortex, the inferior parietal lobule, orbito- and mediofrontal cortex. This effect arises from greater deactivation for concreteness judgments than for valence judgments in the majority of clusters, except for clusters around the left and right superior temporal sulcus and the left fusiform and left lateral occipital clusters (Supplementary Figure 1).

Figure 4

Main effects of A) task, B) concreteness, and C) valence. Significance was set at cluster-level FWE-corrected P < 0.05 (voxel-level uncorrected P < 0.001).

The contrast concreteness judgment > valence judgment activated mostly the bilateral posterior cingulate gyrus and the dorsomedial prefrontal cortex, bilateral inferior parietal cortex, bilateral precuneus, and bilateral basal ganglia, left triangular part of inferior frontal gyrus, and left middle frontal gyrus, regions corresponding to the multiple demand network (Duncan 2010; Assem et al. 2020) (Fig. 4; Supplementary Table 2).

Second, the main effects of stimulus concreteness and stimulus valence were examined (Fig. 4, Supplementary Table 2). Relative to concrete words, abstract word elicited an increase in response amplitude in left inferior frontal gyrus, left posterior superior temporal gyrus, posterior left fusiform gyrus, and right dorsal insula. Concrete words activated bilateral parahippocampal and perirhinal cortex, left posterior cingulate cortex, and left angular gyrus. Relative to negative words, positive words mainly activated right orbitofrontal gyrus, right precuneus, right angular gyrus, right precentral gyrus, and right middle superior frontal gyrus, in addition to left orbitofrontal gyrus and left lingual gyrus. Negative words elicited increased activity in left orbitofrontal and left lingual gyrus.

We investigated the interactions between task and concreteness, and between task and valence (Fig. 5; Table 1). There was a significant interaction between concreteness and task in right posterior fusiform gyrus (medioventral BA37), bilateral precuneus, and left insula. In BA37, the response amplitude was increased for concrete words during concreteness judgment, relative to valence judgments or abstract words. In bilateral precuneus, deactivation was more pronounced for abstract words during concreteness judgment. Third, in the left insula, the response was lower for abstract words during concreteness judgments, relative to the three other conditions.

Figure 5

A) Location of the significant interaction effects with task. Significance was set at whole-brain FWE-corrected P < 0.05 with uncorrected voxel-level P < 0.001. B) Time-activity curve per significant cluster. Error bars represent the standard error of the mean. C) Significant results of the representational similarity analysis with the valence similarity matrix for concreteness and valence judgments separately. Significance was determined by comparing the obtained correlation to a distribution of 100 000 random correlations (Bonferroni corrected P < 0.008; uncorrected P < 0.05). Black lines represent the 95th percentile, red lines the observed correlation values between the fRMI similarity matrix and valence similarity matrix.

Table 1

Interaction effects in the General Linear Model between Task and Concreteness, and Task and Valence. Significance was set at whole-brain FWE-corrected P < 0.05 (with voxel-level uncorrected P < 0.001)

INTERACTION EFFECTS
Concreteness x Task
Label	Size	Peak coordinates			T(21)	FWE-corr. P (voxel)	FWE corr. P (cluster)
Bilateral precuneus	311	16	−52	44	7.15	0.04	2.43 × 10−6
		−10	−50	46	5.29	0.73
		6	−50	46	5.17	0.80
Left insula	83	−42	−2	−14	5.87	0.37	0.03
		−42	−10	−10	5.33	0.71
		−40	−16	−4	4.30	0.99
Right fusiform gyrus	306	28	−70	−16	5.63	0.51	2.89 × 10−6
		28	−78	−14	5.50	0.60
		22	−62	−18	5.13	0.83
Valence x Task
Label	Size	Peak coordinates			T(21)	FWE-corr. P (voxel)	FWE corr. P (cluster)
Left lingual gyrus	184	−20	−78	−16	5.32	0.66	0.0008
		−14	−86	−12	4.50	0.99
		−24	−88	−14	4.50	0.99
Valence x Concreteness
Label	Cluster size	Peak coordinates			T(21)	FWE-corr. P (voxel)	FWE-corr P (cluster)
Left inferior frontal gyrus (BA45)	484	−56	28	12	5.66	0.46	3.39 × 10−8
		−48	20	2	5.41	0.62
		−54	−2	14	5.31	0.69

The interaction between task and valence was significant in left lingual gyrus. In this cluster, BOLD amplitude changes between positive and negative words did only differ when subjects were judging valence, with greatest amplitude change for trials with valence judgments of positive words.

A significant interaction between concreteness and valence was seen in the left inferior frontal gyrus (Table 1; Fig. 6). Abstract words elicited a larger increase in response than concrete words in this region. More importantly, valence had opposite effects for abstract and concrete words: for abstract words, the biggest increase in response was observed for positive words, while for concrete words, the biggest increase was seen for negative words.

Figure 6

A) Location of the significant interaction effects between valence and concreteness. Significance was set at whole-brain FWE-corrected P < 0.05 with uncorrected voxel-level P < 0.001). B) Time-activity curve per significant cluster. Error bars represent the standard error of the mean.

Three-way interactions between valence, task and modality and concreteness, task and modality did not yield significant results.

We used Representational Similarity Analysis to examine the correlation between fMRI similarities and concreteness similarities and valence similarities as a function of task. All analyses reported below were done for all stimuli pooled. Regions were included in this analysis if the region showed a significant interaction effect with task in the univariate analysis (Table 1; Fig. 5; L lingual gyrus, bilateral precunues, L posterior fusiform gyrus, and L insula). Of these clusters, right posterior fusiform gyrus and left lingual gyrus showed task-dependent representational similarities.

In the right posterior fusiform gyrus and the left lingual gyrus, a significant correlation between valence similarities and similarity in fMRI activation patterns was found only during valence judgments (right posterior fusiform gyrus: rho = 0.03, P = 0.006; left lingual gyrus: rho = 0.04, P = 0.0002; Table 2; Fig. 5). There was no significant correlation between valence similarities and similarity in fMRI activation patterns when subjects were judging concreteness (L lingual gyrus: rho = −0.007, P = 0.72; R posterior fusiform gyrus: rho = −0.005, P = 0.66). In both regions, the correlation with the valence similarity matrix was significantly higher during valence judgments than during concreteness judgments (R posterior fusiform gyrus: z = −2.1 and P = 0.04; L lingual gyrus: z = −2.8 and P = 0.005). We tested whether the significant effects depended on stimulus modality by repeating the analysis for written and spoken trials separately. The correlations were not significantly different between modalities in right posterior fusiform gyrus (written: rho = 0.007; P = 0.26; spoken: rho = −0.005; P = 0.61; difference: z = 0.71; P = 0.47) or left lingual gyrus (written: rho = 0.01; P = 0.11; spoken: rho = 0.02; P = 0.03; difference: z = −0.59; P = 0.74).

Table 2

Results from the region-of-interest-based Representation Similarity Analysis. The first four regions were included based on the observed interaction effect between task and stimulus type. Left inferior frontal gyrus and left posterior superior temporal sulcus were included based on the whole-brain RSA. We report Spearman correlations between the neural similarities and the association-based Small World of Words semantic similarities, the valence similarities and concreteness similarities for both tasks separately. The reported P values were obtained from 100 000 random permutations, with Bonferonni-corrected P < 0.008 for the number of regions (n = 6). 95% bootstrap confidence intervals are reported between square brackets. The reported cluster size is the average number of voxels ± standard deviation over subjects after overlaying the subject-specific gray matter masks

		Left lingual gyrus	Right posterior fusiform gyrus	Bilateral precuneus	Left insula	Left inferior frontal gyrus (BA44-BA45)	Left posterior superior temporal sulcus
Semantic  similarity	Extent (in voxels)	154.9 ± 15	301.2 ± 4	216.9 ± 14	61.1 ± 7	353.2 ± 24	541.6 ± 18
	Concreteness probe	0.01 [−0.01, 0.03] p = 0.18	−0.004 [−0.03, 0.02] p = 0.64	−0.01 [−0.04, 0.01] p = 0.89	0.0007 [−0.02, 0.02] p = 0.48	0.04 [0.02,0.7] p = 0.0001	0.05 [0.03,0.07] p < 0.00001
	Valence probe	0.004 [−0.02,0.02] p = 0.37	0.01 [−0.01, 0.03] p = 0.20	−0.02 [−0.04, 0.01] p = 0.93	−0.003 [−0.2, 0.02] p = 0.60	0.03 [0.01,0.06] p = 0.002	0.02 [−0.001,0.05] p = 0.03
	Pooled	0.0008 [−0.02, 0.02] p = 0.47	−0.02 [−0.04, 0.005] p = 0.93	0.02 [−0.005, 0.04] p = 0.06	−0.005 [−0.02, 0.01] p = 0.65	0.09 [0.06, 0.11] p < 0.00001	0.07 [0.05, 0.09] p < 0.00001
Valence  similarity	Concreteness probe	−0.007 [−0.03, 0.02] p = 0.72	−0.005 [−0.03, 0.02] p = 0.66	−0.02 [−0.04,0.002] p = 0.96	−0.004 [−0.02, 0.01] p = 0.64	0.003 [−0.02,0.02] p = 0.41	0.005 [−0.01,0.03] p = 0.32
	Valence probe	0.04 [0.02, 0.06] p = 0.0002	0.03 [0.01, 0.05] p = 0.006	−0.01 [−0.04, 0.01] p = 0.85	0.01 [−0.01, 0.03] p = 0.16	0.03 [0.01, 0.05] p = 0.006	−0.02 [−0.04,0.002] p = 0.96
	Pooled	0.03 [0.005, 0.05] p = 0.007	0.02 [−0.007, 0.04] p = 0.08	−0.02 [−0.04, 0.004] p = 0.94	0.02 [−0.01, 0.03] p = 0.09	0.04 [0.01, 0.06] p = 0.001	−0.006 [−0.02, 0.02] p = 0.71
Concreteness  similarity	Concreteness probe	0.02 [−0.01, 0.04] p = 0.1	0.01 [−0.02, 0.03] p = 0.25	0.02 [−0.001, 0.05] p = 0.03	0.01 [−0.01, 0.03] p = 0.23	0.07 [0.04,0.08] p < 0.00001	0.04 [0.01,0.06] p = 0.001
	Valence probe	−0.02 [−0.04, 0.005] p = 0.93	−0.01 [−0.03, 0.02] p = 0.75	−0.007 [−0.03, 0.1] p = 0.73	0.01 [−0.01, 0.3] p = 0.18	0.06 [0.03, 0.08] p < 0.00001	0.05 [0.03,0.07] p = 0.00001
	Pooled	0.003 [−0.02, 0.02] p = 0.41	−0.01 [−0.03, 0.01] p = 0.83	0.007 [−0.01, 0.03] p = 0.28	0.009 [0.01, 0.03] p = 0.22	0.13 [0.11, 0.15] p < 0.00001	0.08 [0.05, 0.10] p < 0.0001

The correlation between concreteness similarities and the neural similarities in bilateral precuneus during concreteness judgments did not survive correction for multiple comparisons (rho = 0.02, P = 0.03). We did not find significant correlations in any of the other regions with the two models included in this analysis (Table 2).

To obtain a more comprehensive view of the functional-anatomical context of the effects described, we also ran a whole-brain RSA. A searchlight analysis was performed across tasks with concreteness similarities, valence similarities, and semantic similarities. This yielded significant results for concreteness similarities in left inferior frontal gyrus (peak coordinate −46, 32, 0; cluster size = 567; FWE-corrected cluster-level P = 0.006) and left posterior superior temporal sulcus (peak coordinate −58, −38, 6; cluster size = 649; FWE-corrected cluster-level P = 0.002; Fig. 7). No significant effects were observed for the valence similarities and the semantic similarities in the whole-brain searchlight RSA. To compare between tasks, we compared task-specific whole-brain RSA results for the valence and concreteness similarity matrices using a paired t-test. Correlations between neural similarities and valence similarities were significantly stronger during valence judgments than during concreteness judgments in bilateral lingual gyrus (Fig. 8; −10, −78, −6; 8, −80, −6; cluster size = 1242; FWE-corrected P = 0.00003). No significant between-task effects were observed for the concreteness similarity matrix.

Figure 7

A) Results of searchlight Representational Similarity Analysis for concreteness similarities, pooled over tasks. Significance was set at cluster-level FWE-corrected P < 0.05 and uncorrected voxel-level P < 0.001. These regions were further investigated to evaluate task-dependent changes in representations. B-C) Results of the regions-of-interest-based Representation Similarity Analysis in posterior superior temporal sulcus and inferior frontal gyrus. Histograms represent the 100 000 random correlation. Black lines indicate the 95th percentile and red lines the observed correlation. The Bonferroni-corrected threshold for this analysis was P < 0.008 (uncorrected P < 0.05). No significant differences in representational similarity were observed between tasks.

Figure 8

Results of paired t-test between the whole-brain RSA correlations maps with the valence matrix during valence and concreteness judgments. In the highlighted region, the correlation between neural and valence similarities was higher during valence judgments than during concreteness judgments. Significance was set at cluster-level FWE-corrected P < 0.05 and uncorrected voxel-level P < 0.001.

When repeating the analysis for written and spoken trials separately, the cluster in the left posterior superior temporal sulcus remained for spoken, but not written words (peak coordinate −64, −34, 2; cluster size = 411; FWE-corrected cluster-level P = 0.03). None of the effects depended on task (Fig. 7). The Spearman correlations, P values, and confidence intervals for these regions are listed in Table 2.

Discussion

We investigated how attention to two fundamental dimensions of word meaning, concreteness and valence, modulates activity patterns in the semantic brain network. A priori we hypothesized that the correlation between pairwise similarity in activity patterns evoked by words and pairwise similarity between the word concreteness and valence would be altered by orienting of attention to either concreteness or valence. This hypothesis was partially confirmed: when subjects performed a valence judgment task, the correlation of activity patterns in ventral occipitotemporal cortex with the valence matrix were stronger than during a concreteness judgment task. These effects were located relatively posterior and outside the core perisylvian language network.

In accordance with our hypothesis, valence judgments activated (para)limbic and semantic regions (e.g., lateral temporal cortex, BA37), confirming that the evaluation of affect in linguistic stimuli is closely linked to the language system (Meersmans et al. 2020). The (para)limbic system has been associated with the processing of emotional language (Kuchinke et al. 2005; Lewis et al. 2007; Citron 2012; Schlochtermeier et al. 2013; Lindquist et al. 2016). The increase in activity of the default mode network (DMN; e.g., precuneus, posterior cingulate cortex, dorsomedial frontal cortex) should be interpreted as smaller deactivation for valence judgments than for concreteness judgments (Greicius et al. 2003; Mineroff et al. 2018; Jackson et al. 2019). As such, the effect in the DMN is related to a difference in cognitive demand between tasks, rather than to the explicit evaluation of valence (Supplementary Figure 1). Univariate effects of concreteness judgment were mainly related to the multiple demand network (Duncan 2010; Assem et al. 2020). This is in line with behavioral evidence that concreteness judgments require more extensive evaluation and integration than judging valence, which is processed fairly automatically (Citron 2012; Pauligk et al. 2019). This was also reflected in the current experiment by longer reaction times for concreteness judgments. The multiple demand network is involved in domain-general, goal-directed organization of cognitive processes.

Univariate analysis revealed interaction effects between task and concreteness in multiple regions: precuneus, insula, and posterior fusiform gyrus. An interaction between task and valence was observed in the lingual gyrus. Only the left lingual gyrus and right posterior fusiform gyrus showed a task-dependent interaction as well as representational similarity effects, suggesting that under the right circumstances semantic information permeates into the early visual processing stream. In these two posterior regions, pairwise distance in word valence correlated with similarity in activity patterns during the valence judgment task but not during the concreteness judgment task. There was no analogous effect of word concreteness during concreteness judgment in these regions. The predominant effect of valence rather than concreteness may relate to the fact that valence has more weight within the semantic association network than concreteness. This was also evidenced by the stronger correlation of the valence matrix with the semantic similarity matrix than that of the concreteness matrix.

One could easily imagine that orienting attention to visual features of a referent may alter representations of the meaning of that word in visual cortex, however the effect of orienting to a fundamental and generic dimension such as valence in occipitotemporal cortex was surprising. Other studies have implicated the lingual gyri in the processing of affective information (Nielen et al. 2009; Tettamanti et al. 2012; Kehoe et al. 2013; Schlochtermeier et al. 2013). Most studies focussed on pictorial stimuli, with Schlochtermeier et al. (2013) as a notable exception (using both pictures and words). Functional connectivity between the amygdala and the lingual gyri during threat word processing (Weisholtz et al. 2015) is supported anatomically by a direct pathway known as the inferior longitudinal fasciculus (Catani et al. 2003; Latini 2015). This white matter tract runs from occipital regions to anterior and medial temporal regions including the amygdala. It functions as a bidirectional route and can project affective information from the amygdala back to the occipital cortex (Catani et al. 2003). For instance, reduced influence of valence in fusiform cortex during emotion face processing has been described in patients with lesions in the amygdala (Vuilleumier et al. 2004; De Winter et al. 2016). Vuilleumier and Driver (2007) discuss attentional and emotional modulation of visual processing in ventral occipitotemporal cortex, asking whether these are two distinct processes or whether they rely on the same mechanism (i.e., emotional stimuli attracting more attention). The task-dependency of the word valence effect suggests the latter: only when the valence dimension is attended, do BOLD responses between positive and negative words differ, can neural patterns between positive and negative words be discriminated, and do representations of valence similarities become stronger. Without this explicit attention, the neural patterns did not contain sufficient information on valence to be detected. In sum, despite its location at the earlier stages of the ventral processing stream, valence information can be detected from verbal stimuli in the lingual gyrus, when this is required by contextual demands. These results suggest that task demands modulate the representational space so that words with similar valence are closer together (Nosofsky 1986).

The effect of written word valence in relatively early occipital processing regions is in line with electrophysiological studies: positive and negative valence words elicit an early posterior negativity (EPN, around 200 ms following stimulus onset) and a late positive potential (LPP, around 300–400 ms) compared to neutral words (Schacht and Sommer 2009a). The source of the EPN and LPP lies in inferior occipital gyrus and fusiform gyrus, respectively, near or identical to where the current effects are found (Schindler and Kissler 2016). The timing of the EPN strongly suggests that word valence becomes available after processing of the surface features and that this is an effect of word valence per se, or of word saliency rather than an endogenous orienting effect. The timing of the EPN provides strong evidence that word valence is processed already at an early lexical stage simultaneously with or immediately after word recognition (Kissler et al. 2007; Scott et al. 2009) and before a more elaborate activation of word meaning.

Two differences with the electrophysiological effects are worth noting. First, the EPN is known to be relatively task-independent, whereas the LPP is more task-dependent (Schacht and Sommer 2009b; Scott et al. 2009). Similar effects have been reported for nonverbal emotional stimuli (Schupp et al. 2007). The ventral occipital and fusiform effects of valence distance in our study are clearly task-dependent. Hence, the task-dependency in the current data may arise from feedback signals, and the timing of the task-effect is likely later than the timing of the EPN. Second, the early effect of word valence on occipital cortex has been consistently documented for written words as a modulation of the visually evoked response. In the current study, the effect was seen when responses to auditory and written words were pooled. Subset analyses of only auditory or only written words did not yield significant effects and no significant difference was observed between written and spoken words. Hence the current data do not allow to draw firm conclusions about modality-specificity. For instance, the consonant letter strings may provoke a visual response that is modulated by the auditory word valence through connectivity between the auditory and the visual processing stream underlying multimodal integration, but this would require further investigation.

In the right posterior fusiform region, the response to concrete words increases more during concreteness judgment compared to a valence judgment. This can be accounted for by the role of right fusiform cortex in retrieval of visual knowledge in response to verbal input (Vandenbulcke et al. 2006). Attentional effects have been commonly observed in univariate analyses in functionally specialized visual cortex when attention is directed towards specific features processed in a given area. The univariate right posterior fusiform effect may reflect the explicit retrieval of the visual features of the referent of the concrete word during a concreteness judgment. The univariate effect was not associated with an RSA effect of concreteness distance: the cosine similarity between activity patterns is by definition independent of overall differences in response amplitude. Hence, mathematically a univariate effect due to an overall average increase is perfectly reconcilable with the absence of a multivariate effect as measured using cosine similarity.

For valence a different pattern was seen in the right posterior fusiform region than for concreteness: no interaction in the univariate analysis but a significant task effect in the RSA analysis. The mechanism behind the RSA effect of valence in the right posterior fusiform gyrus is likely to be different from the mechanism behind the univariate effect of concreteness. The posterior fusiform gyrus’ role in valence processing may be more restricted e.g., limited to unidimensional valence tagging. The task-dependent RSA effect of valence may arise from a change in tuning curves or other changes in the properties of the representation of word valence that are beneficial for making fine word valence distinctions. Thus, the univariate and multivariate results in right posterior fusiform gyrus reflect two distinct functions: a role in retrieval of visual knowledge in the evaluation of concreteness and a role in valence tagging of words, each based on different underlying mechanisms.

So why, to the left, is there a congruency between the univariate effect of valence and the multivariate RSA effect during valence judgment? In the ERP literature the word valence effect is left-sided during subliminal processing and becomes bilateral during supraliminal processing (Bernat et al. 2001). This already indicates that the left side has a preferential involvement in word valence processing compared to the right side, which may contribute to the difference in effects we are observing. As of yet, a loss of visual knowledge of entities with unilateral ventral occipitotemporal lesions has been mostly described with right-sided lesions. This may explain the right lateralization of the interaction between task and stimulus concreteness.

Earlier research has successfully demonstrated an effect of task on representational similarity in the visual word form area (Wang et al. 2018) and intraparietal, pericentral, and ventral temporal cortex (Nastase et al. 2017). These studies used tasks that require the categorization of stimuli at different levels (judging thematic vs. taxonomic similarity in Wang et al. and judging animal behavior vs. animal taxonomy in Nastase et al.). To complete our tasks, subjects focussed on a single dimension of word meaning. They also focussed primarily on concrete words (e.g., animals, objects, but also places and people). The ventrotemporal region highlighted in Nastase et al. (2017) is located more anteriorly and laterally of the regions discussed here.

The function of regions where univariate analysis revealed an interaction between task and stimulus type but that did not exhibit a representational similarity, such as the precuneus, remain more open for interpretation. For instance, visual imagery may play a role in precuneus (Kosslyn et al. 1995; Cavanna and Trimble 2006) as it showed strong responses for concreteness judgments of concrete words. For abstract concepts, concreteness judgments could rely more on the generation of a scene, rather than a well-delineated entity, resulting in lower activation in fusiform gyrus.

Inferior frontal gyrus showed an interaction effect between concreteness and valence: valence has opposite effect for abstract and concrete words. The time-activity curve also confirms the stronger activation of this region for abstract words. Pauligk et al. (2019) also report in inverse effect of valence on the processing of abstract and concrete nouns in inferior frontal gyrus. However, their explanation that high/low valence generates additional needs for top-down control in concrete word processing does not hold here. Rather, we observed stronger activation in this region for abstract words, an effect that has been previously reported (Noppeney and Price 2004; Binder et al. 2005; Hoffman et al. 2015; Della Rosa et al. 2018). Therefore, we argue that recruitment of additional resources is necessary to differentiate abstract words, which are known to be more ambiguous (Hoffman et al. 2011), and positive words, which are known to be more similar to one another than negative words (Alves et al. 2017). Combined, this could explain the stronger recruitment of inferior frontal gyrus during abstract, positive word processing. In addition, concreteness judgments were more difficult than valence judgments, and presumably require more top-down control to select the appropriate features (Hoffman et al. 2015).

We observed significant representational similarity effects of concreteness similarities in lateral temporal cortex and inferior frontal gyrus without a significant effect of task on the correlation strength. These regions also showed strong semantic similarity effects that were task-independent. Semantic similarity is by definition multidimensional and it is understandable that directing attention to one single, be it important dimension (valence or concreteness) does not substantially alter this multidimensional representation. This is also why the primary analysis of this study relies on the effect of valence and concreteness judgment on the representation of valence and concreteness distance rather than semantic similarity. We included the analysis of semantic similarity mainly to sketch the broader functional-anatomical context of the valence and concreteness effects we are reporting. The effect of semantic similarity on the posterior lateral temporal neocortex [x = −58, y = −38, z = 6] and on BA44/45 [x = −46, y = 32, z = 0] confirms earlier reports (Liuzzi et al 2017; Meersmans et al. 2020 [x = −57, y = −37, z = 5] and [x = −48, y = 26, z = −1]).

In these perisylvian regions, colocalization was found between a univariate task effect (valence minus concreteness judgment) and a representation of meaning within the same region in the RSA (semantic similarity), without an observable effect of task on the representation. This remarkable pattern was mainly found in the cortex surrounding the left superior temporal sulcus, the posterior middle temporal gyrus, and the inferior parietal cortex. The current findings indicate that semantic representation and control may be operationally dissociable even when they overlap anatomically. Conceivably, in regions where task and representation overlap, the effects of task and of representation may occur at different frequency bands. In any case, these findings in the lateral temporal and inferior parietal cortex do not support a view where semantic representation and semantic control are anatomically segregated. Furthermore, the absence of a task effect on semantic representations in the posterior temporal and inferior frontal cortex may suggest that the representation of meaning in the core language network may possibly be less susceptible to cognitive control than the semantic representations in more peripheral regions.

Study limitations—The absence of a task effect on the semantic representations in the perisylvian language regions can reflect a lack of sensitivity. For instance, the tasks may have been too similar or the effect may be short-lived and beyond the temporal resolution of fMRI. As a further limitation, we did not evaluate task-dependency of semantic representations in each of the regions showing a main effect of task or stimulus, only in those showing an interaction between task and stimulus. The clusters exhibiting main effects were so numerous that we opted to conduct a whole-brain representational similarity with proper correct for multiple comparisons.

In sum, orienting attention to stimulus valence enhances the correlation between pairwise between-word similarity of ventral occipital activity patterns and valence distance. These findings further establish the role of ventral occipital cortex in representing valence of abstract and concrete words in a context-dependent manner.

Funding

KU Leuven (grant C14/17/108) and Research Foundation Flanders (FWO; grant G094418N). RB is a senior postdoctoral fellow of the FWO.

Notes

Conflict of Interest: None declared.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.