The neurocognitive signature of focus alternatives.

Focus alternatives are words/phrases that can substitute for the focused constituent of an utterance. In "Carsten has picked [CHERRIES]F from the tree.", (marked by pitch focus on cherries), the speaker wants to not only convey the fact that Carsten has picked cherries, but also to contrast cherries with other fruit that could have been picked, such as plums. Although focus alternatives are key to understanding the implicit aspects of an utterance, nothing is known about their neural representation. We directly contrasted neural representations of lexico-semantic similarity and focus alternative status using fMRI. Semantic relatedness was reflected in decreased activation in the bilateral superior temporal gyri. By contrast, processing of focus alternatives induced increased activations in the precuneus and the fronto-median wall, two regions previously implicated in discourse processing. These results suggest that focus alternative status is processed separately from semantic relatedness, at the level of discourse integration.

Introduction

Understanding a long stretch of coherent communication requires (among other things) integrating novel information with preceding utterances, with world-knowledge, or with both. The linguistic means to signal new and given information and to guide a listener’s (reader’s) attention are jointly referred to as information structure. In the present paper we are concerned with a particular information structural phenomenon: focus. The following example sentences illustrate the function of focus. Capital letters denote focus accent, and the focused constituent is marked with the subscript F:

The cook picked some fresh [PARSLEY].

[The COOK] picked some fresh parsley.

The two sentences are lexically and syntactically identical. Also, they do not differ in their meaning. However, they do cause listeners to draw different inferences. Example 1) implies that it was parsley and not, for example, basil, that the cook picked. That is, listeners consider “parsley” in the context of other herbs or vegetables. By contrast, Example 2) implies that it was the cook rather than, for example, the gardener, who picked some parsley. Thus, here listeners consider “cook” in the context of other people who could have done the picking. According to a prominent definition, focus indicates the importance of alternatives for the interpretation of an utterance (Krifka, 2008, see also Rooth, 1992). The two example sentences differ in the set of their focus alternatives. This set would be {parsley, basil, cilantro, …} for Example (1) and {cook, gardener, farmer, …} for Example (2). In the last decade, psycholinguistic research has shown that alternatives are activated when a listener (reader) processes a focused expression (e.g., Braun and Tagliapietra, 2010, Gotzner et al., 2016, Fraundorf et al., 2010). Nothing, however, is known about the neural representation of focus alternatives.

In the present study we investigated the neural processes underlying the representation of focus alternatives under two alternative hypotheses that invoke distinct neural correlates: semantic priming and discourse integration.

Words are usually recognized faster if they are preceded by semantically related content (see Neely, 1991, for a review). Focus alternatives often belong to the same superordinate category (animals, occupations, herbs, …) and therefore they might show differential processing in brain areas involved in semantic priming. This prediction is supported by a number of recent electrophysiological studies, which argue that focused constituents undergo a deeper semantic processing than constituents which are not focused (Wang et al., 2009, Wang et al., 2011).

Focus is realised within a single sentence. Consider Example 1) again: The single sentence about the cook picking some fresh [PARSLEY] establishes an alternative set of “things that can be freshly picked and added to a dish”. In fact, in our experiment, we worked with stimuli consisting of single sentences, followed by a potential focus alternative. Still, it is important to bear in mind that focus is in essence a discourse-level phenomenon that provides information about how a given utterance should be integrated with the discourse context. The example sentence “The cook picked some fresh [PARSLEY]” could be the response to the question “What did the cook pick?”, it could be the correction of the assumption “I believe the cook picked some fresh basil.” or it could be the beginning of a narrative sequence that is continued with a phrase like “He didn’t like cilantro in Asian dishes.” Therefore, the alternative set could also be evoked during the course of utterance interpretation, thus involving neural networks that are associated with discourse processing, in particular the evaluation of coherence.

In the following, we will review relevant neuroimaging studies on semantic priming and discourse processing in order to develop anatomical hypotheses about the locus of focus processing and in particular the representation of focus alternative status.

Neuroimaging studies consistently find that semantic priming leads to reduced activation for related as compared to unrelated target words within a mostly left lateralized brain network involved in word and sentence processing (for reviews see Indefrey, 2011, Price, 2012). More specifically, previous studies showed reduced activations for related as compared to unrelated targets in superior temporal gyri and the temporal plane (Kotz et al., 2002, Rissman et al., 2003) and in middle (Copland et al., 2003, Giesbrecht et al., 2004) and inferior temporal gyri (Giesbrecht et al., 2004, Kuperberg et al., 2008), as well as in the left inferior frontal gyrus (primed reading task: Wheatley, Weisberg, Beauchamp, & Martin, 2005; relatedness judgment task: Kuperberg et al., 2008; primed lexical decision: Copland et al., 2003, Giesbrecht et al., 2004, Kotz et al., 2002, Kuperberg et al., 2008, Matsumoto et al., 2008; Rissman et al., 2003), independently of task demands (but see Kotz et al., 2002, Rossell et al., 2001, Kuperberg et al., 2008). Similar effects were also observed for semantic priming in the context of written or spoken sentences, where priming does not occur through associative links between two words, but through the expectancy built up by a sentence (Baumgaertner et al., 2002, Hartwigsen et al., 2017). In addition to the left lateralized areas listed above, Hartwigsen et al. also observed activation in the pre-supplementary motor area, the right superior parietal lobule, and the right intraparietal sulcus.

Studies of discourse-level processing have found quite a different pattern of neural activations. Ferstl and von Cramon (2001) presented pairs of written sentences to participants, that could either be coherent or not. Coherent sentences induced stronger activation than incoherent sentences in left superior and medial frontal areas, the left posterior cingulate cortex and the left inferior precuneus (see Ferstl & von Cramon, 2002, and Martín-Loeches, Casado, Hernández-Tamames, & Álvarez-Linera, 2008, for similar findings). A meta-analysis of the neuroimaging of story comprehension reported activations in temporal regions in addition to ventro- and dorsomedial prefrontal cortex and left precuneus for contrasts targeting coherence processing (Ferstl, Neumann, Bogler, & von Cramon, 2008). Xu, Kemeny, Park, Frattali, and Braun (2005) compared the processing of single words, sentences, and narratives and found that, while the perisylvian language network was active in all tasks, the activation of precuneus, medial prefrontal and dorsal temporo-parietal-occipital cortices was indeed characteristic for the comprehension of narratives.

In summary, while semantic relatedness most typically invoked reduced activation in temporal and inferior frontal areas, discourse-level processing lead to increased brain activity in parietal and prefrontal regions. Thus, if processing and representation of focus alternative status is part of semantic processing, we expect to see reduced temporal and prefrontal activations for alternatives as compared to non-alternatives. If, however, processing and representation of focus alternatives happens as part of discourse-level processing, we expect increased activations for focus alternatives in medial frontal and parietal areas, most notably the precuneus.

To test these two hypotheses, we created spoken sentences like Example (3):

(3) Carsten hat Kirschen vom Baum gepflückt.

(Carsten has picked cherries from the tree).

Each sentence contained a name (Carsten), an object (Kirschen = cherries) and an additional noun phrase (vom Baum = from the tree), whereby either the name or the object carried focus accent. The spoken sentence was followed by a written probe word on the screen. Participants had to decide whether the probe had occurred in the sentence. The full design contained three critical conditions that are exemplified in Table 1. In critical trials, the probe was a semantically related alternative (rel condition), semantically related but not a possible alternative (rel condition), or unrelated and not an alternative (unr condition) to the focused element. The two related probes (rel and rel) were both related to the sentence object. However, in the rel condition, the object of the prime sentence was focused, making the probe a focus alternative. By contrast, in the rel condition, the subject of the prime sentence was focused, and therefore, the probe was not a focus alternative in the given context. Crucially, the probe words were physically identical in all experimental conditions (cf. Table 1); the difference was in their relation to the content of the preceding sentence. Thus, we probed the neural activation induced by focus alternatives after processing of a focused element by presenting one potential alternative. This design choice is ecologically valid as focus frequently increases the likelihood of encountering an alternative in the following discourse (see, for example, Spalek & Zeldes, 2017, for corpus data demonstrating this).

Table 2

Properties of prime and target words (standard deviations in brackets).

Measure	Prime	Target	p-value for comparison
Length: number of letters	6.49 (1.42)	6.29 (1.29)	0.28
Length: number of syllables	2.15 (1.29)	2.17 (0.58)	0.81
Frequency: type, normalized, log10	0.33 (0.68)	0.34 (0.73)	0.95
Frequency: lemma, normalized, log10	0.68 (0.65)	0.70 (0.63)	0.85
Cumulative bigram frequency, normalized, log10	5.81 (0.22)	5.75 (0.22)	0.08
Levenshtein distance, cumulative frequency, normalized, log10	1.02 (0.94)	0.94 (0.93)	0.69

In the experimental trials, the probe words were always different from the prime words. However, in a portion of the filler items, either the subject, the object or the prepositional object (Baum/tree in Example 3) was repeated (see procedure for Experiment 1 and Experiment 2).

We carried out a behavioral pilot study to ensure that our stimuli and design would reveal processing differences between focus alternatives and non-alternatives. In this task, participants had to decide whether the visual probe word had been presented in the auditory sentence or not. We expected to see longer reaction times for related words than for unrelated words. This is because the related words fit the general content of the sentence whereas the unrelated words did not. Gotzner et al. (2016) observed this pattern in a probe recognition task, and it is also well-known in the literature on recognition memory in list learning (Roediger & McDermott, 1995).

Additionally, we expected to see faster rejections of related targets when they were part of the focus alternative set. Fraundorf et al. (2010) had shown that focus markings improve participants’ recognition memory when they had to judge if a particular sentence was true or not, given what they had learned a day before. Performance improved both for accepting sentences with correct target words and for rejecting sentences containing an alternative to the correct word (see also Sturt, Sanford, Stewart, & Dawydiak, 2004, for similar findings). We expected this increased sensitivity for focus alternatives to translate into faster reaction times in our study.

Experiment 1: Behavioral pre-study

Methods

Participants

Thirty native German participants were recruited via the participant pool of the Berlin School of Mind and Brain. Participants were students aged 18–30 years. Participants were paid 6€ for their participation in the experiment.

Materials

Spoken (prime) sentences

From an original pool of 90 items of the type described in Table 1, 72 items were chosen as experimental primes. The items were recorded by a female native German speaker who was a trained phonetician.

Sentences with subject (SF) and object (OF) focus were acoustically and phonetically distinct. In particular they had distinct pitch profiles, with significantly higher pitch (F0) over the focused element (Fig. 1, subject noun phrase, SF: mean = 379, OF: mean = 218, t = 45.31, p < 0.001; object noun phrase, SF: mean = 204, OF: mean = 377, t = −45.16, p < 0.001).2

Fig. 1

Pitch (F0) contours for an example sentence (Carsten picked cherries from the tree) spoken with focus on the sentence subject (grey) and focus on the sentence object (black). The inset graphs show the mean F0 contour of the accented syllable averaged across all utterances. The critical syllable was split into five equal parts and the mean pitch excursion extracted for each part.

(Target) words

The spoken prime sentences were followed by a written target word that was related to the sentence object. Unrelated trials were constructed by re-combining prime sentences and target words. The critical word (object) in the spoken prime sentence and the target word (written probe) were matched on length, frequency, bigram frequency, and lexical neighbourhood (see Table 2).

Table 3

Summary of linear-mixed effect regression for log-transformed reaction times.

Predictor	Estimate	S.E.	t-value	p-value
Intercept	2.8860	0.0194	148.92	<0.001
Contrast 1 (relalt + relno_alt vs. unr)	−0.0045	0.0015	−2.97	<0.01
Contrast 2 (relalt vs. relno_alt)	0.0065	0.0028	2.33	<0.05
Jitter	−0.0079	0.0031	−2.55	<0.01
Trial (centered)	−0.0003	0.0001	−7.56	<0.001

In two online pilot studies (n = 120 and n = 100) in the written modality we ascertained that (1) prime sentences were judged as equally grammatical and equally meaningful with the original sentence objects and with the target words in place of the original sentence objects (grammaticality ratings: with prime (original): mean 4.72, with target: mean 4.7, t(71) = −0.79, p > 0.20; meaning ratings: mean 4.73 vs. 4.72, t(71) = 0.43, p > 0.50) and (2) related word pairs were indeed perceived as related (mean = 3.74, SD = 0.42), whereas unrelated word pairs were not (mean = 1.35, SD = 0.22; t(71) = −42.88, p < 0.0001). The pair with the lowest rating for related pairs was still rated higher than the pair with the highest rating for unrelated pairs (2.57 vs. 1.96), allowing for a clear separation of the conditions.

Design

The 72 target words were evenly split across the three experimental conditions, resulting in three stimulus lists that were pseudo-randomly assigned to participants. In each condition 24 target words were presented. Items in the rel condition were necessarily preceded by spoken sentences with object focus, and items in the rel condition by spoken sentences with subject focus. Target words in the unr condition were preceded by object focus sentences in half of the cases and by subject focus sentences in the other half of the cases (see Table 1). A list of all critical sentences and their combinations with the target words is provided in the Appendix.

The remaining 18 items from the originally recorded item set were used as fillers. These items were presented twice and additionally, 12 of the experimental items were randomly chosen to be repeated as filler items, resulting in 48 fillers (see Table 1 for examples of filler items). While the 72 experimental items necessarily required a “no”-response (cf. procedure), eight fillers required a “no”-response and 40 a “yes”-response, resulting in 80 “no”- and 40 “yes”-responses overall. Twenty-four fillers presented the mentioned object (e.g. row 4, Table 1), eight presented the mentioned subject (proper name, e.g. row 6, Table 1) and eight presented a mentioned place (e.g., row 8, Table 1), while eight (twice four) presented an unmentioned name or place (e.g., rows 5 and 7, Table 1). In this design, object related probe words occur more often than probe words related to the other part of the sentence. This might cause participants to always double-check a probe word against the object before checking it against the entire sentence. In addition, since 2/3 of these require a no-response and only 1/3 requires a yes-response, participants might strategically respond “no” if a target has something to do with the object (i.e., if it is neither a name nor a place). Note that such a strategy would actually work against our hypotheses since increased attention on the object due to a bias in the design would decrease any differences between the conditions based on the focus manipulation in the prime sentences.

Previous behavioral studies had used a fixed and relatively short interval between prime sentence and target word. By contrast, we needed to employ variable jitters in the neuroimaging study in order to be able to computationally separate the hemodynamic response to the prime sentence from the hemodynamic response to the target word. To test whether jitters of different length between the prime sentences and the target words affected the behavioral effect of focus, the original three experimental lists were doubled, and half of the items were assigned a time interval of 6 sec between sentence and word (the median jitter used in the fMRI study) and the other half were assigned a time interval of 10 sec (the longest jitter used in the fMRI study). In the remaining three lists, this assignment was reversed.

For each participant, a novel pseudo-randomised version of the stimuli was created, using the free software Mix (Van Casteren & Davis, 2006) with the constraints that (1) a given condition for the target word (rel, rel, unr, filler) was only presented once in a row; (2) no more than three successive prime sentences carried the same focus accent and (3) there was a minimum distance of four between items from the same semantic category.3

Procedure

Participants signed an informed consent form and an information form about data protection. They were then instructed on the screen that they would listen to a sentence via headphones and that this sentence was followed by a word. Their task was to press one of two buttons, depending on whether or not the word had occurred in the previous sentence. Speed and accuracy were likewise emphasized, and participants received feedback if they replied incorrectly and if they did not reply within 5 s.

A fixation cross was presented in the center of the screen, while the auditory sentence was played. 6 sec or 10 sec after the sentence, the word appeared on the screen and stayed for 4000 ms or until the participant responded. The entire session lasted about 60 min.

Data analysis

Participants made only 35 errors overall (1% of all observations). 23 of these errors concerned catch trials (1.6% of all catch trials) and 12 concerned experimental trials (0.6% of all experimental trials), pointing towards a bias to respond with “no”, given the 1/3 vs. 2/3 distribution of “yes” and “no” responses in the experiment.

Reaction times were analysed for experimental trials only. Error trials were removed from the analysis. Observations deviating more than 2 standard deviations from a participant’s condition mean were removed as outliers (n = 123, i.e., 5.7% of the original sample).

The base 10 log-transformed reaction times were analysed with linear mixed effects models with the software R using the package lmerTest (Kuznetsova, Brockhoff, & Christensen, 2016). The factor condition was Helmert-coded such that the first contrast tested for the effect of relatedness (rel + rel vs. unr) and the second contrast compared alternatives with non-alternatives (rel vs. rel). Random intercepts for subjects and probe words were modelled as well as random slopes for condition for subjects and probe words. We started with a maximal model including the fixed factors condition, jitter, centered trial (i.e., the position of an item in the experimental list) and all interactions. Non-significant interactions were removed progressively if they did not improve model fit. The final model only included the main effects for condition, jitter, and centered trial. Fifteen additional data points were excluded because their residuals were outliers (c. f. Baayen, 2008).

Results and discussion

The behavioral data showed that our material and set-up are well suited to reveal differential results of the focus alternative status: Participants reacted to targets in the unr condition on average in 785 ms (sd = 254). Reactions to targets in the rel condition were almost as fast (mean = 794 ms, sd = 243). By contrast, it took participants considerably longer to react to targets in the rel condition (mean = 827 ms, sd = 276). The results of the LME analysis are presented in Table 3. There was an effect of relatedness (p < 0.01) such that is was more difficult to reject probe words as not having occurred in the sentence when they were semantically related to the sentence object. This is immediately understandable: When a sentence deals with fruit, it is easier to decide that cows had not been mentioned than that peaches had not been mentioned. However, the effect was qualified by the difference between the rel and the rel condition (p < 0.05): The inhibitory effect of semantic relatedness was only present for words that were not focus alternatives for the given sentence. A focus alternative, by contrast, was as easy to reject as an unrelated probe word, b = −0.003, s.e. = 0.003, t = −1.09, p = 0.28 (additional analysis with only rel and unr conditions but same fixed- and random-effects structure). Jitter, that is the time interval between the sentence and the word, affected reaction times such that participants reacted faster after a longer time interval (p < 0.01). However, modelling the interaction of jitter and condition did not improve model fit. Thus, while jitter had an overall effect on reaction times, it did not affect the theoretically important contrasts. Finally, trial also had an effect – participants became faster in the cause of the experiment (p < 0.001). The power of the statistical design was analysed post-hoc with the R-package simr (Green & MacLeod, 2016). With 81% it was above the recommended ideal power of 80%.

Table 4

Regions showing significant activation for the contrasts tested.

Contrast	Cluster pfwe-c	Cluster size	Anatomic localisation	t	Peak MNI coordinates
					x	y	z
Unrelated > related	0.001	129	Right STG/Insula/Precentral Gyrus (BA 13, 22)	6.08	45	−19	5
	<0.001	155	Left STG/Insula/Transverse TG (BA 13, 41, 22)	5.95	−48	−16	5
Alternative > no_alternative	0.009	83	Right postcentral/precentral gyrus (BA 3/4)	5.24	18	−34	77
	0.020	70	Left/right precuneus, cingulate gyrus (BA 31, 7)	4.31	6	−58	26
	0.041	58	Medial frontal gyrus/bilateral superior frontal gyrus (BA 10)	4.51	−6	59	8

Note: No differences were found between neural responses to sentences with object and subject focus.

The results of the behavioral experiment argue against a semantic locus of the focus alternative effect. In case of a semantic locus, we would have expected reactions to focus alternatives (rel) to be slowed down as much as (or even more than) the reactions to “merely” related words (rel), compared to the unrelated condition. Instead, rejecting focus alternatives was as easy for our participants as rejecting clearly unrelated probe words. This is consistent with change detection and memory studies, which showed that it is easier to detect deviations from a focused item than from a non-focused item (e.g. Sturt et al., 2004, and Fraundorf et al., 2010, Fraundorf et al., 2013). The authors of these studies argue that focus acts upon the same semantic co-activation processes as semantic priming, but that focus has the additional effect of casting the meaning of a word into sharper relief, contrasting it more strongly against other semantically related items. By contrast, semantic relatedness effects in the absence of focus are based on unspecific activation spread to items sharing features and/or a category with the target item. However, as reaction times reflect only the end result of a number of different processes, these data cannot distinguish whether semantic relatedness and focus act upon the same or distinct neural processes. Thus, in Experiment 2 we aimed to disentangle the contributions of semantic priming and discourse processing to processing of focus alternatives using functional neuroimaging (fMRI).

Experiment 2: Neural correlates of focus alternative processing

Materials and methods

Twenty-eight volunteers aged 18–30 (20 women) were recruited from the Berlin universities. All volunteers were native speakers of German, right-handed, and reported no psychiatric or neurological diseases as assessed with the screening questionnaire of the Center for Cognitive Neuroscience Berlin before the testing session. All participants had normal or corrected to normal vision and reported no hearing difficulties. Informed consent was obtained from participants prior to scanning. The study was approved by the ethics committee of the German Linguistic Society (DGfS) in accordance with the declaration of Helsinki. One participant had to be excluded because, due to a hardware error, she did not hear the prime sentences. One additional participant was excluded due to excessive motion in the scanner.

The same 72 critical sentences as the ones in Experiment 1 were chosen as critical items. Eight additional sentences were used for filler trials. These sentences were repeated twice. A random sample of eight experimental items was also repeated for the filler trials. The critical sentences were combined with probe words as illustrated in Table 1. The filler sentences were paired with eight object probes, eight name probes and eight place probes. All object probes were identical to the sentence object, four names were identical to the sentence name and four were not, and the same was true for the places. Thus, a total of 80 probe words had not occurred in the sentences and 16 probe words (i.e., 1/6) had occurred in the sentences.

Stimuli were presented with the Presentation software (Neurobehavioral Systems). A fixation cross was presented in the center of the screen, while the auditory sentence was played. Silent pauses were inserted at the end of the sound files such that their entire duration was always 3000 ms (the actual length of the sentences ranged from 1652 to 2910 ms). After a variable jitter of 2–10 sec the word appeared on the screen for 4000 ms. Participants had to press a button if the word had appeared in the sentence. After an inter-trial interval of 4000 ms, the next trial started. Fig. 2 illustrates the time course of one experimental trial. The overall scanning time was about 60 min.

Fig. 2

Schematic of a single trial in the fMRI experiment. Prime sentences were presented auditorily, whereas the target word was presented visually.

To control for the distribution of items across conditions, 9 different lists from the behavioral experiment were chosen. Each item appeared once in each list, but across all nine lists, items appeared equally often in each of the experimental conditions. Each list was used for 3 participants, but with a different trial and jitter order for each participant. Stimuli were presented in 6 runs of 16 trials each. Each run contained 4 trials per condition, whereby 1–3 of the fillers were catch trials that required a button press. Jitter durations between 1 and 5 TRs (2–10 s) and condition order were optimized using optseq2 (Dale, 1999; ). Three optimized orders of conditions and jitters for a single experimental run were created, so that each order was used twice during the experiment for each participant. These three orders were permuted within each group of three participants that received the same list (orders were 123123, 312312, 231231), such that no two participants received the exact same experimental order.

Prior to scanning, participants completed a practice set of 10 trials containing examples of all conditions and jitters. The scanning session started with the anatomical scan. During the shimming period of the first EPI scan, participants heard a number of sample sentences (from the fillers only) and told the experimenter if the audio volume had to be adjusted.

Participants were instructed to listen to the sentences and read the words carefully and to press a button if the word had occurred in the sentence. Response devices were the keyboard for the practice experiment and an MR-scanner compatible response box for the actual experiment. Stimuli were presented to both ears through headphones.

Image acquisition

MR data were acquired with a 3 T Siemens Magnetom TimTrio MR scanner. The scanning session started with the acquisition of a high-resolution T1-weighted structural image (MPRAGE, TE = 2.52 ms, TR = 1900 ms, 256 mm FOV, 256 × 256 mm matrix, 1 mm3 isotropic resolution). The acquisition of the structural image was followed by six functional runs of 5.68 min length each (EPI, 37 slices, TE = 30 ms, TR = 2000 ms, 70° flip angle, 192 mm FOV, 64 × 64 mm matrix, voxel size 3 × 3 × 3 mm, 3 mm slice thickness, ascending slice acquisition).

fMRI data analysis

Data preprocessing

The data were preprocessed and analysed with SPM12 (Wellcome Trust Centre for Neuroimaging, University College, London, UK). Preprocessing steps were applied in the following order: realignment, slice time correction, coregistration to mean image, segmentation, normalisation, and smoothing. Realignment was based on the 2-pass procedures available in SPM12 using six-parameter rigid body transformation. Subsequently, a participant’s anatomical image was co-registered to the mean functional image. Coregistration was based on the normalized mutual information algorithm provided in SPM12. Gray and white matter were segmented, and segmentation results were used for normalization of functional images to SPM12′s standard T1 template based on the Montreal Neurological Institute (MNI) reference brain with isotropic 3-mm voxels. For the univariate analyses, normalized images were smoothed with an isotropic 8-mm FWHM Gaussian kernel.

Single participant analysis

For each participant, fMRI time series were regressed onto a general linear model (GLM). The two auditory prime sentence types (subject focus and object focus) were modelled with a boxcar function of the duration of the individual sentences (ranging from 1652 ms to 2910 ms). The visual target words were dummy-coded by condition (rel, rel, unr, filler) or as an error, if participants had either missed a catch trial or made a false alarm. The BOLD response to correct visual targets was modelled with a stick-function of 0 duration. All events were convolved with the standard hemodynamic response function, and high-pass filtered at 128 sec.

Additionally, to account for variance induced by participants’ movement, the 6 realignment parameters estimated during preprocessing were added as nuisance regressors. This analysis results in whole-brain beta-maps, estimated for each participant and for each predictor variable.

Univariate group analysis

Statistical inference at the group level was performed with paired sample t-tests between conditions of interest. All reported results were significant after setting the uncorrected voxel-wise threshold at p < 0.001 and the whole-brain family-wise error (FWE)-corrected threshold at p < 0.05 at the cluster level. All univariate analyses were performed with SPM12. Anatomical labels are based on the AAL atlas (built-in the WFU toolbox for MATLAB, Maldjian et al., 2003, Maldijan et al., 2004, Tzourio-Mazoyer et al., 2002) and refer to the location of peak voxels.

Behavioral results

1.3% of all trials were responded to erroneously. Participants made between 0 and 4 errors on 96 trials. Of these, 0–3 were misses on the 16 catch-trials, with an overall miss-rate of 5%. The false alarm rate was 0.57%. Due to the low number of false alarms, the distribution across conditions was not analysed (note that misses could only occur on filler trials).

Neuroimaging results

We tested two orthogonal, a priori defined contrasts on the data. Semantic priming was tested by comparing both related conditions to the unrelated condition. The focus alternative status was tested by contrasting neural activation to alternative (rel) and non-alternative (rel) target words.

Relatedness

Activation was reduced for related target words relative to unrelated targets in bilateral superior temporal lobes (Table 4 and Fig. 3). In contrast, and in line with previous reports, no brain region was more active for related than for unrelated target words. This pattern replicates well-reported findings of reduced neural activation for semantically related prime words, which localized semantic access to the bilateral temporal lobes (Copland et al., 2003, Giesbrecht et al., 2004, Kotz et al., 2002, Rissman et al., 2003).

Table 1

Stimulus examples for the different experimental and filler conditions.

	Critical items
	Condition	Spoken sentence	Target word	Response
1	RELALT	Carsten has picked [CHERRIES]F from the tree	PEACHES	no/NULLa
2	RELNO_ALT	[CARSTEN]F has picked cherries from the tree	PEACHES	no/NULLa
3	UNRb	Thorsten has fed [the GOATS]F on the farm	PEACHES	no/NULLa
	Filler itemsb
4	Filler	[KATHARINA]F has put cucumbers in the trolley	CUCUMBERS	yes/yes
5	Filler	Armin has planted [CROCUSSES]F in his flower bed	ANDREAS	no/NULLa
6	Filler	[NADINE]F has found bugs in her bed	NADINE	yes/yes
7	Filler	Dirk has snacked [POPCORN]F in the cinema	THEATRE	no/NULLa
8	Filler	[MARIA]F has washed the pans in the sink	SINK	yes/yes

In the behavioral Experiment 1, participants replied “no”, in the neuroimaging Experiment 2, participants did not respond (go/no go task).

In the UNR condition and all filler conditions, focus could either be on the subject or on the object.

Fig. 3

Clusters in bilateral superior temporal gyri found at the a-priori significance level of p–unc. < 0.001, cluster p-fwec < 0.05 showing reduced activation for semantically related probe words. Bar graph shows model betas (mean, s.e.) for contrast against the baseline condition (UNR).

Focus alternative status

The whole brain analysis revealed increased activation for alternatives (rel) compared to non-alternatives (rel) in three clusters: Bilaterally in the precuneus, in the right post-/precentral gyrus, and in the fronto-median wall and the bilateral superior frontal gyri (see Fig. 4 and Table 4). No activations were found for the opposite contrast. No effects of focus alternative status were observed in classic language areas (cf. Indefrey, 2011, Price, 2012).

Fig. 4

Clusters showing increased activation in the pre-/postcentral gyrus, the fronto-median wall, and the precuneus for focus alternatives as compared to semantically-related non-alternatives, found at the a-priori significance level of p-unc. < 0.001, cluster p-fwec < 0.05. Bar graph shows model betas (mean, s.e.) for contrast against the baseline condition (UNR).

While the finding of an effect of focus alternatives in the precuneus and in the fronto-median wall is in line with the discourse processing literature and our hypotheses, the effect in the primary motor and primary sensory area was unexpected. We suspected that this effect might reflect response selection processes: In the behavioral experiment, participants gave slower responses in the rel condition than in the rel and unr conditions. While participants did not respond to the critical trials in the MRI study, they still had to decide whether or not to press the button for each trial. Deciding not to respond (because the word had not occurred in the sentence) might have been especially hard in the rel condition. In order to test this post-hoc explanation we computed an additional contrast, which we named easy – hard, between the rel and unr conditions and the rel condition. While we found no effect at our pre-set significance level, with a lowered threshold (peak level: p < 0.005, cluster level: pfwe < 0.05) a cluster in the pre-/postcentral gyrus showed higher activations for the easy than for the hard conditions. This cluster (peak voxels: 21–34 74, peak p-value: 0.001, pfwe-c: 0.087) overlapped to a large extent with the cluster found for rel minus rel in the motor and premotor area (peak voxels: 18–34 77, peak p-value: 0.001, pfwe-c: 0.009). Crucially, no shared activation was obtained in the precuneus or the fronto-median cortex. Thus, it seems likely that this sensory-motor activation is a preparatory effect, reflective of the ease of the go/no-go decision. By contrast, effects in the precuneus and in the fronto-median cortex seem to reflect genuine focus processing.

There was no overlap between the activations for semantic priming and the alternative effect, not even with a lowered significance threshold or with a small-volume correction for the alternative contrasts within the unr – (rel + rel) contrast.

General discussion

The aim of the present paper was to determine the neural signature of focus alternative processing and to contrast it with lexico-semantic processing, as indexed by semantic priming. Participants listened to sentences in which the object was either realized with intonation focus or not. After a spoken target sentence, a written word was presented that could either be semantically related to the focused object (rel semantically related to the unfocused object (rel) or unrelated (unr).

A first behavioral experiment showed processing differences between focus alternatives and non-alternatives. Reaction time data yielded slower responses to primed non-alternative targets than to unrelated prime words and focus alternatives, showing differential processing of focus alternatives and non-alternatives. However, this pattern also raised the question whether focus alternatives undergo semantic priming at all.

In the neuroimaging experiment we first assessed semantic priming by comparing the two related conditions to an unrelated condition. In the second step, we investigated the additional impact of the focus alternative status by comparing the two related conditions which differed only in whether or not the probe word was an alternative to the focused element (rel) or not (rel). The neuroimaging data showed that if a target word was a good semantic fit to the presented prime sentence, priming occurred independently of the focus alternative status, as reflected in the reduced activations observed in the superior temporal gyri. That is, semantic priming took place independently of whether the prime word in the preceding sentence was emphasized or not. As can be seen in Fig. 3, the priming effects for the rel and the rel condition are of almost equal magnitude. This alleviates potential concerns that the necessary confound between focus placement and alternative condition (cf. Table 1) had obscured the results. If the placement of the accent on the subject in the rel condition had made the object more difficult to process, the priming effect in the rel would have been reduced, which is not the case here.

By contrast, no effect of focus alternative status was found within the areas activated for semantic relatedness even after lowering the statistical significance threshold or with small-volume correction. Together with the almost identical contrasts values in bilateral STG (Fig. 3) this suggests that this null-result is real and not the result of low statistical power. Instead, with the addition of the focus alternative status, a distinct brain network became involved, namely the precuneus and the fronto-median cortex. As these areas were previously found in tasks that required to integrate information across utterances (Ferstl and von Cramon, 2001, Ferstl and von Cramon, 2002), focus alternative status seems to be evaluated separately from semantic similarity in a wider brain network involved in coherence building. The theoretical implications of finding an effect of focus alternative status in these areas will be discussed in turn.

Precuneus and fronto-median wall

Activation in the precuneus was significantly increased for the rel condition compared to the rel condition. The precuneus was found to be more active for coherent than for incoherent texts (Ferstl and von Cramon, 2001, Ferstl and von Cramon, 2002).

The precuneus is part of the default mode network, which is usually more active during rest and deactivated during cognitively demanding tasks (see Gusnard & Raichle, 2001, for a review). As our design lacked a true baseline rest condition, we cannot evaluate whether the precuneus activation can be interpreted as evidence for modulation of the default mode network. Yet, some authors have argued that the precuneus is more active when information has to be accumulated and integrated over minutes-long timescales (Simony et al., 2016, p. 2), which presumably happens during free thought at rest but also during narrative building. Even though integration in our task was only possible within a time scale of several seconds, still, the rel condition afforded more integration of information than the rel condition.

Also, a cluster in the medial frontal cortex extending bilaterally to the superior frontal gyrus was more active for the rel than the rel condition. This cluster was located close to an area reported by Ferstl and von Cramon (2001) for coherent compared to incoherent sentence pairs, which they referred to as the fronto-median wall. While this brain region had often been reported in studies on theory of mind (Fletcher et al., 1995, Gallagher et al., 2000, Stone et al., 1998, Stuss et al., 2001), Ferstl and von Cramon (2002) tested explicitly whether its involvement in coherence processing depended on having an insight in other people’s thought processes. They compared a condition where coherence could be derived logically with a condition where coherence had to be derived through theory of mind. As the fronto-median wall was involved in both cases, Ferstl and von Cramon concluded that it underlies general coherence building processes beyond purely lexico-semantic processing, irrespective of theory of mind. This explanation fits with our current data – the state of being a focus alternative goes beyond a purely lexico-semantic relation which was probed with the semantic contrast.

Thus, both precuneus and the fronto-median wall seem to subserve coherence detection or possibly even establishing coherence relations. This agrees well with our findings – if the prime sentence had focused the object, it appears more coherent to continue with an alternative to the object than if the prime sentence had focused the subject. This is in accordance with empirical findings (Kaiser, 2010) and corpus data (Spalek & Zeldes, 2017), which show that alternatives to a focused element are likely to be mentioned in later discourse. Thus, returning to the introductory example, a valid continuation for “The cook picked some fresh [PARSLEY]F” would be “because he didn’t like cilantro in Asian dishes”. By contrast, this continuation would sound odd after “[The COOK]F picked some fresh parsley.” Hence, both precuneus and fronto-median cortex are sensitive to coherence. The distinct aspects of discourse integration which are subserved by each of these areas should be targeted by future research.

Pre-/postcentral gyrus

An unexpected effect of alternative processing surfaced in the contrast rel minus rel in sensory-motor areas in the pre/postcentral gyrus. Activation was stronger for the rel than for the rel condition. We speculated that the observed effect might be connected to response preparation, since we had seen in the behavioral experiment that the rel condition was much slower than the other two. This post-hoc explanation was confirmed through an additional analysis in which we contrasted easy and hard conditions (based on the reaction time data from Experiment 1) and found activations in a largely overlapping cluster.

Differences between the behavioral measures in Experiment 1 and the brain imaging results in Experiment 2

One noticeable difference between the behavioral study and the neuroimaging study was observed in the rel condition. While the reaction time measures showed no difference between unr and rel, the neuroimaging data indicate a clear priming effect for rel which is of equal magnitude as the priming effect for rel. This shows that reaction time measures capture only one aspect of complex processing. The probe recognition task reflects the difficulty of deciding whether or not a probe word had occurred in a previous context, that is, of integrating the word with its context. We had chosen this task because it required participants to process both the sentence and the word attentively whereas in a task like lexical decision on the probe word, there would have been no reason for participants (apart from compliance with the instruction) to process the sentence content. However, it is more than likely that a task like lexical decision which taps into word recognition and lexical retrieval would have yielded a differential effect between the unrelated condition and the rel condition. It might even be the case that focus alternatives are harder to suppress because they are more strongly activated than the unrelated items (as is, in fact, evident in the semantic priming contrast) but at the same time, because the focused element is processed with more attention, its alternative can be rejected more easily as not having occurred in the sentence. Thus, the two effects would cancel each other out.

Conclusions

The aim of this study was to describe the neural signature of the focus alternative status. The results suggest a role of areas outside the classic language network for the integration of semantic relatedness and focus structure, as conveyed here by pitch. Effects in the precuneus and the fronto-median wall suggest that focus alternatives are a discourse-level phenomenon and contribute to the overall congruence of a segment of language. The comparison between effects of semantic priming and pitch focus on processing of subsequent inputs provides direct evidence for the involvement of distinct neural networks in processing of lexico-semantic associations and associations based on discourse inference.

Statement of significance to the neurobiology of language

The present study provides a first and differential description of the neural network that processes focus alternative status during speech comprehension. Focus alternatives are central to correct comprehension of utterances in context, thus understanding their neural underpinnings is necessary to elucidate the neurobiological basis of discourse comprehension.

Sentence	Related probe	Unrelated probe
Thorsten has fed the goats on the farm	Cows	Peaches
Nicole has watched the sparrows when they searched for food	Chickadees	Roses
Nico has provided wine for the party	Beer	Elves
Wolfgang has drawn dragons for an animated film	Elves	Beer
Frank has seen badgers during the night walk	Lynxes	Spoons
Hans has served rice with the fish	Pasta	Pistols
Sebastian has soaked the pullovers before doing the laundry	Socks	Giants
Holger has cut mangoes into the fruit salad	Bananas	Bombs
Britta has watched sharks for her research project	Whales	Tables
Elena has played the trumpet in the big band	Trombone	Juice
Mia has listened to the thrushes during their singing	Blackbirds	Curtains
Jakob has collected crabs at the beach	(Sea) shells	Masons
Falk has assembled the trains from his model kit	Boats	Tomatoes
Greta has ordered beds for her bed-and-breakfast	Tables	Whales
Maria has rinsed the pans in the sink	Bowls	Lions
Patrick has left the glasses in the dishwasher	Cups	Perches
Daniela has washed the pears before eating	Plums	Statues
Carsten has picked cherries from the tree	Peaches	Cows
Alex has chosen physics as an advanced course	Chemistry	Kiwi fruits
Anja has tuned the guitars in the music school	Harps	Bugs
Lukas has photographed bridges at dusk	Towers	Lungs
Moritz has looked for Chile on a map	Peru	Oranges
Oskar has played handball in a club	Soccer	Skirts
Ralf has sold marbles at the flea market	Spinning tops	Pianos
Georg has polished the forks until they sparkled	Spoons	Lynxes
Anna has chosen shelves for her living room	Curtains	Blackbirds
Gustav has bought screws in the DIY shop	(Wall) plugs	Flute
Wiebke has taught English for beginners	German	Foxes
Kristin has chosen lilies for her wedding bouquet	Roses	Chickadees
Yvonne has spoken to doctors at the job fair	Teachers	Food processors
Leonie has fertilized the violas on her balcony	Dahlias	Rubies
Susanne has changed the brakes for the MOT	Tires	Milk
Sonja has cut carnations for a bouquet	Primula	Shampoo
Peter has photographed the apes at the zoo	Lions	Bowls
Ben has provided the starlings with a nesting site	Finches	Torches
Iris has pressed lemons for the cocktails	Oranges	Peru
Tim has grated carrots into the stew	Turnips	Song
Julia tried dresses in front of the mirror	Skirts	Soccer
Tamara has sold the pearls from her vault	Rubies	Dahlias
Lars has bought the jacket at the sales	Coat	Mathematics
Emil has studied pharmacy at university	Theology	Slide
Jürgen has copied kidneys from the biology textbook	Lungs	Towers
Claudia has tried swings for her children	Slide	Theology
Luise was sickened at the spiders in the hotel	Bugs	Harps
Jörg has lit candles in the twilight	Torches	Finches
Angelika has put coke in the fridge	Juice	Trombone
Thomas has caught trouts in the pond	Perches	Cups
Tina has dropped biology in sixth form	Mathematics	Coat
Pia has asked for lollies for her first day at school	Boiled sweets	Singers
Ole has engaged dancers from the theatre	Singers	Boiled sweets
Felix has reaped the peas in his garden	Tomatoes	Ships
Franziska has found the sponges in the bathroom	Brushes	Gardeners
Laurenz has brought the horses to the meadow	Sheep	Sapphires
Florian has bought a toaster at the electric shop	Food processor	Teacher
Erik has left his violoncello on stage	Flute	(Wall) plug
Ulrike has taken care of the gymnasts at the Olympic games	Fencers	Axes
Dominik has interviewed the swimmers after the championship	Runners	Teddy bears
Eva has tried out violins at the music shop	Pianos	Spinning tops
Bernd has disarmed landmines in war zones	Bombs	Bananas
Stefan has looked at daggers at the weapons museum	Pistols	Pasta
Heike has composed a poem for the birthday	Song	Turnip
Saskia has made an arrangement with the farmers at the market	Gardeners	Brushes
Birgit has bought raspberries for the cake	Kiwi fruits	Chemistry
Lisa has seen deer during the walk through the woods	Foxes	German
Romy has given away her dolls to refugee children	Teddy bears	Runners
Klaus has talked to the locksmiths at the construction site	Masons	(Sea) shells
Jens has ordered saws for his DIY store	Axes	Fencers
Walter has admired the emeralds at the jeweller’s shop	Sapphires	Sheep
Simon has dreamt of the witches from his book of fairy tales	Giants	Socks
Janine has mentioned the paintings in her review	Statues	Plums
Isabell has forgotten the cheese when shopping	Milk	Tires
Michael has taken the soap out of his sponge bag	Shampoo	Primulas