Shared syntax in language production and language comprehension--an FMRI study.

During speaking and listening syntactic processing is a crucial step. It involves specifying syntactic relations between words in a sentence. If the production and comprehension modality share the neuronal substrate for syntactic processing then processing syntax in one modality should lead to adaptation effects in the other modality. In the present functional magnetic resonance imaging experiment, participants either overtly produced or heard descriptions of pictures. We looked for brain regions showing adaptation effects to the repetition of syntactic structures. In order to ensure that not just the same brain regions but also the same neuronal populations within these regions are involved in syntactic processing in speaking and listening, we compared syntactic adaptation effects within processing modalities (syntactic production-to-production and comprehension-to-comprehension priming) with syntactic adaptation effects between processing modalities (syntactic comprehension-to-production and production-to-comprehension priming). We found syntactic adaptation effects in left inferior frontal gyrus (Brodmann's area [BA] 45), left middle temporal gyrus (BA 21), and bilateral supplementary motor area (BA 6) which were equally strong within and between processing modalities. Thus, syntactic repetition facilitates syntactic processing in the brain within and across processing modalities to the same extent. We conclude that that the same neurobiological system seems to subserve syntactic processing in speaking and listening.

Introduction

Successful communication relies on both efficient production and comprehension of language. Is there 1 integrated system for comprehension and production or are there 2 separate systems? How are comprehension and production processes related and which information is shared by the 2 processing modalities? We can ask these questions in regard to the individual word level or the sentence level, where words are combined in a syntactic structure. The latter is the focus of the current study. Specifically, in this study, we investigate whether the neurobiological substrate for coding and processing syntactic representations is shared between speaking and listening.

Naturally, the input for speaking and listening is different. A speaker starts with a communicative intention or a message representation that she wants to communicate to a listener. Over several processing stages, this intention is converted into a sequence of sounds which are articulated. The listener in turn receives this stream of auditory information and has to retrieve its meaning and the intention of the speaker. A core process during both production and comprehension is syntactic processing: specifying the syntactic relations between words in the sentence.

The starting point and the context of syntactic processing are different for production and comprehension. A speaker first converts the intended message into a representation with a specified thematic role structure (i.e., who does what to whom, how, when, and where). During syntactic encoding, the thematic role structure is encoded as one particular syntactic structure; for example, a passive transitive structure like “The boy was kissed by the girl yesterday at the cinema.” This is achieved by a unification or integration operation on the syntactic information which is connected to the different lexical elements of the message (Vosse and Kempen 2000). The syntactic building blocks which are used in this unification operation include the syntactic category (e.g., it is a verb) and a frame specifying the possible structural environment (e.g., it takes a subject and an object). During comprehension, this information is retrieved from the recognized words in the input and the sentence structure is then parsed or decoded. From “The boy was kissed by the girl” a listener has to recover that it is a passive transitive structure and that the girl is the agent and the boy the patient of the kissing event.

Certain aspects of syntactic processing thus differ between production and comprehension. During language production, there are many ways for a speaker to convey the same message: one thematic role structure can be expressed by several different syntactic structures. The message that a girl was kissing a boy, can be expressed in the following syntactic structures: “The girl kissed the boy,” “The boy was kissed by the girl,” or “It is the girl that kissed the boy.” A speaker can choose to encode the message as a passive transitive structure when she for instance wants to emphasize the thematic role of the patient (instead of the agent). During language comprehension, the order of the words in the incoming information has been determined by a speaker but it is the listener who has to reconstruct the correct syntactic structure. For the 2 utterances “The boy kissed the girl” and “The boy was kissed by the girl,” the words ‘boy,’ ‘kissed,’ and ‘girl’ hit the ear of the listener in the same order, but the syntactic structure and the message of the utterance are different. In addition, syntactic assignments are often based on partial information during comprehension, since utterances reach the listener incrementally and therefore ambiguities may arise at any given point in the utterance.

Shared Syntax?

This study aims to answer the question to what extent syntactic encoding and decoding rely on the same neurobiological system. Traditionally, psycholinguists have investigated syntactic processing separately in comprehension and production, sometimes with the assumption that these are 2 separate systems. For example, Clark and Malt (1984) argued that comprehension must have access to more information than production, since speakers can understand syntactic forms in dialects or in literary texts (e.g., Shakespeare) which they themselves cannot produce. Developmental as well as neuropsychological research is often put forward as evidence for the view that the comprehension and production systems are separate. Developmental research suggests that children can understand more than they can produce, and it has been argued that this is the case for complex syntactic constructions. Children can generally understand syntactic forms well before they begin to produce them (Fraser et al. 1963; Clark and Hecht 1983; Bates and Bretherton 1988). Early neuropsychological research uncovered an apparent double dissociation between aphasias: patients with damage to Broca's region are characterized by impaired production and relatively intact comprehension and patients with damage to Wernicke’s region show impaired comprehension and relatively intact production (Lichtheim 1885). This contributed largely to the idea of 2 separate systems. Although the idea of 2 separate anatomical systems is outdated, and comprehension as well as production is thought to engage both Broca's and Wernicke's regions to some extent, the idea of 2 functionally separated systems still commands a sizable following. For example, Grodzinsky (2000) argued that the mechanisms underlying production and comprehension must be (partially) different based on linguistic differences (“tree pruning” vs. “trace deletion”) in the production and comprehension deficits of agrammatic patients.

Others have contested the position that there are separate systems for syntax in production and comprehension. Instead, they advocate a unitary system with shared representations or shared processes manipulating representations. Kempen (2000) argued that syntactic encoding and decoding rely on a single processing mechanism operating in different processing contexts. He based his claim on a series of shared characteristics of syntactic processing across processing modalities: sensitivity to conceptual factors, direct mapping between thematic relations and syntactic relations, incremental processing, and determinism (the process ends with one result). In a recent study, Kempen et al. (forthcoming) found evidence for a common grammatical workspace: the mechanism that constructs (in production) or deconstructs (in comprehension) syntactic structures and the short-term storage of the result of this computation is shared between the modalities. Also, the interactive alignment model of dialogue assumes that speakers and listeners share representations, although this does not necessarily imply that the processes operating on them are also shared between modalities (Pickering and Garrod 2004).

Syntactic Priming between Processing Modalities

The tendency to repeat syntactic structures across utterances is called syntactic priming (Bock 1986; for a review: Ferreira and Bock 2006; Pickering and Ferreira 2008). This phenomenon is a valuable tool to tap into syntactic processing. Syntactic priming leads to facilitated processing, evidenced not only by the increased likelihood to choose the same structure in successive sentences (Bock 1986) but also by speeded speech onset or reading times for repeated syntactic structures (Smith and Wheeldon 2001; Traxler and Tooley 2008) and by repetition effects in the brain measured with functional magnetic resonance imaging (fMRI) (Weber and Indefrey 2009; Menenti et al. forthcoming).

Syntactic priming from one processing modality to another provides insight into whether syntactic information is shared between modalities. If syntactic information is shared, syntactic processing in one modality should lead to adaptation effects in the other modality. Several behavioral experiments have shown that syntactic comprehension-to-production priming is possible. Reading or hearing a sentence with a particular syntactic structure increases the likelihood of using the same structure instead of an alternative during the production of a successive sentence (Branigan et al. 1995; Potter and Lombardi 1998; Branigan et al. 2000; Bock et al. 2007). Also evidence for syntactic production-to-comprehension priming has been reported: production of a particular syntactic structure influenced subsequent picture matching for ambiguous descriptions (Branigan et al. 2005). These behavioral between-modality syntactic priming experiments seem to suggest that syntactic information is shared between comprehension and production.

However, 2 issues complicate the picture. First, it is very difficult to compare a behavioral measure of syntactic priming in production (e.g., which structure does a speaker choose?) with a behavioral measure of syntactic priming in comprehension (e.g., how fast is it read?). Therefore, syntactic comprehension-to-production priming effects cannot easily be compared with production-to-comprehension priming effects. The second issue is that—strictly speaking—the results from these behavioral experiments do not rule out that there is a close link between the 2 modalities while syntactic information is not shared. Comprehension-to-production priming may be influenced by production-based predictions during comprehension (Pickering and Garrod 2007). Likewise, a production-to-comprehension effect may be influenced by comprehension-based monitoring during production (Levelt 1989) (although see Branigan et al. 2005).

The present study aims to address these concerns by 1) examining the neuronal substrate of syntactic encoding and decoding using fMRI, with the advantage that the brain activity measured by fMRI serves as common index of the production and the comprehension system and 2) examining syntactic comprehension-to-production as well as production-to-comprehension priming and comparing these between-modality effects with within-modality effects in one experiment.

Syntactic Processing in the Brain

Do the neural substrates for syntactic encoding and decoding overlap in the brain? Several neuroimaging studies have examined syntactic processing either in comprehension or in production. Investigating language production, Haller et al. (2005) compared sentence generation with word reading and with sentence reading using fMRI. They found effects in Brodmann's areas (BAs) 44/45 of the left inferior frontal gyrus as well as in BA 6, BA 7, and right BA 13. Indefrey et al. (2001) found a neural correlate of syntactic encoding during production in left BA 6 and BA 44 using positron emission tomography. Additionally, they found evidence for a graded response dependent on the syntactic complexity. In comprehension, Snijders et al. (2009) found the left inferior frontal gyrus (IFG) and left posterior middle temporal gyrus (MTG) involved in syntactic processing. Noppeney and Price (2004) found a syntactic processing effect in comprehension in the left anterior pole. Also during language comprehension, Ni et al. (2000) found increased activity in left inferior frontal regions for syntactic anomalies. Taken together, these studies mainly found left frontal or temporal regions involved in syntactic encoding or decoding.

Menenti et al. (forthcoming) systematically compared syntactic effects during speaking and listening using an fMRI adaptation paradigm. fMRI adaptation is a phenomenon whereby the blood oxygen level dependent (BOLD) response in areas sensitive to a stimulus property, for example syntax, is reduced or enhanced when this stimulus property is repeated (Henson 2003; K Segaert, K Weber, FP de Lange, KM Petersson, P Hagoott, unpublished data). Popular models on the source of fMRI adaptation are the fatigue model, the sharpening model, and accumulation model (for a review, Grill-Spector et al. 2006). These models propose, respectively, that neurons in a neuronal population generally respond less strongly when the stimulus property is repeated, that fewer neurons in a neuronal population respond, and that neuronal activity of the neurons peaks earlier. Menenti et al. (forthcoming) found repetition suppression effects for the repetition of syntactic structure in the left posterior MTG and the left IFG during production as well as during comprehension. However, the involvement of the same regions does not necessarily mean that the same neuronal substrate underlies both modalities. Only when the same neuronal populations are involved, one can speak of a shared neuronal substrate. The results of Menenti et al. (forthcoming) can strictly speaking not exclude the possibility that different sets of neuronal populations within a particular brain region underlie syntactic decoding versus syntactic encoding. However, one can conclude that neuronal populations are shared by modalities if we can show that there are between-modality fMRI adaptation effects and that these are equally strong as within-modality fMRI adaptation effects. Irrespective of one's view on the source of fMRI adaptation (fatigue, sharpening, or accumulation), fMRI adaptation is assumed to be a consequence of a modulation within the same neuronal population.

In the present event-related fMRI study, we aimed to investigate whether there is a common neuronal substrate for syntactic decoding and syntactic encoding. We investigated fMRI adaptation effects to the repetition of syntactic structures and compared within-modality adaptation effects (syntactic production-to-production and comprehension-to-comprehension priming) with between-modality adaptation effects (comprehension-to-production and production-to-comprehension priming). Comparable within-modality and between-modality syntactic fMRI adaptation effects would suggest that the same neuronal populations are involved in syntactic encoding and syntactic decoding.

Materials and Methods

Participants

Twenty-four right-handed native Dutch speakers without neurological or language impairments and with normal or corrected to normal vision (12 males; mean age 22 years, standard deviation 4.8) participated in the experiment. All participants had attended or were attending university education in the Netherlands. All participants gave written informed consent prior to the experiment and were compensated for their participation.

Stimulus Material

The stimulus material used in this study is largely identical to the material used in Menenti et al. (forthcoming). There were 1728 photographs and 432 auditory sentence descriptions of transitive events. These depicted or described 36 different events such as “kissing,” “helping,” or “strangling” with the agent and patient of this action. The patient of an event is the one who is acted upon. Each event was enacted in the photographs by 4 couples (2 × man/woman; 2 × boy/girl), each of these once with the male actor as agent and once with the female actor as agent. Each photograph also had a version with the agent on the left and with the agent on the right. Of each transitive photograph, there were 2 color-coded versions and 1 grayscale version. Color-coded photographs elicited either active or passive sentence descriptions because participants were instructed to describe these photographs naming the green actor before the red actor. There was an active version with a green agent and a red patient and a passive version with a red agent and a green patient. The 2 color-coded versions were used during production trials. During comprehension, we presented grayscale photographs, identical to the photographs used in the production trials. During comprehension, photographs were accompanied by auditory sentence descriptions of either active or passive syntactic structures.

There were also 795 photographs and 303 concomitant auditory sentence descriptions serving as fillers. These fillers depicted or described intransitive events such as “singing” and “running” or locative actions such as “standing” and “lying.” The intransitive photographs depicted 1 actor in green or in red for production trials or 1 actor in grayscale (accompanied by an auditory description) for comprehension trials. The locative photographs depicted 2 objects or 1 actor and 1 object. There were 2 color-coded versions of the locatives to elicit a locative state (“The ball lies on the table.”) or a frontal locative (“On the table lies a ball.”) for production trials. For comprehension trials, there was a grayscale version which would be accompanied by a locative state or a frontal locative description. The intransitive and locative filler items were added to provoke variability in syntactic structures and in the lexical items that participants produced/heard during the experiment. For intransitives, the actors were sometimes famous people, animals, or people that could be named by their profession; for locatives, inanimate objects were used.

For the comprehension trials, there were also 97 auditory sentence descriptions that did not match the accompanying grayscale photograph. These mismatch trials were used for attentional control. The descriptions were grammatically correct but did not describe the situation depicted in the photograph. There were mismatch descriptions of intransitive photographs (50%) and of transitive photographs (50%). Semantic and syntactic processing was necessary to be able to detect the mismatches between photograph and auditory description. For example, for a photograph that depicted a man kissing a woman, mismatch descriptions could be: “The man punishes the woman,” “The girl kisses the woman,” “The woman kisses the man.” The transitive mismatch items were not target items.

We pretested the materials to establish whether the depicted actions were clear and to measure which verb was most commonly used to describe the action. During the actual experiment this verb was presented preceding the photographs.

Experimental Design

We used a 2 × 2 × 2 × 2 design with the factors Syntactic Repetition (syntax was novel vs. repeated compared with the sentence that preceded it), Modality Repetition (processing modality, i.e., speaking vs. listening was novel vs. repeated compared with the sentence that preceded it), Target Modality (listening vs. speaking), and Target Structure (active vs. passive voice). This resulted in 16 conditions. The design (8 conditions resulting from crossing the first 3 factors, thus, leaving out the factor target structure) is illustrated in Figure 1.

Figure 1.

(A) Design and stimuli. Participants either described colored photographs or listened to descriptions of grayscale photographs, containing action, agent, and patient. To guide production, participants were instructed to name the green actor before the red actor. Between subsequent sentences, that is, prime and target, the syntactic structure and the processing modality could be repeated (for syntax: active–active or passive–passive, for modality: production–production or comprehension–comprehension) or novel (for syntax: active–passive or passive–active, for modality: production–comprehension or comprehension–production). (B) Procedure. We used a running priming paradigm where each target item also served as a prime sentence for the next target item. The verb always preceded the photographs. Green verbs indicated a “production photograph” would follow, gray verbs indicated a “comprehension photograph” would follow.

We used a running priming paradigm where each target item also served as the prime sentence for the next target item (Fig. 1). Therefore, we had an equal amount of active and passive transitive structures and choose to manipulate target structure as a factor. However, we do not expect any differential syntactic repetition effects for actives and passives. Furthermore, while actives sentences are shorter than passives sentences, this is the case in production as well as in comprehension and thus orthogonal to the effects we are interested in.

The verb was always repeated between prime and target. Behavioral syntactic priming studies have shown that verb repetition is critical for syntactic priming within language comprehension (Arai et al. 2007; Tooley et al. 2009). Because a crucial aspect of the present study is the comparison of effects within the comprehension modality to between-modality effects, we opted to manipulate syntactic priming while always repeating the verb between prime and target sentence. Because we used a running priming paradigm, the verb was repeated within each block of transitive syntactic structures.

The target items were presented in 80 blocks with an average length of 5 transitive structures (range 3–7 items). The conditions followed each other in a random order that was different for every participant, with 2 constraints on the order of conditions: The first constraint was that no condition was repeated twice in a row. The second constraint was that a target item with adults was always followed by a target item with children and vice versa, so that there was no lexical repetition other than the verb. In a full list of items presented to the participant, the same action or the same actors could occur several times, but the combination of actors and actions was unique.

The target blocks were alternated with filler blocks with an average length of 3.5 (range 2–5 items). Most of the time, the verb was repeated between filler items within one block. For 10% of the filler items, this was not the case to bring in some extra variation. A full list of items presented to the participant consisted of approximately 59% transitive structures and 41% fillers. Fifty percent of the items were production items and 50% were comprehension items.

There were 20 items in each of the 16 conditions. In addition to this, in the beginning of each of the 80 blocks of transitive structure items, there was one transitive structure item serving as a prime only item. This increased the number of transitive structure items to 400. Each participant received 680 trials in total (transitive and filler structures), which were divided over 2 scanning sessions. Each photograph could occur only once in the experiment and every participant saw a different list of items.

Task and Procedure

The stimuli were presented in the following way. First, the verb was presented. Then a photograph followed, which only during comprehension trials was accompanied by an auditory description. The presented verb was colored–coded to let the participant know whether a “comprehension photograph” or a “production photograph” would follow. Green verbs preceded colored production photographs and gray verbs preceded black/white comprehension photographs (Fig. 1).

Production

During production trials, the task was to describe the colored–coded photographs overtly with a short sentence using the presented verb. Participants were instructed to name the green actor before the red actor (stop light paradigm: Menenti et al. forthcoming). There was no cue for the participants to start the descriptions; they could freely start whenever they were ready.

Comprehension

During comprehension trials, we used a sentence-picture matching paradigm (Clark and Chase 1972): participants were presented with a photograph and an auditory description. The photographs were the grayscale version of the ones used in the production trials. The sentence-picture matching paradigm has been used extensively and a recent study supports that it is suitable for studying online situated language comprehension (Knoeferle et al. 2011). By choosing situated paradigms for both production and comprehension trials, we maximize comparability and ensure that the difference between the 2 only lies in linguistic processing. To make participants pay attention, we instructed them to listen carefully to the description of the black/white photographs and use the response box to indicate when this description was incorrect (the response hand was counterbalanced between participants). During 10% of the comprehension trials, there was a mismatch between the description and the photographs. Only for those trials, a response had to be given.

Participants completed a short practice block in the scanner before the actual experiment started. The experiment consisted of 2 runs of 45 min. Between the 2 runs, the participants got an anatomical T1 scan and a short break outside the MRI scanner. Each trial consisted of the following events: first, the verb was presented for 500 ms. After an ISI of 500—2500 ms, the photograph was presented for 2000 ms and then the screen turned black. The photograph thus had a fixed presentation time during production as well as comprehension trials. For the production trials, the participants started speaking during the presence of the photographs. For the comprehension trials, the auditory sentence was presented following the photograph with an ISI of 0–1000 ms, so that we could differentiate between the onset of the photograph and the auditory description in our analyses. The total trial duration of one trial was 7000 ms.

The experimenter coded the participant's production responses online for correctness. Target trials were considered for analysis if during both prime and target trial 1) the correct structure was used and 2) both actors were named accurately and the verb was used correctly.

fMRI Data Acquisition

Participants were scanned with a Siemens 3-T Tim-Trio MRI scanner, using a 12-channel surface coil. To acquire functional data, we used parallel-acquired inhomogeneity-desensitized fMRI (Poser et al. 2006). This is a multiecho echo-planar imaging sequence, in which images are acquired at multiple time echos (TEs) following a single excitation (time repetition [TR] = 2.398 s; each volume consisted of 31 slices of 3 mm thickness with slice gap of 17%; isotropic voxel size = 3.5 × 3.5 × 3 mm3; field of view [FOV] = 224 mm). The functional images were acquired at following TEs: TE1 at 9.4 ms, TE2 at 21.2 ms, TE3 at 33 ms, TE4 at 45 ms, and TE5 at 56 ms, with echo spacing of 0.5 ms. This entails a broadened T2* coverage because T2* mixes into the 5 echoes in a different way, and the estimate of T2* is improved. Accelerated parallel imaging reduces image artifacts and thus is a good method to acquire data when participants are producing sentences in the scanner (causing motion and susceptibility artifacts). However, the number of slices did not allow acquisition of a full brain volume in most participants. We made sure that the entire temporal and frontal lobes were scanned because these were the regions where the fMRI adaptation effects of interest were expected. This meant that data from the superior posterior frontal lobe and the superior parietal lobe (thus data from the top of the head) were not acquired in several participants. A whole-brain high-resolution structural T1-weigthed magnetization prepared rapid gradient echo sequence was performed to characterize participants' anatomy (TR = 2300 ms, TE = 3.03 ms, 192 slices with voxel size of 1 mm3, FOV = 256), accelerated with GRAPPA parallel imaging.

Data Analysis

Preprocessing

fMRI data were preprocessed using SPM5 (Friston et al. 2007). The first 5 images were discarded to allow for T1 equilibration. Then the 5 echoes of the remaining images were realigned to correct for motion artifacts (estimation of the realignment parameters is done for one echo and then copied to the other echoes). The 5 echoes were combined into one image with a method designed to filter task-correlated motion out of the signal (Buur et al. 2009). First, echo 2–5 (i.e., TE2, TE3, TE4, and TE5) were combined using a weighting vector with the weights depending on the measured differential contrast to noise ratio. The time course of an image acquired at a very short echo time (TE1) was then used in a linear regression as a voxelwise regressor for the other image (i.e., the result of combining TE2, TE3, TE4, and TE5) in the same echo train acquired with high BOLD sensitivity. The resulting images were coregistered to the participants' anatomical volume, normalized to Montreal Neurological Institute space, and spatially smoothed using a 3D isotropic Gaussian smoothing kernel (full-width at half-maximum = 8 mm).

Whole-Brain Analysis

We performed first- and second-level statistics using the general linear model framework of SPM5 (Friston ). Our 2 × 2 × 2 × 2 design resulted in 16 conditions and thus 16 main regressors for the statistical analysis of the fMRI data. We used an implicit baseline. In the first-level linear model, we modeled the individual start time of the photograph (during production trials) or the auditory sentence description (during comprehension trials). We modeled the hemodynamic response function only as related to these onsets and set the duration as a constant event. Separate regressors were included for the verbs, photographs during comprehension trials, fillers items, items which were only primes, and incorrect responses. The events of the model were convolved with the canonical hemodynamic response function provided by SPM5. Also the temporal derivatives were included in the model. Furthermore, 6 motion parameters (realignment parameters: translation along, and rotation around, the x, y, and z axes) and 2 parameters which correct for global intensity fluctuations (compartment signal parameters: white matter and cerebral spinal fluid; Verhagen et al. 2008) were added as regressors. For the second-level random-effects analysis, we used the beta-images of the 16 main regressors. The cluster size was used as the test statistic and only clusters significant at P < 0.05 corrected for multiple nonindependent comparisons are reported. Local maxima are also reported for all clusters with their respective Z values.

Region of Interest Analysis

We performed a region of interest (ROI) analysis in the activation clusters for which we found a main effect of syntactic repetition in the whole-brain analysis. The sole aim of the ROI analysis was to establish with higher sensitivity than in the whole-brain analysis whether there was an interaction between the effect of syntactic repetition and modality change in these clusters. We thus tested an interaction effect which is orthogonal to the main effect that defined the ROI, thereby avoiding biasing the analyses (Kriegeskorte et al. 2009). Of each cluster, we calculated the average time courses using Marsbar (http://marsbar.sourceforge.net/). For the ROI analysis at the second level, we carried out a repeated measures analysis of variance with the factors syntactic repetition, modality repetition, target modality, and target structure on the subject contrast values using SPSS. We corrected for multiple comparisons by using a threshold for significance of P = 0.05 divided by the number of clusters showing a main effect of syntactic repetition in the whole-brain analysis. In the Supplementary Material, we describe the methods and results (Supplementary Fig. 4) of ROI analyses in 2 clusters, 1 in left IFG and 1 in left MTG, found by Menenti et al. (forthcoming) for syntactic processing in comprehension and in production.

Results

Behavioral performance

In the production task, participants responded correctly on 96% of the trials. In the comprehension task, the participants detected on average 92% of the mismatch trials. The average d-prime was 0.91. These results show that participants performed well on both tasks.

Whole-Brain Analysis

For the whole brain comparisons, we used a cluster-level threshold corrected for multiple comparisons of P < 0.05 and an uncorrected voxelwise threshold of P < 0.001 (Fig. 2 and Supplementary Fig. 5, Table 1 and Supplementary Table 3).

Figure 2.

Whole-brain results (see also Table 1). (A) The adaptation effects for syntax repetition. In left MTG, left IFG, and supplementary motor area, there was a repetition suppression effect for repeated compared with novel syntactic structures. (B) Interaction between syntax repetition and modality repetition. No regions showed an interaction between syntax repetition and modality repetition.

Table 1

The effect of syntactic repetition

Anatomical label	BA	Global and local maxima			Cluster-level		Voxel-level
		x	y	z	K	P(corr)	Z
Main effect syntax repetition (no syntactic repetition > syntactic repetition)
L middle temporal	21	−50	40	2	197	0.023	4.92
L inferior frontal (pars orbitalis)	47	−42	24	−2	567	0.000	4.07
L inferior frontal (pars triangularis)	45	−40	32	8			3.67
L inferior frontal (pars triangularis)	45	−40	26	16			3.60
L supplementary motor area	32/6	−10	20	46	190	0.027	3.97
L supplementary motor area	6	−2	14	56			3.58
R supplementary motor area	32/6	8	18	50			3.51
Interaction syntax repetition × modality change.
No significant clusters
Interaction syntax repetition × target modality
No significant clusters

Note: Listed are the Montreal Neurological Institute coordinates for 3 local maxima for each significant cluster in the relevant comparisons (P < 0.05 corrected cluster-level, threshold P < 0.001 uncorrected voxelwise). Anatomical labels are derived from the Automated Anatomical Labeling map (Tzourio-Mazoyer et al. 2002) and from Brodmann's atlas.L, left; R, right.

As displayed in Figure 2 and Table 1, there were several regions showing an adaptation effect to repeated syntax (conditions with novel syntax minus conditions with repeated syntax): the left MTG (BA 21), left IFG (BA 45, extending into BA 47), and bilateral supplementary motor area (BA 6). These regions are thus less activated for sentences with a repeated syntax than for sentences with novel syntax. That is, they show repetition suppression for syntax. We tested whether there was an interaction between syntactic repetition and modality repetition (i.e., whether there was less syntactic adaptation across processing modalities than within processing modalities). Crucially, there was no evidence of such an interaction. We also tested whether there was an interaction between syntactic repetition and target modality (i.e., whether there was less syntactic for comprehension targets than for production targets). There was no evidence of such an interaction. There were no repetition enhancement effects. In the Supplementary Material (Fig. 5 and Table 3), we describe the network of regions that is activated more during production than comprehension, the network of regions that is activated more during comprehension than production, and the network of regions involved in switching between processing modalities. In all 3 cases, we took the conditions with syntactic repetition and without syntactic repetition together.

ROI Analysis

In each cluster that showed an adaptation effect for syntactic repetition, we checked with an ROI analysis whether there was an interaction between the size of the syntactic adaptation effect and modality change. These analyses confirmed the results of the whole-brain analysis: there was no interaction between the adaptation effect for syntactic repetition and within versus between modality priming. In all 3 ROIs, the interaction was clearly absent: left MTG (F1,23 = 0.09, P = 0.77), left IFG (F1,23 = 0.016, P = 0.90), the supplementary motor area (F1,23 = 0.35, P = 0.56). Figure 3 illustrates for each ROI the relative decrease in mean BOLD amplitude for repeated syntax compared with novel syntax, separately for the effect within a processing modality and the effect across processing modalities.

Figure 3.

ROI analysis in the 3 clusters showing a main effect of syntactic repetition—left MTG, left IFG, and supplementary motor area—confirmed that there was no differential repetition suppression effect for syntactic structures within and across processing modalities.

Discussion

In this study, we investigated whether the neuronal infrastructure for coding and processing syntactic representations is shared between language production and language comprehension. We tested this by comparing fMRI adaptation effects for the repetition of syntactic structures within and between processing modalities. While within-modality syntactic adaptation effects in comprehension and production show that the same brain regions are involved, only comparable between-modality adaptation effects indicate that the neuronal populations within these regions are shared. Our results demonstrate that syntactic repetition indeed facilitates syntactic processing in the brain within and across processing modalities to the same extent. Our results disclose the following organizational principles of syntactic processing in comprehension and production: 1) not just the same brain regions, but the same neuronal populations subserve syntactic encoding in production and syntactic decoding in comprehension. Hence, there is a shared neuronal substrate; 2) this neuronal substrate involves left IFG (BA 45), left MTG (BA 21), and bilateral supplementary motor area (BA 6).

Left IFG (BA 45), left MTG (BA 21), and bilateral supplementary motor area (BA 6) are regions that have been found to support syntactic encoding or decoding in previous research (Indefrey et al. 2001; Haller et al. 2005; Snijders et al. 2009; Lee and Newman 2010; Menenti et al. forthcoming). Previous work supports a division of labor between left IFG and left MTG: while the MTG supports the retrieval of lexical–syntactic information from memory, left IFG supports the unification of this information into multiword utterances (Hagoort 2003, 2005; Snijders et al. 2009). Left IFG and the lateral prefrontal cortex are particularly suited for actively maintaining, manipulating, and integrating information in general (Fuster 2001). They might provide the appropriate neurobiological infrastructure for unification processes on syntactic information. The buildings blocks of information used in this unification process are proposed to be lexical–syntactic frames (Vosse and Kempen 2000). These frames are stored in long-term memory and it is left MTG that is involved in the storage as well as retrieval of this lexical–syntactic information.

In the present study, we also found bilateral involvement of supplementary motor area (BA32/6). Our activations lie in pre-SMA, the region of SMA which is more anterior than the coronal plane passing through the anterior commissure (Picard and Strick 2001). More posterior than the level of the anterior commissure lies SMA proper. Unlike SMA proper, which is connected to primary motor cortex, pre-SMA has strong connections to the dorsolateral prefrontal cortex (Bates and Goldman-Rakic 1993; Geyer et al. 2000). Therefore, pre-SMA is functionally considered to be part of the prefrontal cortex and has been associated with a variety of cognitive tasks (Picard and Strick 2001). Pre-SMA has been associated with establishing and retrieving sensorimotor associations at an abstract level which is independent of the input modality and more generally with processing or maintaining relevant sensory information (Picard and Strick 2001). Pre-SMA has furthermore been associated with internally guided word generation at the level of single word production (Crosson et al. 2001; Alario et al. 2006) and encoding of syllable frames and their serial position (Bohland and Guenther 2006; Ghosh et al. 2008). The role of pre-SMA in our study might lie in the process of sequencing syllable structures. The sequence of syllables for 2 passives is more common than the sequence for an active and a passive. Likewise, the sequence of syllables for 2 actives is more common than the sequence for an active and a passive. For instance, for the verb “meten” in Dutch (which translates to “to measure” in English), 2 passives would share the following sequence of syllables: “wordt gemeten door.” Two actives would share the following sequence of syllables: “meet.” In other words, when a syntactic structure is repeated also the sequence of syllable frames is in part repeated. This may be the reason we find fMRI adaptation effects for repeated syntactic structures in pre-SMA.

We investigated the effect of syntactic repetition while always repeating the verb between prime and target sentence. Behavioral syntactic priming studies have shown that verb repetition is critical for syntactic priming within language comprehension (Arai et al. 2007; Tooley et al. 2009). To guarantee that we could compare effects within the comprehension modality with between-modality effects, we needed to establish syntactic repetition effects in the brain within the comprehension modality. A future study would be needed to confirm that the present results are replicated even in the absence of verb repetition.

From our finding that there is a shared neuronal substrate for syntactic processing in speaking and listening, we can infer that there is a shared cognitive system with shared representations (Pickering and Garrod 2004) and/or processes manipulating these representations (Kempen 2000). Therefore, theories of syntactic processing in the comprehension or production domain that propose modality specific aspects are problematic. Our findings do not entirely exclude the possibility that there are some differences between syntactic encoding and syntactic decoding. There may be a dissociation that has to do with the difference in direction between syntactic encoding and decoding. When constructing syntactic structures, a speaker knows the concepts and thematic role structure because she has determined them herself. The difficulty lies more in specifying the word order. On the other hand when deconstructing syntactic structures, the word order is given but the difficulty lies more in reconstructing the thematic role structure. So there may be a difference between syntactic encoding and decoding in terms of where difficulties or ambiguities are likely to arise. Moreover, in comprehension, one might be able to bypass full syntactic decoding in the presence of semantic, lexical, and nonlinguistic information (Indefrey et al. 2004). In production, one usually cannot bypass syntactic encoding.

Developmental findings suggesting that there are differences in understanding versus producing syntactic structures (Fraser et al. 1963; Clark and Hecht 1983; Bates and Bretherton 1988), indicate that we should leave open the possibility that there are some differences between deconstructing and constructing syntax, but these are not final arguments in favor of such differences. These developmental findings might be due to the fact that we, children as well as adults, can understand a lot without paying attention to syntax. During comprehension, meaning can be derived from purely lexical information and from the context, in combination with general conceptual world knowledge; this is the case for children and also for adults listening to dialects or foreign languages they only know to some extent.

In conclusion, there is an extensive amount of overlap in syntactic decoding and encoding. There are good arguments and evidence that the workspace for the assembly and short-term storage of syntactic structures is shared between processing modalities (Vosse and Kempen 2000; Kempen et al. forthcoming). In the present study, we have shown that there is a shared neural substrate of syntactic encoding in production and syntactic decoding in comprehension. This substrate involves left IFG (BA 45) and left MTG (BA 21). The idea of a shared processor for syntax thus deserves sincere attention in future research.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/

Funding

K.W. was supported by an NWO Toptalent Grant (number 021.001.007).