Functional neuroanatomy of language without speech: An ALE meta-analysis of sign language.

Sign language (SL) conveys linguistic information using gestures instead of sounds. Here, we apply a meta-analytic estimation approach to neuroimaging studies (N = 23; subjects = 316) and ask whether SL comprehension in deaf signers relies on the same primarily left-hemispheric cortical network implicated in spoken and written language (SWL) comprehension in hearing speakers. We show that: (a) SL recruits bilateral fronto-temporo-occipital regions with strong left-lateralization in the posterior inferior frontal gyrus known as Broca's area, mirroring functional asymmetries observed for SWL. (b) Within this SL network, Broca's area constitutes a hub which attributes abstract linguistic information to gestures. (c) SL-specific voxels in Broca's area are also crucially involved in SWL, as confirmed by meta-analytic connectivity modeling using an independent large-scale neuroimaging database. This strongly suggests that the human brain evolved a lateralized language network with a supramodal hub in Broca's area which computes linguistic information independent of speech.

INTRODUCTION

Sign languages (SLs) are natural languages in the visuogestural domain with complex linguistic organization, primarily used by deaf people (Klima et al., 1979; Mathur & Rathmann, 2014; Sandler & Lillo‐Martin, 2008). The extent to which linguistic similarities between SL and spoken and written language (SWL) lead to the recruitment of similar neural resources during language comprehension remains unclear. Several studies suggest that SL may rely on more right‐hemispheric neural components (Bavelier et al., 1998; Campbell, MacSweeney, & Waters, 2007; Emmorey, 2015; Neville et al., 1998; Peperkamp & Mehler, 1999), thereby differing from the canonical left‐lateralized neural response to SWL (Friederici, 2011; Hervé, Zago, Petit, Mazoyer, & Tzourio‐Mazoyer, 2013; Karolis, Corbetta, & Thiebaut de Schotten, 2019; Mazoyer et al., 2014). However, lesion studies indicate that the patterns of SL aphasia align with well‐known aphasic syndromes and occur primarily after left‐hemispheric damage to Broca's or Wernicke's area, two canonical language regions (Atkinson, Marshall, Woll, & Thacker, 2005; Hickok, Bellugi, & Klima, 1998; Poizner, 1987; Poizner, Bellugi, & Klima, 1990).

Neuroimaging studies of SWL and SL converge on a partially overlapping set of cortical regions. During language processing in the auditory and visual domains, Broca's area (Brodmann area [BA] 44 and 45), in the inferior frontal gyrus (IFG) of the left hemisphere, and posterior temporal cortex (so‐called Wernicke's area; usually associated with BA 22) have been shown to be major hubs involved in processing abstract linguistic information such as syntax, semantics, and phonology (Friederici, Chomsky, Berwick, Moro, & Bolhuis, 2017; Hagoort, 2014, 2017; Price, 2010; Walenski, Europa, Caplan, & Thompson, 2019; Zaccarella, Schell, & Friederici, 2017). These amodal regions are supplemented by modality‐specific primary auditory and visual regions that are recruited for lower‐level processing (Buchweitz, Mason, Tomitch, & Just, 2009; Jobard, Vigneau, Mazoyer, & Tzourio‐Mazoyer, 2007; Mesulam, 1998; Walenski et al., 2019) and dynamically interact with other networks, for example, for processing prosody (Meyer, Alter, Friederici, Lohmann, & von Cramon, 2002; Meyer, Steinhauer, Alter, Friederici, & von Cramon, 2004; Sammler, Grosbras, Anwander, Bestelmeyer, & Belin, 2015; van der Burght, Goucha, Friederici, Kreitewolf, & Hartwigsen, 2019) as well as integrating information from co‐speech gesture (Holle, Gunter, Rüschemeyer, Hennenlotter, & Iacoboni, 2008; Jouravlev et al., 2019; Willems, Özyürek, & Hagoort, 2007, 2009). Neuroimaging studies of SL processing have also reported activation of bilateral IFG (Campbell et al., 2007; Emmorey, 2006, 2015; Newman, Supalla, Hauser, Newport, & Bavelier, 2010b), and bilateral posterior temporal cortex (Emmorey, Xu, & Braun, 2011; Petitto et al., 2000) which motivates the hypothesis that left IFG and posterior temporal cortex may constitute hubs in a lateralized modality‐independent core language network. This modality‐independent frontotemporal network may be specialized for the processing of different kinds of abstract linguistic information (Friederici et al., 2017; Hagoort, 2014; Zaccarella et al., 2017). During language processing, this left‐hemispheric core language network dynamically interacts with other networks depending on the modality of language use (spoken, written, or signed).

On this account, observed differences in the neural response between SL and SWL may be inherent to the different modalities in which language is externalized and thus depend upon the different ways in which SL and SWL are routinely studied in experimental settings. The left‐hemispheric dominance for SWL could result from a specialization for the processing of rapidly changing temporal information (Schönwiesner, Rübsamen, & von Cramon, 2005; Zatorre, Belin, & Penhune, 2002), given that the speech stream requires the strictly sequential arrangement of elements. In contrast, SL conveys linguistic information by means of a signer's hands, face, and body—thereby allowing for a certain degree of simultaneous articulation (Cecchetto, 2017; Klima et al., 1979; Lillo‐Martin & Gajewski, 2014; Mathur & Rathmann, 2014; Sandler & Lillo‐Martin, 2008) that may require less left‐hemispheric resources. Furthermore, differences in lateralization between SL and SWL might depend upon differences in the nature of the stimulus material (MacSweeney, Woll, Campbell, Calvert, et al., 2002; MacSweeney, Woll, Campbell, McGuire, et al., 2002; Sakai, Tatsuno, Suzuki, Kimura, & Ichida, 2005): While during face‐to‐face communicative interaction, speech is naturally supplemented by co‐speech gesture (Özyürek, 2014), this additional communicative channel is typically stripped away in experimental studies. That is, materials used in studies of SWL mostly consist of either auditory or written input in isolation not requiring the presence of an actor to be understood (e.g., Fedorenko, Behr, & Kanwisher, 2011; Goucha & Friederici, 2015; Matchin, Hammerly, & Lau, 2017; Pallier, Devauchelle, & Dehaene, 2011; Schoffelen et al., 2019), whereas SL stimuli necessarily require the visual presentation of an actor carrying out the signing. The presence or absence of this communicative channel in the stimulus material may impose additional processing demands in terms of visual, spatial, and social cognition which, similar to the processing of discourse information and prosody, involve primarily right‐hemispheric networks (Caplan & Dapretto, 2001; Kuperberg, Lakshmanan, Caplan, & Holcomb, 2006; Meyer et al., 2002, 2004; Sammler et al., 2015; van der Burght et al., 2019; Jiang Xu, Kemeny, Park, Frattali, & Braun, 2005).

Here, we shed light on these issues by quantitatively reviewing functional neuroimaging data for SL processing in a meta‐analytical fashion. We employed an activation likelihood estimation (ALE) approach (Eickhoff et al., 2009; Eickhoff, Bzdok, Laird, Kurth, & Fox, 2012; Eickhoff, Laird, Fox, Lancaster, & Fox, 2017; Turkeltaub et al., 2012) to identify brain regions showing consistent responses to SL comprehension across 23 different functional magnetic resonance imaging and positron emission tomography studies with more than 300 deaf signers as participants. In neuroimaging, ALE is the most frequently used meta‐analysis technique as it combines low susceptibility to false positives with high reproducibility and comparability of results (Müller et al., 2018). This is achieved by modeling activation foci as centroids of a Gaussian probability distribution which is weighted by sample size and then iteratively summed. Therefore, ALE tests for the statistically reliable clustering of brain activations in a standardized space (Eickhoff et al., 2017), avoiding errors of spatial distinctions and inconsistencies in labeling across studies that may have affected qualitative reviews. We further evaluated the neuroanatomical localization of the thus identified SL network using mass‐overlap analysis and by computing lateralization indices for the observed convergence mass. To distinguish the neural response to linguistic information from the response to other visuospatial properties of the stimuli, we contrasted the ALE analysis of SL comprehension against a second independent ALE analysis of nonlinguistic sign‐like actions (SLA). The resulting convergence map was then supplemented by meta‐analytic connectivity modeling (MACM; Laird et al., 2011) across BrainMap (Fox & Lancaster, 2002), a large‐scale database of neuroimaging studies, to establish robust functional associations for those voxels that were specifically involved in processing the linguistic aspects of SL stimuli in deaf signers. These functional roles and coactivation patterns of SL‐specific voxels in studies with hearing nonsigners were then used to determine the overlap between the identified SL network and the canonical SWL network.

MATERIALS AND METHODS

This systematic review and meta‐analysis was conducted and is reported according to the PRISMA guidelines (Moher et al., 2009) and the guidelines for neuroimaging meta‐analyses (Müller et al., 2018). Details on search strategy, study selection, quality assessment, and data extraction are presented in Appendix A. Appendix B shows a flow diagram of the process in line with PRISMA guidelines. Appendix C includes our responses to the recommended checklist for neuroimaging meta‐analysis (Müller et al., 2018). Appendix D provides a summary of the studies included in the dataset for SL comprehension analyzed in this study. Appendix E provides a summary of studies included in the dataset of SLA observation.

ALE for SL comprehension

All coordinates from studies reporting activation foci in Talairach space (Talairach & Tournoux, 1988) were converted to MNI space using the Lancaster transform as implemented in the GingerALE toolbox (Eickhoff et al., 2009, 2012; Turkeltaub et al., 2012). Data conversion and the further ALE analyses were carried out using GingerALE version 2.3.6 (Eickhoff et al., 2017), available from https://brainmap.org/ale. In order to correct for within‐experiment effects derived from foci proximity, we performed our analyses using the more conservative Turkeltaub ALE method (Turkeltaub et al., 2012). As is customary for ALE analyses of functional data, we used GingerALE's more conservative gray matter mask. Recommended thresholds of p < .001 (cluster‐forming threshold) and .05 for cluster‐level family‐wise error with 10,000 thresholding permutations were applied to output images (Müller et al., 2018). Anatomical labels were obtained by inputting peak coordinates from our cluster analysis into the SPM Anatomy Toolbox version 2.2c (Eickhoff et al., 2005, 2007). For peaks in regions not covered by the current version of the Anatomy Toolbox the corresponding BA was determined using the MNI‐BA map included in the Yale BioImaging Suite Web version 1.0.0 (Lacadie, Fulbright, Arora, Constable, & Papademetris, 2008), accessible at https://bioimagesuiteweb.github.io. For this analysis, the SL comprehension foci dataset was used as input to the ALE algorithm to observe spatial convergence for SL comprehension in deaf signers across studies. Input and output files are available as part of the online material for this study.

Lateralization indices (cross‐hemispheric and IFG)

Functional hemispheric lateralization of the convergence map for SL comprehension from the first ALE analysis was determined by computing a weighted laterality index (LI), the so‐called AveLI (Matsuo, Chen, & Tseng, 2012). An LI value represents the degree of lateralization across all voxels with positive values of a given test statistics within chosen volumes of interest (VOIs). The classical approach to computing a LI first counts the number of voxels that exceed a set threshold in a VOI in each hemisphere. As a second step, the index is computed according to the standard formula: (left − right)/(left + right). Here, we employed the so‐called AveLI as an average of all sub‐LIs computed by setting the threshold at the (positive) value of a test statistic in each voxel within the respective VOI. Therefore, AveLI has the advantage of weighting each voxel by its value in the test statistics in a data‐driven manner. In this study, we report AveLIs, ranging from 1 (completely left‐lateralized) to −1 (completely right‐lateralized). These were computed using the AveLI script (version April 3, 2017), available from http://aveli.web.fc2.com. Significance of AveLI scores was determined by implementing a permutation test: The values (i.e., ALE scores) of all nonzero voxels within the input masks were randomly reassigned within the masks across hemispheres and input to the AveLI script. After 1,000 permutations, the data were standardized, and the z‐score converted to a p‐value to determine if the actually observed AveLI differed significantly from the permuted data.

We first created VOIs covering either the entire left or right hemisphere by dividing the standard space of our template (Colin27_T1_seg_MNI_2x2x2.nii) into identical halves. These latter volumes were used to assess hemispheric asymmetry of the whole brain during SL comprehension. Our initial aim was to assess the lateralization of functional clusters in the hypothesized left‐hemispheric modality‐independent frontotemporal core language network and homologous right‐hemispheric regions. However, as we only observed functional convergence for SL comprehension in bilateral IFG but not bilateral posterior temporal cortex we restricted further analyses to language‐relevant subregions of the IFG. To characterize the lateralization of functional clusters in neuroanatomical terms in the IFG, we performed our analyses using probabilistic cytoarchitectonic maps of Broca's area, which consists of the posterior area 44 and the more anterior area 45 (Amunts et al., 2010; Zilles & Amunts, 2018). These cytoarchitectonically defined VOIs (Amunts et al., 1999) were extracted from the SPM Anatomy Toolbox (Eickhoff et al., 2005, 2007). In addition, we created a composite mask of Broca's area and its right‐hemispheric homolog by combining the maps for areas 44 and 45. Details of the anatomical VOIs are described in Appendix F. These VOIs were used as primary VOIs in for the lateralization analyses in IFG.

Mass overlap analysis in the IFG

The functional convergence map obtained from the ALE analysis of SL comprehension was used to perform a mass overlap analysis in bilateral IFG to further explore the subregional spatial distribution of these functional clusters. Neuroimaging studies of SWL over the past decades have yielded a number of models with regard to the neuroanatomical basis of language, which have all highlighted the role of left IFG and especially Broca's area as a core language‐related region embedded in a wider language network (Friederici, 2011; Friederici et al., 2017; Hagoort, 2017). Similar results have been reported for studies of SL, whereas many studies also report involvement of the right hemisphere and right IFG in SL comprehension (Campbell et al., 2007; Emmorey, 2015; MacSweeney, Capek, Campbell, & Woll, 2008). As mentioned above, we only observed bilateral functional convergence in posterior IFG as the only directly language‐relevant region which is why this analysis was restricted to Broca's area and its subregions as well as right‐hemisphere homologs. To achieve an accurate neuroanatomical characterization of the distribution mass observed in IFG, the cytoarchitectonically defined VOIs of left and right area 44, area 45, and the composite masks of Broca's area and its right‐hemispheric homolog (see Appendix F) were used as volumes for this analysis. The respective VOIs were first transformed into the space of our MNI template (Colin27_T1_seg_MNI_2x2x2.nii) and then multiplied with the left or right IFG clusters extracted from the convergence map.

ALE for SLA observation

In this analysis, we tested the SLA observation dataset derived from an independent meta‐analysis (Papitto, Friederici, & Zaccarella, 2020), as described in Appendix A. Here, we sought to determine brain regions recruited in (hearing) subjects during processing of visual stimuli showing humans performing manual and facial actions devoid of linguistic content. The parameters of this ALE analysis were identical to those reported ALE analysis of SL comprehension, except for the input dataset.

Contrast and conjunction analysis between SL comprehension and SLA observation

The goal was to identify unique and overlapping voxels in the two datasets that had previously entered into the two ALE analyses. By contrasting these two datasets from groups with different hearing status we adapt an established practice in the field (e.g., Bavelier et al., 1998; Capek et al., 2010; MacSweeney, Woll, Campbell, Calvert, et al., 2002; MacSweeney, Woll, Campbell, McGuire, et al., 2002; Neville et al., 1998) to a meta‐analytic level while, at the same time, ensuring that the observed stimulus materials were maximally similar (see Appendix A). The contrast SL comprehension > SLA observation revealed voxels being specific to the linguistic aspects of SL comprehension, whereas the inverse contrast showed voxels specific to SLA observation. The conjunction analysis identified voxels where both datasets showed convergence, thereby indicating nonlinguistic regions recruited for processing visuospatial properties of the stimuli. All analyses were performed in GingerALE. Contrast analysis in GingerALE is implemented in a way that corrects for study sizes by permuting the data (we used 10,000 thresholding permutations). The analysis results in contrast images that report z scores showing significance instead of p values, which would be more difficult to interpret in this context. Standard thresholds of p < .001 (cluster‐forming threshold), p < .05 for cluster‐level inference, and a minimum cluster size of 100 mm3 were then applied to these output images. Different from the statistical procedure used for contrast analysis, conjunction analysis as implemented in GingerALE simply uses voxel‐wise minimum value of the input ALE images to create the output image.

Meta‐analytic connectivity modeling

To meta‐analytically characterize the functional attributions and connectivity of the brain regions uniquely involved in SL comprehension identified in the contrast analysis, we performed MACM using the BrainMap database, adopting the established procedure by Laird et al. (2011) for investigating whole‐brain coactivation patterns across a range of tasks. This approach allowed us to determine the overlap between the identified SL network and the canonical SWL network, without the necessity to perform a statistically unwarranted direct comparison of the SL comprehension dataset to an inevitably unproportionally larger set of studies of SWL comprehension. After binarizing the contrast images, we extracted individual clusters, resliced the resulting images to the 1 × 1 × 1 voxel size resolution required by Sleuth and used them as input to different searches in the BrainMap database. At the time of search (March 11, 2019), the database contained the results of 3,406 papers describing 16,901 experiments. All database queries used Sleuth version 2.4, available from http://brainmap.org/sleuth, and were restricted to (a) the location as defined by nonzero voxels in a cluster from the respective contrast from the contrast analysis; (b) healthy subjects by setting the parameter “Experiments: Context” IS “Normal Mapping”; and (c) only activations and no deactivations by requiring that “Experiments: Activations” IS “Activations Only.”

The meta‐analytic connectivity of a given VOI across studies listed in the BrainMap database was determined by performing an ALE analysis using the whole‐brain foci data from experiments in the database that have shown coactivation with the respective VOI. These additional ALE analyses were carried out with the exact same parameters described for the primary analysis of the SL comprehension dataset. Because of our reliance on the Turkeltaub ALE method, which corrects for within‐experiment effects (Turkeltaub et al., 2012), search results were exported by subject group and duplicates were removed, if necessary. In total, four different queries and consecutive ALE analyses were performed using the following clusters obtained in in the contrast analysis: Three clusters associated with SL comprehension in left IFG, right STG, and left middle frontal and precentral gyrus (MFG/PCG); as well as one cluster in left PCG associated with SLA observation.

RESULTS

Neural response during SL comprehension

The ALE results for our dataset of studies of SL comprehension revealed seven significant clusters in both hemispheres (Figure 1; table Appendix G; the clusters will be described following their size). The first and largest cluster spanned over the pars opercularis (BA 44) and pars triangularis (BA 45), both subregions of Broca's area, with peaks in both regions. The second cluster comprised the posterior portions of middle and inferior temporal gyri (BA 37) in the right hemisphere. A third cluster was found in right pars triangularis (BA 45). The fourth cluster comprised the left middle occipital gyrus (BA 19). A fifth cluster was located in right superior temporal gyrus (STG; BA 22). Finally, a sixth cluster included the PCG (BA 6) and MFG (BA 8) with peaks in both regions, whereas a seventh cluster was confined to the left anterior insula. Unlike expected, we did not observe a cluster in left posterior temporal gyrus which is why detailed analyses of the convergence mass were limited to the IFG as the only potentially language‐related region where clusters were also observed bilaterally.

FIGURE 1

Activation likelihood estimation (ALE) map of significant clusters associated with sign language (SL) comprehension, superimposed onto a standard cortical surface of the left and right hemisphere (LH/RH). Convergence mass of the ALE analysis for all “SL comprehension > control/baseline” contrasts with above‐chance overlap (p < .05, cluster‐level family‐wise error [cFWE] corrected) is shown. The color bar indicates the ALE score of any given voxel which represents the degree of nonrandom convergence in activation between contrasts in the dataset

When using hemispheric masks, a significant global left‐lateralization (AveLI: 0.24, p < .001) of the convergence mass for SL comprehension could be observed (Figure 2). Lateralization indices for SL comprehension in the IFG revealed a strong left‐lateralization with an AveLI of 0.68 (p < .001) in Broca's area (BA 44 and 45). Subregional analyses showed a very pronounced left‐lateralization in area 44 (AveLI: 0.78, p < .001) and a slightly less‐pronounced significant left‐lateralization in area 45 (AveLI: 0.54, p < .001). These findings indicate that the left hemisphere is generally dominant for SL comprehension. This holds true when Broca's area is specifically investigated, whereas the lateralization is most pronounced in left area 44.

FIGURE 2

Lateralization indices for sign language (SL) comprehension. (a) AveLI for SL versus mean AveLI after 1,000 permutations. Lateralization indices were computed using different anatomical masks: BA 44 (green), BA 45 (blue), Broca's area (red) as a composite of BA 44 and 45, and hemispheric masks. Further details of volumes of interest (VOIs) are described in Appendix F. The reported AveLI indices may range from 1 (completely left‐lateralized) to −1 (completely right‐lateralized). Significant differences of AveLI from the mean (i.e., no lateralization) in the permutation test are indicated using common significance levels: * for p < .05, ** for p < .01, and *** for p < .001. All VOIs showed significant left‐lateralization for SL comprehension, whereas the difference was most pronounced in BA 44. (b) Activation likelihood estimation (ALE) score distribution for SL comprehension in BA 44. Left panel: Anatomical masks for BA 44 in both hemispheres and the observed convergence cluster for SL comprehension. Right panel: Shuffled permutation showing the random redistribution of observed ALE scores within the masks across hemispheres

Results aligned with the those of the lateralization analyses reported above. We found that 92.98% of the whole convergence mass for SL comprehension in this analysis fell within the anatomical mask (see Appendix F) of Broca's area in the left IFG, while it reached 98.26% in the right IFG. Within Broca's area, 36.78% of the convergence mass could be unambiguously assigned to area 44 and 15.44% to area 45 (Figure 3a,d–f; table Appendix H). The largest portion of the left IFG cluster (47.78%) sat at the intersection between the two areas so that it could not unambiguously be assigned to either region due to interindividual variability captured by the cytoarchitectonic mask. Within the right homolog of Broca's area, 53.55% could be unambiguously assigned to area 45 (Figure 3c,f–h), while the rest of the cluster (46.45%) fell into the intersection of areas 44 and 45. No voxels could unambiguously be assigned to right area 44. Replicating the analysis with more conservatively thresholded VOIs confirmed the involvement of left area 44 but not right area 44 in SL comprehension (Appendix I).

FIGURE 3

Mass overlap analysis in inferior frontal gyrus (IFG) for convergence map of the activation likelihood estimation (ALE) analysis for sign language (SL) comprehension. (a) Sagittal plane at x = −50 showing the largest cluster (2,336 mm3) in left IFG spanning areas 44 and 45. (b) Cytoarchitectonic maps for areas 44 (green) and 45 (blue) in both hemispheres extracted from the SPM Anatomy Toolbox. (c) Sagittal plane at x = 54 showing the cluster in right IFG constrained to area 45. (d) Coronal plane of the left hemisphere at y = 14. (e) Transverse plane of the left hemisphere at z = 26. (f) Mass overlap of functional clusters with anatomical regions in % of total mass within volume of interest (VOI). (g) Transverse plane of the right hemisphere at z = 26. (h) Coronal plane of the right hemisphere at y = 20

Contrast and conjunction analyses between SL comprehension and SLA observation

The ALE analysis of an independent dataset of studies in which nonsigners observed nonlinguistic SLA showed significant clusters in both hemispheres (Figure 4 and table Appendix J). For the contrast SL comprehension > SLA observation, we found convergent clusters in left IFG with peak in BA 44, in the right superior temporal gyrus (BA 22), as well in the left MFG and PCG (Table 1). The inverse comparison, SLA observation > SL comprehension, revealed the following regions to be involved in action observation independently of SL comprehension (table Appendix K): A large cluster in the left superior and inferior parietal lobules (BA 40 and 7) extending into postcentral gyrus (BA 1), and a second large cluster in left PCG (BA 6) extending in the dorsal most portion of the maximum probability map of BA 44 (see Appendix F). In addition, we observed clusters surviving the comparison in right middle temporal gyrus (MTG; BA 37), right inferior temporal and middle occipital gyri (BA 19), left MTG (BA 21 and 39), right IFG (BA 44), left inferior occipital gyrus (BA 19), right superior parietal lobule (BA 7), right STG (BA 40), and right PCG (BA 6). The conjunction analysis revealed an overlap between the convergence mass in both datasets in bilateral MTG (BA19) extending into inferior temporal gyrus and inferior occipital gyrus, right IFG (BA 45), right MTG (BA 21), and left PCG (BA 6) extending into the most dorsal portion of BA 44 (table Appendix L).

FIGURE 4

Activation likelihood estimation (ALE) map of significant clusters associated with sign‐like action (SLA) observation, superimposed onto a standard cortical surface of the left and right hemisphere (LH/RH). Convergence mass of the ALE analysis for all “SLA observation > control/baseline” contrasts with above‐chance overlap (p < .05, cluster‐level family‐wise error [cFWE] corrected) is shown. The color bar indicates the ALE score of any given voxel which represents the degree of nonrandom convergence in activation between contrasts in the dataset

TABLE 1

Results of “SL comprehension > SLA observation” comparison, revealing significant clusters specific to SL comprehension after subtracting SLA observation. Mask dimensions = 77 × 96 × 79; number of within‐brain voxels = 229,781; thresholding method = uncorrected p‐value; thresholding value = .05; volume > threshold = 3,872 mm3; minimum cluster size = 100 mm3

Cluster	Hemisphere	Brain region	BA	MNI coordinates (mm)			Z score	Cluster size (mm3)
				x	y	z
1	Left	Inferior frontal gyrus	44	−42.7	16.7	30.7	3.72	2,336
2	Right	Superior temporal gyrus	22	44	−32	2	3.43	920
	Right	Superior temporal gyrus	22	42	−34	6	3.19
3	Left	Middle frontal gyrus	6	−42	6	52	3.29	616
	Left	Middle frontal gyrus	44/8	−40	12	42	1.92
	Left	Precentral gyrus	44/6	−40	8	42	1.91

Abbreviations: BA, Brodmann area; SL, sign language; SLA, sign‐like action.

The first Sleuth query using the SL‐specific left IFG cluster from the contrast analysis (Figure 5a,b) returned hits for 342 papers with 363 experiments that report data from a total of 5,448 subjects and 6,392 locations matching our search criteria. Across all studies in the BrainMap database, the voxels in the left IFG cluster involved in SL comprehension in our analysis were functionally primarily associated with studies from the domain of cognition (Figure 5c). Within this domain, the vast majority of studies reporting peaks in the location of our cluster were categorized as language studies with a smaller amount of experiments pertaining to attention and memory (Figure 5d). Within the language subdomain, studies with peaks in the same voxels included in our left IFG cluster active for SL comprehension were mostly categorized as having investigated phonology (14% of studies in language subdomain), semantics (46%), and speech (27.2%). See Appendix M for further details on the behavioral domains associated with this cluster in the BrainMap database. The ALE analysis revealed that the strongest convergence of foci for coactivation with the left IFG cluster derived from SL comprehension was observed in the core and extended language network, including bilateral IFG, insula, MFG and PCG, middle temporal gyrus, as well as inferior parietal lobule, thalamus and other subcortical structures (table Appendix N). In all these regions there, was a clear left–right asymmetry with regard to cluster size and extent, that was especially pronounced in Broca's area, left MFG (BA 10), left PCG, as well as left posterior temporal cortex and superior parietal lobule.

FIGURE 5

Sign language‐specific voxels (resulting from the “sign language (SL) comprehension > SLA observation” contrast) in Broca's area and their function in nonsigners according to the BrainMap database which contained the results of 3,406 papers describing 16,901 experiments at the time of search. (a) Transverse plane of the left hemisphere at z = 14, showing voxels in left inferior frontal gyrus (IFG) spanning BA 44 and BA 45 which survived the “SL comprehension > SLA observation” contrast. (b) Sagittal plane at x = −48 showing this largest SL‐specific cluster (2,336 mm3) in left IFG spanning BA 44 and BA 45. (c) Number of studies with nonsigners in the BrainMap database that report peaks in voxels of the SL‐specific cluster in left IFG (BA 44 and 45), organized by behavioral domain. For details, see Appendix M. (d) Number of studies with nonsigners in the BrainMap database that report peaks in voxels of the SL‐specific cluster in left IFG (BA 44 and 45) organized by behavioral subdomain within the domain of cognition. For details, see Appendix M

Three additional MACM analyses were performed using the remaining SL‐specific clusters located in (a) right STG, (b) left MFG/PCG (Table 1), as well as (c) the SLA‐specific cluster in left PCG (table Appendix K) as seeds. As for (a), SL‐specific voxels in right STG were primarily associated with the domains of cognition (subdomain: language) and auditory processing (Appendix O). The corresponding ALE analysis showed the strongest convergence of foci for coactivation with the SL‐specific right STG cluster in right STG (BA 22), left MTG and STG (BA 21 and BA 22), as well as bilateral IFG (left BA 45 and right BA 44 and 45; see Appendix P). With regard to (b), convergence in the voxels of the SL‐specific left MFG/PCG cluster was primarily associated with the domain of cognition and especially language but also action execution (Appendix Q). The ALE analysis revealed coactivation across studies with this SL‐specific cluster in left precentral and MFG (BA 6 and BA 9), a cluster in the interhemispheric cleft extending into posterior‐medial frontal regions and middle cingulate cortex, right MFG and IFG (BA 8 and BA 45), left IFG (BA 44 and 45), left superior parietal lobule, left thalamus, and bilateral insula (Appendix R). As for (c), SLA‐specific voxels the left PCG cluster were mostly associated with the domains of cognition, action, and perception (Appendix S). The ALE analysis showed coactivation with this cluster in bilateral IFG and MFG (BA 44, 45, 6, and 9), cingulate cortex (BA 32), bilateral inferior and superior parietal lobules including supramarginal and angular gyri (BA 7, 39, and 40), thalamus and pallidum, bilateral inferior temporal gyrus (BA 37), bilateral insula, and right middle occipital gyrus (BA 18 and 19; see Appendix T).

DISCUSSION

There are three important findings in the present study. First, we have shown that deaf signers process SL in bilateral fronto‐occipito‐temporal networks, involving Broca's area (left BA 44 and 45) and its right‐hemispheric homolog, right STG (BA 22), as well as left premotor cortex (BA 6 and 8) and insula. These regions have been associated with different aspects of SWL processing in previous works (Friederici, 2011; Friederici et al., 2017; Hagoort, 2014, 2017; Price, 2010; Walenski et al., 2019; Zaccarella et al., 2017; Zaccarella & Friederici, 2015). Convergence in bilateral fusiform gyrus (BA 37) and middle occipital gyrus (BA 19), regions associated with different aspects of visual processing (Weiner & Zilles, 2016), was found for SL comprehension as well as SLA observation. Secondly, we find that the left hemisphere is generally dominant for SL comprehension, with robust involvement of left but not right BA 44. Convergence in insula and PCG was also left‐lateralized. Thirdly, the contrast analysis between SL comprehension and SLA observation revealed convergent clusters specific to SL in Broca's area with the peak in BA 44, in the right superior temporal gyrus (BA 22), as well in the left MFG/PCG. These regions thus appear to be the key regions which deaf signers recruit for attributing linguistic information to stimuli showing manual, facial, and bodily gestures that carry linguistic structure and meaning in their respective SL. In contrast to Broca's area in the left IFG, its homolog in right IFG shows cluster convergence for both, SL comprehension and SLA observation. The MACM analysis confirmed that the SL‐specific voxels in Broca's area observed in deaf signers are recruited for SWL processing in hearing nonsigners and co‐activate with the classical and extended cortical language network (Friederici, 2011; Friederici et al., 2017; Hagoort, 2014, 2017), including subcortical language‐relevant regions.

Lateralization of SL comprehension

When looking at the whole brain, SL comprehension recruits both hemispheres, yet exhibits significant left‐lateralization. This pattern suggests that the left‐hemispheric dominance for language is independent of the modality of language use (Corina, San Jose‐Robertson, Guillemin, High, & Braun, 2003; Hickok et al., 1998; Hickok, Bellugi, & Klima, 1996; Mazoyer et al., 2014; Poizner et al., 1990). If one assumes that the left‐hemispheric dominance for SWL processing constitutes a specialization for processing of rapidly changing temporal information as conveyed by the speech stream (Schönwiesner et al., 2005; Zatorre et al., 2002) this finding is surprising. In contrast, our results suggest that the left hemisphere is not specialized for SWL, but linguistic processing more generally. This is in line with the more recent observation that speech processing is actually bihemispheric, whereas the processing of abstract linguistic information is left‐lateralized and especially recruits left IFG (Bozic, Tyler, Ives, Randall, & Marslen‐Wilson, 2010).

Functional asymmetry in IFG

We found bilateral convergence in the IFG for SL comprehension with substantial differences in the subregional distribution of the convergence mass in Broca's area and its right‐hemispheric homolog. Only left but not right BA 44 showed convergence for SL comprehension, despite the fact that this region is part of the bilateral mirror network (Kilner, Neal, Weiskopf, Friston, & Frith, 2009). We take this to indicate that deaf signers specifically recruit portions of left BA 44 and BA 45 for the processing of abstract linguistic information. Generally, the response to SL is lateralized in Broca's area and the distribution of convergence mass exhibits a hemispheric asymmetry in IFG for SL comprehension similar to that observed for processing of SWL (Hervé et al., 2013; Karolis et al., 2019; Mazoyer et al., 2014). Interestingly, such language‐related functional asymmetry in IFG has a potential neuroanatomical basis in the asymmetry of the pars opercularis (BA 44; Amunts et al., 1999; Keller et al., 2007; Zilles & Amunts, 2018). Furthermore, it is reflected in receptor architectonics of language regions interacting during SWL comprehension in the left hemisphere, but not in the right hemisphere (Zilles, Bacha‐Trams, Palomero‐Gallagher, Amunts, & Friederici, 2015).

Functional specialization in the (sign) language network

Contrasting spatial convergence across studies for our separate ALE analysis of SLA observation to our initial ALE analysis of SL comprehension revealed that only the convergence mass in left IFG (BA 44 and 45), right STG (BA22), and left PCG/MFG (BA 6 and 8) survived the statistical comparison. This suggests that these regions do not process stimulus properties or aspects of manual, facial, and bodily gestures but are specifically recruited by deaf signers for attributing linguistic information to the observed gestures. Convergence in right IFG, parts of right STG, bilateral insula cortex, as well as bilateral occipital and posterior middle and inferior temporal gyrus was also observed during observation of SLA in hearing nonsigners, indicating that these regions subserve perceptual as well as other nonlinguistic processes involved in action observation (Papitto et al., 2020). Interestingly, such a functional specialization of the left hemisphere and especially the left IFG for processing abstract linguistic information has also been reported in studies of the comprehension of conventionalized symbolic gesture by hearing nonsigners (Özyürek, 2014; Xu, Gannon, Emmorey, Smith, & Braun, 2009), thereby hinting at the supramodal nature of the brain's core network for language processing independent of stimulus properties.

Functional specialization in IFG

Our comparison of SL comprehension with the neural response to SLA points toward a different functional specialization in Broca's area and its right‐hemispheric homolog: The convergence in Broca's area reflects the processing of linguistic aspects of SL stimuli (Clos, Amunts, Laird, Fox, & Eickhoff, 2013; Sakai et al., 2005), whereas the right IFG may be involved in the processing of nonlinguistic properties of the stimulus material such as visuospatial aspects of the performed actions as well as social information about the actor (Hartwigsen, Neef, Camilleri, Margulies, & Eickhoff, 2019). Hence, the involvement of the right IFG during SL comprehension is not specific to language processing but instead appears to reflect stimulus‐specific processing demands imposed by the visuogestural modality. Such differential involvement according to the linguistic status of the stimulus material is not predicted under a mirror neuron hypothesis (see Tettamanti & Moro, 2012 for related discussion) and points to a functional specialization for the processing of abstract linguistic information in subregions of the left IFG. Our MACM analysis strongly confirms this functional role assigned to the left IFG: The majority of the SL‐specific voxels in the left IFG cluster are found active in studies of SWL in hearing nonsigners (Figure 5c,d), specifically during processing of phonology, semantics, and speech. Notably, these SL‐specific voxels co‐activate with the core and extended language network across studies of SWL.

Subregional specializations in MFG and PCG

We observed spatial convergence across studies in left PCG extending into MFG during both SL comprehension and SLA observation. The MFG/PCG cluster observed during SL comprehension was found to be primarily associated with the domain of cognition (language, explicit memory, and working memory) and to a lesser extent with action execution. In contrast, the left PCG cluster observed during SLA observation was also associated with cognition but showed a strong association with action execution and inhibition, as well as shape and motion perception. In the MACM analyses, both clusters exhibited differential coactivation patterns which may reflect different subregional specializations: Voxels in the MFG/PCG cluster supplement language processing by interacting with Broca's area and other frontal regions, whereas the voxels active during SLA observation are part of more generic and bilateral networks underlying action processing.

The role of posterior temporal cortex

A cluster in the right STG located in BA 22 (the right‐hemispheric homolog of so‐called Wernicke's area) was also found to be specific to SL comprehension. In studies of SWL, bilateral temporal cortex has been implicated in speech and single word processing (Bozic et al., 2010; Hickok & Poeppel, 2007) and right‐hemispheric activation more generally has been linked to processing of suprasegmental information such as prosody and discourse information (Kuperberg et al., 2006; Meyer et al., 2002, 2004; Sammler et al., 2015; van der Burght et al., 2019; Jiang Xu et al., 2005). In deaf signers, right‐hemispheric activation has also been linked to processing of suprasegmental and discourse level information (Newman, Supalla, Hauser, Newport, & Bavelier, 2010a). Therefore, the observed convergence in right STG may result from the complexity of stimuli used in the studies in our dataset which ranged from single signs to complex discourse and reflect single sign processing analogous to single word processing or, alternatively, discourse‐level processes (Hickok & Poeppel, 2007; Newman et al., 2010a). The MACM analysis shows that voxels within this cluster co‐activate with left STG, and MTG, as well as bilateral IFG. Functionally, these voxels were linked to SWL processing and auditory perception, suggesting a certain plasticity and functional reorganization in this region as a consequence of deafness or SL acquisition (Emmorey, Allen, Bruss, Schenker, & Damasio, 2003).

The lack of spatial convergence across studies in left temporal cortex during SL comprehension raises the question whether left posterior temporal cortex actually enables access to abstract lexical/thematic information (Zaccarella et al., 2017) and high‐level semantic integration (Binder, Desai, Graves, & Conant, 2009; Xu et al., 2009) in an amodal manner, as we hypothesized. Previous individual studies have frequently found involvement of bilateral posterior temporal cortex in SL processing (Campbell et al., 2007; Emmorey, 2015; Emmorey et al., 2011; MacSweeney et al., 2008; Petitto et al., 2000). The region's functional organization could either be modulated by language modality (Capek et al., 2008), or may exhibit substantial interindividual variation because of plastic functional reorganization due to deafness (Emmorey et al., 2003). To reveal the underlying neuroanatomical organizational principles and their relation to the modality of language use, future studies should attempt to link the structural connectivity of Broca's area as a modality‐independent hub in the language network to posterior and middle temporal cortices (Finkl et al., 2019) with functional measures.

Toward a functional neuroanatomy of SL

The following model emerges from our results: SL processing in deaf signers recruits bilaterial frontotemporal networks with distinct functional specializations, especially in IFG. In the left hemisphere, IFG and PCG/MFG are specialized for the processing of abstract linguistic information, as revealed by the present meta‐analysis. Middle and superior temporal gyri, supramarignal gyrus, and bilateral parietal lobule have been shown to also be involved in SL processing in previous works (Campbell et al., 2007; Corina et al., 1999; Emmorey, 2015; Emmorey, McCullough, Mehta, Ponto, & Grabowski, 2013; Emmorey, Mehta, McCullough, & Grabowski, 2016; MacSweeney et al., 2008). In the right hemisphere, posterior STG also processes linguistic information, whereas right IFG and MTG subserve the processing of modality‐specific information during SL comprehension.

OUTLOOK

SL offers a unique opportunity to probe hypotheses regarding the human capacity for language independent of speech. An increasing interest in SL in the neuroimaging community will hopefully make it possible in the future to perform more fine‐grained meta‐analyses of SL comprehension which could help to characterize different linguistic subsystems, as have previously been identified for SWL (Zaccarella et al., 2017). For example, different subregions of Broca's area may be differentially involved in processing semantic as opposed to syntactic information and vice versa also for SL. Similarly, an increase in the number of studies investigating modality‐specific aspects of SL such as, for example, the syntactic as well as topographic use of space (Emmorey et al., 2013; Klima et al., 1979; MacSweeney, Woll, Campbell, Calvert, et al., 2002; MacSweeney, Woll, Campbell, McGuire, et al., 2002) will make it possible to meta‐analytically characterize the interactions of regions in the primarily left‐hemispheric core language network identified in this study with other networks during SL processing in such cases where SL and SWL diverge (Campbell et al., 2007; MacSweeney et al., 2008).

CONCLUSION

We propose that Broca's area (BA 44 and 45) in the left IFG constitutes a hub in the language network that is being recruited during the processing of linguistic information regardless of the modality of language use (spoken, written, or signed). This strongly suggests that the human brain evolved a lateralized core language network specialized for carrying out linguistic computations with a hub in Broca's area, which is not specific to speech but instead may flexibly interface with different externalization systems depending on the modality of language use (Berwick, Friederici, Chomsky, & Bolhuis, 2013).

CONFLICT OF INTEREST

The authors declare no competing interests.

AUTHOR CONTRIBUTIONS

Patrick C. Trettenbrein, Angela D. Friederici, and Emiliano Zaccarella: Conceptualized study. Patrick C. Trettenbrein and Giorgio Papitto: Collected data. Patrick C. Trettenbrein and Emiliano Zaccarella: Analyzed data. Patrick C. Trettenbrein and Emiliano Zaccarella: Wrote manuscript. All authors discussed results, revised manuscript, designed figures, and approved the manuscript for publication.

Appendix S1. Supporting Information.

Click here for additional data file.