Atypical perception of affective prosody in Autism Spectrum Disorder.

Autism Spectrum Disorder (ASD) is characterized by impairments in language and social-emotional cognition. Yet, findings of emotion recognition from affective prosody in individuals with ASD are inconsistent. This study investigated emotion recognition and neural processing of affective prosody in high-functioning adults with ASD relative to neurotypical (NT) adults. Individuals with ASD showed mostly typical brain activation of the fronto-temporal and subcortical brain regions in response to affective prosody. Yet, the ASD group showed a trend towards increased activation of the right caudate during processing of affective prosody and rated the emotional intensity lower than NT individuals. This is likely associated with increased attentional task demands in this group, which might contribute to social-emotional impairments.

Introduction

To humans, voices bear a special significance (Blasi et al., 2011). Besides communicating verbal content, voices also communicate extra-verbal information, allowing for inferences about the intentions and emotional states of the speaker. Meanwhile, language impairments and difficulties with social and emotional communication are key characteristics of Autism Spectrum Disorders (ASD) (APA, 2000; Lord et al., 2000). Delayed language development is one of the earliest signs of ASD (De Giacomo and Fombonne, 1998; Wetherby et al., 2004), and language abnormalities, such as abnormal tone of voice or atypical stress patterns, ranging from monotonic, emotion-less speech to exaggerated intonation, pitch or volume affect large proportions of individuals with ASD throughout life (Ghaziuddin and Gerstein, 1996; Shriberg et al., 2001; Simmons and Baltaxe, 1975). Prosodic impairments are part of most clinical screening instruments for ASD (Lord et al., 2000; Lord et al., 1994; Sparrow et al., 1984), and there is a strong correlation between prosodic abnormalities and social and communicational difficulties in people with ASD (Paul et al., 2005). Thus, better knowledge of language processing and in particular processing of affective prosody in individuals with ASD is central for a better understanding of their impairments in social–emotional communication.

Emotions in speech are conveyed through affective prosody, which consists of variations in pitch, intensity, and duration (Fruhholz et al., 2012). In neurotypical (NT) individuals, language and specifically affective prosody are processed in fronto-temporal brain networks, including the temporal regions along the superior temporal gyrus/sulcus, and frontal regions in the inferior frontal gyrus and orbitofrontal gyrus (Buchanan et al., 2000a; Fruhholz and Grandjean, 2012; Kotz et al., 2013; Leitman et al., 2010; Schirmer and Kotz, 2006). In addition to this, affective prosody is associated with activity in subcortical brain structures, such as the amygdala and the basal ganglia (Fecteau et al., 2007; Grandjean et al., 2005; Wiethoff et al., 2009). While semantic content is typically processed more in the left brain-hemisphere, affective prosody seems to be processed more in the right hemisphere in NT individuals (Bulman-Fleming and Bryden, 1994).

Typically developing children are capable of perceiving and understanding affective prosody from a very early age, and seem to learn this automatically (Blasi et al., 2011). However, for individuals with ASD, this extra-verbal aspect of communication seems to pose a much greater challenge (McCann and Peppe, 2003). Meanwhile, findings from behavioral studies of affective prosody recognition in ASD individuals are mixed. In a large sample of high-functioning children with ASD, Peppe et al. (2007) described systematic deficits in both perception and production of affective prosody in single words. In addition, Philip et al. (2010) investigated emotion recognition in facial expressions, body movements, and speech in a group of adults with ASD, and found a core deficit in emotion recognition affecting all three stimulus-domains, suggesting that prosodic deficits are linked to a broader social–emotional impairment in individuals with ASD. Similar difficulties in recognizing affective prosody and decoding mental states from affective prosody are reported elsewhere (Golan et al., 2007; Heaton et al., 2012; Hobson, 1986; Lindner and Rosen, 2006; Mazefsky and Oswald, 2007). However, impairments in emotion recognition from affective prosody are often correlated with verbal intelligence (Golan et al., 2007; Lindner and Rosen, 2006; Mazefsky and Oswald, 2007), suggesting that impairments in affective prosody might be linked to language impairments rather than to ASD per se. Consistent with this, several studies have demonstrated intact emotion recognition from affective prosody particularly in groups matched on mental-age/verbal IQ (Boucher et al., 2000; Brennand et al., 2011; Chevallier et al., 2011; Grossman et al., 2010; Jones et al., 2011; Loveland et al., 1997; Ozonoff et al., 1990), and in high-functioning individuals with ASD (Doyle-Thomas et al., 2013; Heikkinen et al., 2010; O'Connor, 2007). However, there seem to be an effect of stimulus complexity on emotion recognition abilities in individuals with ASD. Both O'Connor (2007) and Doyle-Thomas et al. (2013) reported equivalent emotion recognition from voice stimuli in high-functioning individuals with ASD and NT participants when stimuli were presented in isolation, but impairments in the ASD group when the voice stimuli were presented alongside emotional faces. This points towards more subtle, but significant, emotion recognition difficulties in high-functioning individuals with ASD.

Despite the large number of behavioral studies investigating affective prosody in ASD, relatively few have looked at neural processing of basic emotions from affective prosody in individuals with ASD compared to NT individuals. Eigsti et al. (2012) investigated angry prosody in a group of high-functioning adolescents with ASD using functional magnetic resonance imaging (fMRI) during an implicit task where no emotion identification was required. They found that NT individuals showed stronger activation in the left inferior frontal gyrus, while the ASD group showed more widespread brain activation, which Eigsti et al. (2012) suggest reflect a less automatic processing of angry prosody, and a higher reliance on cognitive control in the ASD group. The study by Eigsti et al. (2012) is the only fMRI study which directly investigates neural processing of basic emotions in individuals with ASD. However, they only looked at angry prosody. Thus, the brain regions involved in the processing of affective prosody other than anger remain to be investigated in individuals with ASD. Clearer knowledge in this area is essential for understanding ASD individuals' impairments in language and social–emotional processing. Thus, the aim of the present study was to compare the neural activity to happy, sad and neutral prosody in high-functioning adults with ASD and NT adults, matched on age, gender, full-scale IQ and verbal IQ.

Methods

Participants

A total of 43 participants were included in the study, and 23 of these had a formal diagnosis of ASD. Participants with ASD were recruited through the National Autism and Asperger's Association, assisted living services for young people with ASD, and specialized educational facilities. The structural MRI of three participants with ASD showed abnormal ventricular enlargement (this is not an uncommon finding see Gillberg and Coleman, 1996) and were excluded before data analysis was begun. One ASD participant was unable to relax in the scanner and thus did not complete the testing. Consequently, a total of 19 high-functioning adults with ASD (2 females, 17 males) and 20 NT adults (2 females, 18 males) were included in the data analysis.

All participants were right-handed and native speakers of Danish, with normal hearing. Groups were matched on gender, age, IQ, and verbal IQ (Table 1). All participants were IQ-tested using Wechsler's Adult Intelligence Scale (WAIS-III; Wechsler, 1997), and filled out the adult version of the Autism Spectrum Quotient (AQ) (Baron-Cohen et al., 2001). The AQ provides a measure of autistic traits from 0 to 50, and from 0 to 10 on five subscales (social impairments, attention to detail, attention switching, impaired imagination and communication) in high-functioning individuals with ASD as well as in NT individuals. None of the NT participants had any history of neurological or psychiatric illness. All participants with ASD carried a previous formal diagnosis of ASD, which were supported by the Autism Diagnostic Observation Schedule (ADOS-G (Lord et al., 2000)) at the time of the study. All participants with ASD were invited in for the ADOS testing after the brain scanning session, but unfortunately five participants were unable to come back for testing due to long transportation, or because they needed special assistance. Thus a total of 14 participants with ASD completed ADOS testing (Table 2), of these 14 individuals two did not meet the cutoff criteria of 7 (1 female, ADOS score = 5; 1 male, ADOS score = 3). Nonetheless, all participants with ASD were previously diagnosed by specialized psychiatrists and we were given access to their medical records to confirm diagnoses. All ASD participants were medication naive and did not have any comorbid psychiatric disorders. All participants gave written informed consent and were compensated for their time and transportation expenses. The study was approved by the local ethics committee and was in accordance with the Helsinki Declaration.

Table 1

Subject characteristics

	ASDN = 19 (2♀)	NTN = 20 (2♀)	t-valuep-value
Age in years (SD/range)	26.16 (5.6/20–36)	24.45 (4.6/19–41)	0.92ns
Full-scale IQa (SD/range)	108.32 (14.56/78–135)	114.50 (12.4/92–137)	−1.58ns
Verbal IQb (SD/range)	112.68 (23.7/74–186)	118.30 (13.8/90-143)	−1.05ns
AQc total mean (SD)	28.84 (7.43)	16.05 (5.93)	5.96<.001

SD = standard deviation. ns = not significant at p < 0.05.

WAIS-III full-scale (Wechsler, 1997).

Verbal IQ from WAIS-III.

The autism spectrum quotient (Lord et al., 2000).

Table 2

ADOS scores (N = 14, 2♀)

Mean total ADOS score(SD/range)	11.23 (4.48/3–18)
Communication(SD/range)	3.54 (1.71/1–6)
Social reciprocity(SD/range)	7.15 (3.89/1–12)
Stereotyped and repetitive behaviors/interests(SD/range)	1.6 (1.84/0–6)

Summary of ADOS scores (Lord et al., 2000) for the 14 ASD participants who completed testing. Two participants did not meet the cutoff of 7 on the total ADOS score.

Stimuli

The stimuli used during scanning were semantically non-emotional sentences (e.g. “if you go grocery shopping later will you please buy me 1 liter of milk and 10 eggs, I feel like baking a cake today — maybe Ill make muffins”) in Danish. Stimuli were vocal recordings of 12 s duration. Each sentence was recorded with happy, sad, and neutral prosody. Stimuli consisted of both male and female voices recorded from students at the Acting Academy in Aarhus, Denmark. To validate the stimuli they were piloted on a group of NT adults (N = 12) before the fMRI-study. Stimuli were selected from a sample of 90 stimuli, comprised of a sample of 30 sentences recorded with happy, sad and neutral prosody. Based on the piloting procedure, the 12 pilot-participants rated each stimulus on an 11-point Likert-scale (from −5 to 5) ranging from very sad to very happy. Happy stimuli were selected if they were rated as happy or very happy (4 or 5 on the Likert-scale) by all pilot-participants. Sad stimuli were selected if they were rated as sad or very sad (−4 or −5) by all participants. Neutral stimuli were selected if they were rated as neutral (0 on the Likert-scale) by more than half the pilot participants, and a little sad (−1) or a little happy (1) by the remaining pilot participants. A total of 60 speech stimuli (20 happy/20 sad/20 neutral) were included in the study. All stimuli (happy, sad and neutral) were matched on total duration and intensity.

Design

During the fMRI-scan participants were lying in the scanner while listening to the 60 sentences (20 happy, 20 sad and 20 neutral) with their eyes open staring at a fixation cross. Following each sentence, participants were asked to rate the emotion felt by the person who spoke the sentence. An MR-compatible track-ball was used for emotion ratings on a screen displaying a visual analogue scale ranging from very sad (−100) over neutral (0) to very happy (+100). Participants were instructed that neutral was right in the middle, and the cursor started out in the neutral position on each trial. All participants completed 5 trials outside the scanner, to make sure that they were familiar with the task and understood the instructions. Participants were explicitly instructed to listen for the emotion, not the semantic content. Besides the speech task, participants also completed a musical task in the scanner, where participants were asked to decode emotions from musical excerpts. The order of the tasks (speech and music) was randomized between participants. Data from the music task were analyzed independently, for a separate paper (Gebauer et al., 2014).

fMRI data acquisition

Brain imaging was obtained using a Siemens, 3 T Trim Trio, whole-body magnetic resonance scanner located at the Centre of Functionally Integrative Neuroscience at Aarhus University Hospital, Denmark.

Two 10.5 min experimental EPI-sequences were acquired with 200 volumes per session and the parameters: TR = 3000 ms, TE = 27 ms, flip angle = 90°, voxel size = 2.00 × 2.00 × 2.00 mm, #voxels = 96 × 96 × 55, slice thickness 2 mm, and no gaps. Participants wore MR-compatible headphones inside a 12-channel head coil. After the two functional scans a sagittal T1-weighted anatomical scan with the parameters: TR = 1900 ms, TE = 2.52, flip angle = 9°, voxel size = 0.98 × 0.98 × 1 mm, #voxels = 256 × 256 × 176, slice thickness 1 mm, no gaps, and 176 slices, was acquired for later co-registration with the functional data. Participants were instructed to lie still and avoid movement during the scan.

Behavioral data analysis

Continuous emotion ratings from the visual analog scale were analyzed using a 2 (groups: ASD and NT) × 3 (emotion condition: happy, neutral, or sad) mixed model analysis of variance (ANOVA). In order to identify potential differences in emotion categorization between the two groups, rather than the dimensional measure as is acquired with the VAS, we recalculated ratings into categorical measures. All ratings larger than zero were coded as happy and all smaller than zero were coded as sad. Categorization of neutral prosody was not included in this analysis, since cutoff points for this would be fairly arbitrary, and over all participants tended to code neutral stimuli as zero or close to zero (see confidence interval in Fig. 1). Categorical measures were therefore analyzed using a 2 (groups: ASD and NT) × 2 (emotion condition: happy, sad) mixed model analysis (ANOVA).

Fig. 1

Mean emotion ratings (on a visual analog scale from −100 to 100) of sad, neutral and happy speech excerpts. Error bars indicate 95% confidence intervals. Significant difference in emotion ratings for happy and sad affective prosody between the ASD and NT groups.

fMRI data analysis

fMRI data analysis was performed using Statistical Parametric Mapping (SPM8 version 4667; ) (Friston, 2011). Preprocessing was done using default settings in SPM8. The functional images of each participant were motion corrected and realigned (Friston et al., 1995), spatially normalized to MNI space using the SPM EPI template and trilinear interpolation (Ashburner and Friston, 1999), and smoothed using an 8 mm full-width at half-maximum smoothing kernel. For each participant, condition effects were estimated according to the general linear model (Friston et al., 1994). To investigate main effects of group, emotion condition, and interaction effects between group and emotion, a 2 (groups: ASD and NT) × 3 (emotion conditions: neutral, sad, and happy) full factorial ANOVA was run in SPM8. To ensure that we did not ignore any existing effects, the ANOVA was performed with a liberal significance threshold of p < 0.001 uncorrected for multiple comparisons, with an extent threshold at 10 voxels.

To further look into potential between-group differences, random-effects analyses were performed using independent-samples t-tests for the contrasts: happy > neutral prosody, sad > neutral prosody, happy > sad prosody, and sad > happy prosody. Finally, to evaluate between group differences associated with general emotion processing, happy prosody and sad prosody were collapsed into one category “emotional” and an independent sample t-test of the contrast emotional > neutral prosody was performed. One-sample t-tests were performed for all the abovementioned contrasts to examine effects of affective prosody across groups. All t-test results were thresholded at p < 0.05 after family wise error correction (FWE (Friston et al., 1996)) with an extent threshold at 10 voxels. Between group analyses were also performed with a less conservative significance threshold of p < 0.001 uncorrected, with an extent threshold at 10 voxels. Figures are t-statistics displayed on top of standard MNI T1-images. Labeling of brain regions is done according the Wake Forest University (WFU) PickAtlas (Lancaster et al., 2000; Maldjian et al., 2003; Tzourio-Mazoyer et al., 2002). Tables indicate coordinates for peak-voxels significant at both peak and cluster-levels.

Results

Behavioral ratings: emotional vs. neutral speech

Continuous emotion ratings from the visual analog scale were analyzed using a 2 (groups: ASD and NT) × 3 (emotion condition: happy, neutral, or sad) mixed model ANOVA. Mauchly's test indicated that the assumption of sphericity was not met for the main effects of emotion condition in the behavioral analysis χ2(2) = 49.78, p < 0.001, thus Greenhouse–Geisser corrected degrees of freedom are reported here. The ANOVA revealed a significant main effect of emotion, F(1.14, 42.3) = 591.79, p < 0.001, and a significant interaction between group and emotion F(1.14, 42.3) = 7.88, p = 0.006 (Fig. 1). Post-hoc independent sample t-test showed significant group differences (with alpha adjusted for multiple comparisons) between the ASD and NT groups on both the sadness ratings, t(37) = 2.69, p = 0.01 and the happiness ratings, t(37) = –2.53, p = 0.02. Due to the nature of the VAS used for emotion ratings in this experiment, what appears to be an interaction effect is actually a main effect of group, where the ASD group tends to rate the emotional intensity (both happiness and sadness) as less emotionally intense.

Besides the continuous measure of emotion intensity, we also recoded the VAS ratings into categorical measures of happy or sad. This was done in order to see whether differences in emotion intensity found in the mixed model ANOVA was due to miss-categorizations of happiness and sadness in the ASD group. For emotion categorization, assessing incorrect categorization of happy and sad affective prosody, significantly more errors were found in categorizations of sad affective prosody than happy affective prosody F(1, 37) = 16.40, p < 0.001, post-hoc t-test: t(38) = –3.95, p < 0.001. There was however no significant interaction between group and emotion categorization errors, F(1,37) = 2.26, p < 0.141 (mean number of miss-categorizations ASD: happy = 0.30, std. dev. = 0.95, sad = 1.84, std. dev. = 2.34, mean number of missing responses = 0.63, std. dev. = 1.34. NT: happy = 0.40, std. dev. 0.99, sad = 1.10, std. dev. = 1.02, mean number of missing responses = 0.30, std. dev. = 0.57).

fMRI data: main effect of group, emotion condition, and interaction effect

A 2 (groups: ASD and NT) × 3 (emotion conditions: neutral, sad, and happy) full factorial ANOVA revealed a significant main effect of group, with the NT group displaying increased brain activation in left precentral gyrus/rolandic operculum (BA 6) and left superior temporal gyrus (BA 22) at p < 0.05 after FWE-correction. For results at p < 0.001 uncorrected see Table 3. Significant main effects of emotion was found in the bilateral amygdala and anterior cingulate cortex (BA 24/25), precuneus (BA 31), left medial frontal gyrus (BA 10), superior frontal gyrus (BA 9) and middle temporal gyrus (BA 21), and right sub-gyral at the level of p < 0.001 uncorrected (Table 4). No significant interaction between group and emotion condition was found at the level of p < 0.001 uncorrected. To make sure that the lack of an interaction effect did not stem from the two ASD subjects who scored below the cutoff on the ADOS, the analysis was re-run excluding those two. An additional analysis was also done excluding the two ASD participants with low ADOS scores and the five ASD participants who did not complete ADOS testing. None of these analyses revealed any significant interaction effect between group and emotion condition, even with the relatively liberal significance threshold of p < 0.001 uncorrected.

Table 3

ANOVA: main effect of group, p < 0.001 uncorr.

		BA	x	y	z	k	F
Precentral gyrus/rolandic operculum	L	6	−60	4	8	253	34.64**
Superior temporal gyrus	L	22	−60	−46	4	211	28.99**
Postcentral gyrus/rolandic operculum	R	4	66	−10	16	127	27.67*
Superior temporal gyrus	L	38	−38	4	−16	65	26.43*
Postcentral gyrus	L	43	−64	−22	14	90	25.10*
Superior temporal gyrus	L	22	−48	−14	6	140	21.51*
Cingulate gyrus	L	−	−10	−8	42	73	21.00*
Inferior frontal gyrus	R	44	54	18	8	70	19.61*
Inferior parietal lobule	L	40	−48	−46	26	32	16.70
Sub-gyral	L	−	−22	34	10	32	16.39
Postcentral gyrus	R	3	52	−18	60	28	16.22
Brain stem	R	−	4	−30	−20	18	15.42
Medial frontal gyrus	L	10	−14	48	14	15	14.94
Superior temporal gyrus	R	38	36	6	−20	22	14.94
Superior frontal gyrus	R	9	18	58	36	12	14.92
Corpus callosum	L	–	−6	2	26	14	14.45

ANOVA main effect of group independent of emotion condition. Peak coordinates from significant clusters (p < 0.001 uncorrected, extent threshold = 10 voxels). BA = Brodmann area. k = cluster size.

Marks clusters significant at the level of p < 0.05 after FDR correction.

Marks clusters significant at the level of p < 0.05 after FWE correction.

Table 4

ANOVA: main effect of emotion, p < 0.001 uncorr.

		BA	x	y	z	k	F
Amygdala	L	−	−16	−6	−18	318	13.69
Anterior cingulate	L	24	−2	−32	2	145	10.38
Precuneus	–	31	0	−50	30	109	10.02
Anterior cingulate	R	25	4	8	−8	51	9.75
Amygdala	R	−	24	−4	−24	116	9.61
Medial frontal gyrus	L	10	−8	50	22	68	9.59
Superior frontal gyrus	L	9	−10	50	42	55	9.47
Middle temporal gyrus	L	21	−60	−8	−14	16	8.77
Sub-gyral	R	−	46	−2	−20	12	8.66

ANOVA main effect of emotion condition independent of group. Peak coordinates from significant clusters (p < 0.001 uncorrected, extent threshold = 10 voxels). BA = Brodmann area. k = cluster size.

fMRI data: independent sample t-test

Though no interaction effect appeared from the ANOVA, we ran independent-samples t-test for all individual contrasts to make sure that no between group differences were ignored. These analyses showed no significant differences between groups in any of the contrasts; emotional > neutral prosody (max T-value ASD > NT = 4.10; NT > ASD = 3.78; height threshold T = 5.38), happy > neutral prosody (max T-value ASD > NT = 4.00; NT > ASD = 3.15; height threshold T = 5.32), sad > neutral prosody (max T-value ASD > NT = 3.77; NT > ASD = 3.29; height threshold T = 5.44), happy > sad prosody (max T-value ASD > NT = 4.38; NT > ASD = 3.31; height threshold T = 5.45), or sad > happy prosody (max T-value ASD > NT = 3.31; NT > ASD = 4.38; height threshold T = 5.45) at the significance level of p < 0.05 after FWE-correction. See Fig. 2 for percent signal changes between groups in peak voxels in the emotional versus neutral prosody contrast. Nor did any differences appear when applying the less conservative FDR correction for multiple comparisons. Only at the more liberal statistical significance level of p < 0.001 uncorrected, did between-group differences appear. At p < 0.001 uncorrected, the ASD group showed increased activation in response to emotional compared to neutral speech in the right caudate (x = 22, y = 8, z = 22; T = 4.10; cluster size = 27 voxels) relative to the NT group. Meanwhile, the NT group displayed increased activation in the left rolandic operculum/precentral gyrus (x = −56, y = 0, z = 8; BA 44; T = 3.78; cluster size = 22 voxels) compared to the ASD group. Also, for the contrast of happy prosody compared to neutral prosody, the ASD group showed increased brain activation in the right middle/superior frontal gyrus (x = 26, y = 6, z = 66; BA 6; T = 4.00; cluster size = 24 voxels), left sub-gyral (x = −24, y = −14, z = 36; T = 3.61; cluster size = 11 voxels), and superior parietal lobule (x = 24, y = −70, z = 48; BA 7; T = 3.58; cluster size = 16 voxels). Finally, no between-group difference was found when comparing sad to neutral affective prosody.

Fig. 2

Brain activations across all individuals independent of group for affective compared to neutral prosody, p < 0.05 after FWE-correction in: right middle temporal gyrus (right MTG), right inferior frontal gyrus (right IFG), right precentral gyrus, left superior temporal gyrus (left STG), bilateral basal ganglia/lentiform nucleus, and medial orbitofrontal cortex (mOFC) and medial frontal gyrus. See Table 5 for complete list of significant clusters of activation. Box plots show mean effect size for each group in the peak voxel for each region, with 95% confidence intervals.

fMRI data: one-sample t-test emotional vs. neutral speech

To examine brain activation in response to affective prosody across groups a one-sample t-test was run for the contrasts: emotional (happy + sad) > neutral prosody, happy > neutral prosody, sad > neutral prosody, sad > happy prosody, and happy > sad prosody. Comparisons of emotional prosody and neutral prosody showed increased brain activations within language-related fronto-temporal and subcortical brain networks in response to affective prosody, in both ASD and NT individuals (Table 5, Fig. 2). Across groups, emotional speech was associated with increased activation in the right middle temporal gyrus (BA 21), left superior temporal gyrus (STG, BA 22) and right IFG (BA 47, 45), bilaterally in the medial frontal gyrus (BA 6), in the left precentral gyrus (BA 6), bilaterally in the parahippocampal gyrus (BA 28, 34), in the left superior frontal gyrus (SFG, BA 9), and bilaterally in the basal ganglia (lentiform nucleus), after whole-brain FWE correction (p < 0.05 peak level, voxel extent threshold = 10). Comparing happy and neutral, sad and neutral, and happy and sad prosody also demonstrated significant activations across both the ASD and NT groups (see Supplementary tables for a list of peaks for these contrasts). The comparison of sad and happy prosody did not reveal any regions of increased activation after FWE-correction.

Table 5

Main effect of emotional versus neutral prosody.

		BA	x	y	z	k	T
Middle temporal gyrus	R	21	60	−14	−4	4046	17.26
Superior temporal gyrus	L	22	−60	−22	−2	4940	15.47
Middle frontal gyrus	R	6	58	0	46	266	9.10
	L	6	−4	−2	54	393	6.24
Precentral gyrus	L	6	−46	−8	60	871	8.39
Parahippocampal gyrus	L	28	−18	−8	−16	249	8.38
	R	34	22	−10	−20	173	6.57
Orbitofrontal gyrus	−	11	0	50	−14	84	7.15
Inferior frontal gyrus	R	45	60	24	12	192	7.01
Superior frontal gyrus	L	9	−8	52	42	60	6.77
Basal ganglia — lentiform nucleus	R	−	26	−4	6	17	6.02
	L	−	−22	2	2	16	5.67

Emotional versus neutral prosody. Peak coordinates from significant clusters (FWE p < 0.05, extent threshold = 10 voxels). BA = Brodmann area. k = cluster size.

Discussion

In the present study we showed that high-functioning adults with ASD activated mostly identical fronto-temporal and subcortical brain regions in response to affective prosody as did NT individuals. However, when applying a liberal significance threshold the ASD group showed increased activation of the right caudate compared to the NT group in response to emotional compared to neutral prosody, while the NT group displayed increased activation of the left precentral/rolandic operculum. These differences might be attributed to different attentional demands and different levels of processing between the two groups. On the behavioral ratings, individuals with ASD rated both happy and sad affective prosody as less emotionally intense (less happy or less sad) compared to the NT group. These results suggest that subtle differences in emotion perception and brain processing of affective prosody exist in individuals with ASD, which might explain some of the problems this group has with detecting emotions in others.

Independent of the task, we found a main effect of group on brain activation, where the NT group showed increased activation compared to the ASD group in the left precentral gyrus/rolandic operculum and left superior temporal gyrus. This finding corresponds to the decreased left lateralization of language processing commonly reported in individuals with ASD (Harris et al., 2006; Just et al., 2004; Lai et al., 2012; Lai et al., 2011; Redcay and Courchesne, 2008).

In response to emotional compared to neutral prosody, individuals with ASD and NT individuals alike showed neural activation in fronto-temporal brain regions, including the bilateral superior temporal and middle temporal gyri, right inferior frontal gyrus, and orbitofrontal regions, in addition to striatal and midbrain structures including the lentiform nucleus. These brain regions have all previously been associated with processing of affective prosody in NT individuals (Buchanan et al., 2000a; Fruhholz and Grandjean, 2012; Kotz et al., 2003; Redcay, 2008; Schirmer and Kotz, 2006). Affective prosody was also associated with increased activation bilaterally in the parahippocampal gyrus, and in the middle and superior frontal gyri in both groups. The parahippocampal gyrus is primarily engaged in memory encoding and retrieval, which are relevant for processing of extra-verbal information (Rapp et al., 2012; Wallentin et al., 2005) and emotional responses (Blood and Zatorre, 2001; Imaizumi et al., 1997). The middle and superior frontal gyri are implicated in executive functions (Moreno-Lopez et al., 2012) attention and working memory (du Boisgueheneuc et al., 2006). Thus, it seems likely that activity within these regions is related to the cognitive evaluation of the emotional content in both ASD and NT participants. Looking at the main effect of emotion condition from the factorial analysis, the emotional manipulation might be interpreted to have been somewhat weak, however the behavioral data and the post-hoc t-tests do show that it was reliably effective.

No significant interactions were found between group and emotion condition, suggesting that both groups activated highly similar brain regions in response to processing of affective prosody. Yet, post-hoc t-tests showed that trends towards differences between the two groups were apparent when a more liberal statistical significance level was applied. In response to affective prosody, the ASD group showed increased activation of the right caudate. The caudate is part of the ventral striatum, a region rich in dopaminergic receptors, which is found to be central for attention (Volkow et al., 2009). Indeed it seems possible that the evaluation of emotional affective prosody was more attentionally demanding for the ASD group compared to the NT group, which might also have contributed to the lower emotion ratings. In contrast, the NT group showed increased activation of the left precentral gyrus/rolandic operculum. Previous studies on NT individuals have found this part of the precentral gyrus to be more involved in semantic processing relative to processing of affective prosody (Buchanan et al., 2000b; Mitchell et al., 2003). Thus while the ASD group might have devoted extra attention to decode the affective prosody, the NT group likely had the extra capacity also to attend to the semantic content of the stimuli.

Looking into the emotions independently, ASD individuals showed increased activation relative to NT individuals during processing of happy affective prosody. This increased activation was found in the middle/superior frontal gyrus, left sub-gyral and superior parietal lobule. These increased activations might support the notion of increased attentional demands during emotion recognition for affective prosody, and particularly happy prosody, since the middle and superior frontal gyri are implicated in executive functions (Moreno-Lopez et al., 2012) attention and working memory (du Boisgueheneuc et al., 2006), while the superior parietal lobule has been found to be more active during explicit compared to implicit decoding of affective prosody (Bach et al., 2008). It should however be underlined that these between-group differences do not survive statistical correction for multiple comparisons. Thus, while interesting and potentially important to the behavioral differences identified in the emotion ratings, these differences in brain activation should be interpreted with some caution. Meanwhile, they do seem to correspond well with the findings by Eigsti et al. (2012), who found increased activity in adolescents with ASD in brain areas associated with executive functioning and mentalizing in response to angry prosody, while using a similar significance threshold (p < 0.001 uncorrected). This suggests that common differences in processing of affective prosody in ASD individuals exist in the frontal and sub-cortical brain regions for angry prosody, as was studied by Eigsti et al. (2012), and for happy and sad prosody as was the focus of the present study, and that differences in affective prosody processing at both behavioral and neural levels might be stable from adolescence into adulthood.

Previous studies indicate that individuals with ASD often perform more similarly to controls when given explicit instructions, relative to spontaneous behavior (Nuske et al., 2013; Wang et al., 2007). Yet, despite the fact that we used an explicit task, where participants were instructed actively to decode the emotional valence of the stimuli and Eigsti et al. (2012) used an implicit emotion task there seem to be no extensive differences in neural processing between explicit and implicit processing of affective prosody in individuals with ASD. This might be because individuals with ASD use similar, potentially more analytical, and cognitively and attentionally demanding strategies in both cases.

In general, studies of emotion recognition abilities in individuals with ASD show very heterogeneous results, and large variations seem to exist in the ASD population depending on age, diagnostic status and intellectual abilities (Boucher et al., 2000; Brennand et al., 2011; Chevallier et al., 2011; Golan et al., 2007; Grossman et al., 2010; Heikkinen et al., 2010; Lindner and Rosen, 2006; Mazefsky and Oswald, 2007). Yet, other studies support the notion that more subtle differences in emotion perception are present in high-functioning individuals with ASD (Doyle-Thomas et al., 2013; O'Connor, 2007). Our findings of lower happiness and sadness ratings in the ASD group compared to the NT group correspond with findings of impaired emotion recognition found in high-functioning individuals with ASD in other studies (Heaton et al., 2012; Philip et al., 2010). At the same time, several studies have described intact emotion recognition from affective prosody in high-functioning ASD individuals (Brennand et al., 2011; Doyle-Thomas et al., 2013; Grossman et al., 2010; Heikkinen et al., 2010; Jones et al., 2011; O'Connor, 2007). A possible explanation for the discrepancy between these studies and our findings might be that these studies all use categorical responses as measures of judgments of emotion, as opposed to the continuous judgments of emotion intensity which we used. Here the use of a continuous measure allow for more subtle differences in emotion perception to be observed. To be able to compare our results with studies using categorical measures of emotion recognition we re-coded emotion ratings of affective prosody into categorical measures of ‘happiness’ and ‘sadness’. Comparable to other studies on recognition of affective prosody, which used categorical measures (Brennand et al., 2011; Doyle-Thomas et al., 2013; Grossman et al., 2010; Heikkinen et al., 2010; Jones et al., 2011; O'Connor, 2007), we did not see any group differences in emotion categorization abilities, suggesting that important information is lost in collapsing the scale to a categorical response. Therefore, it is likely that individuals with ASD have the ability to categorically distinguish vocally expressed emotions in isolation, but that they are not as emotionally affected by affective prosody as NT individuals. Thus, ASD individuals might not attribute the same significance to affective cues in speech in every-day face-to-face interactions as NT individuals do. This interpretation is supported by findings by O'Connor (2007) and Doyle-Thomas et al. (2013) of intact unimodal emotion recognition, but impaired emotion recognition in ASD individuals when audio-visual integration is required. Thus, the complexity and the instructions for the emotion recognition task seem to be central for how well the ASD group manages the task.

A specific strength of this study is the closely matched ASD and control groups. Anderson et al. (2010) showed that ASD participants with higher verbal IQ scores demonstrated more ‘typical’ brain activations during a language task than those with low verbal IQ. Similarly, a number of studies have found emotion recognition impairments in ASD to be correlated with verbal IQ (Golan et al., 2007; Lindner and Rosen, 2006; Mazefsky and Oswald, 2007). Also, only medication free participants were included in this study. Many people with ASD take medication regularly (Dove et al., 2012), however the impact of medication on brain function is not well-established and might confuse differences due to medication with differences associated with having ASD. It should however be noted that the participants with ASD included in this study were all high-functioning and had normal language abilities. Thus, samples with less verbally able individuals with ASD might show different patterns of brain processing of affective prosody compared to the findings presented here. Two of the included ASD individuals did not meet the cutoff on the ADOS, this might have been due to years of interventions, and might suggest that their symptoms have ameliorated to a level where they have achieved an optimal outcome (Fein et al., 2013). Nevertheless, removing these subjects from the ASD sample did not change the results. Future studies should investigate brain processing of more complex stimulus material requiring greater levels of integration and more immediate, naturalistic responses to be made, to examine these more subtle differences in emotion processing in people with ASD.

Conclusion

In response to affective prosody, high-functioning adults with ASD activated mostly identical fronto-temporal brain regions relative to NT individuals, including the superior and middle temporal gyri, inferior frontal gyrus, as well as subcortical brain structures. However, individuals with ASD rated emotions in affective prosody as less intense than NT individuals. Similarly, there was a tendency for the ASD group to show increased brain activation in the right caudate relative to NT individuals during emotional compared to neutral prosody. This might be due to the higher attentional demands placed by the emotional stimuli in this group, which are potentially also contributing to general social–emotional impairments.