Neural dynamics underlying emotional transmissions between individuals.

Emotional experiences are frequently shaped by the emotional responses of co-present others. Research has shown that people constantly monitor and adapt to the incoming social-emotional signals, even without face-to-face interaction. And yet, the neural processes underlying such emotional transmissions have not been directly studied. Here, we investigated how the human brain processes emotional cues which arrive from another, co-attending individual. We presented continuous emotional feedback to participants who viewed a movie in the scanner. Participants in the social group (but not in the control group) believed that the feedback was coming from another person who was co-viewing the same movie. We found that social-emotional feedback significantly affected the neural dynamics both in the core affect and in the medial pre-frontal regions. Specifically, the response time-courses in those regions exhibited increased similarity across recipients and increased neural alignment with the timeline of the feedback in the social compared with control group. Taken in conjunction with previous research, this study suggests that emotional cues from others shape the neural dynamics across the whole neural continuum of emotional processing in the brain. Moreover, it demonstrates that interpersonal neural alignment can serve as a neural mechanism through which affective information is conveyed between individuals.

Introduction

Our emotional life is inherently social (Boiger and Mesquita, 2012; van Kleef ). Indeed, emotion research suggests that emotional signals are constantly transmitted from person to person and might significantly shape the emotional responses of recipients (Butler, 2015; Fischer and Manstead, 2008; Van Kleef, 2009). These emotional transmissions frequently occur when people are merely co-present in a particular situation, without directly attending to each other (Gallotti and Frith, 2013; Parkinson, 2011). For example, studies have shown that when individuals co-view emotional movies together, their emotional (Bruder ) and physiological (Golland ) responses to the movies become similar, marking contagious transmissions of non-verbal affective cues between them. These and other findings suggest that individuals continuously monitor and process emotional signals from co-present others, which serve as contextual cues to the affective meaning of the current situation (Parkinson, 2011; Van Kleef, 2009). And yet, the neural mechanisms involved in processing affective cues from co-attending others are poorly understood. The aim of the current research was to gain a better understanding of such mechanisms by studying how the human brain processes continuous emotional feedback coming from another individual who co-attends to the same affective input.

Previous research focusing on the social impact of emotions has suggested that emotions of others might influence one’s responses through two distinct routes, that is inferential processes and affective reactions (Parkinson and Simons, 2009; Van Kleef, 2009). For instance, fear of a dark street expressed by one individual may spread to another individual directly, through bottom-up channels of fear contagion. It can also elicit top-down inferential processes by changing one’s interpretation of what is happening, making the street look more dangerous than before (Parkinson, 2011). The stimulus-driven and the inferential processing routes have been linked with distinctive neural mechanisms of emotional processing (Ochsner ). Specifically, the bottom-up, low-level emotional processes, are commonly associated with ‘core affect’ systems, which monitor and encode the emotional value of the input, with such prominent hubs as the amygdala and the insular-cingulate circuit, in addition to other sub-cortical and para-limbic structures (Kober ; Lindquist ; Phelps and LeDoux, 2005). The top-down, evaluative emotional processes are linked with activations in the medial pre-frontal (MPFC) structures (Kober ; Ochsner ; Touroutoglou ), which support cognitive appraisals and re-evaluation of the affective inputs in light of the broader motivational context and stored knowledge (Barrett, 2014; Etkin ; Ochsner , 2009; Roy ).

In the social domain, neuroimaging studies have demonstrated that emotions of others can trigger activations both in the core affect systems and in the MPFC regions. Specifically, perceptually rich stimuli depicting other people in profound affective and visceral states most commonly elicit activations in the core affect systems of the perceivers, employing the bottom-up path of emotional processing (Adolphs, 2010; Bastiaansen ; Öhman and Mineka, 2001; Raz ). In contrast, social cognition tasks, which require thinking about the minds of others, such as theory of mind, or TOM, are associated with MPFC activations (Frith and Frith, 2006; Mitchell ; Zaki and Ochsner, 2012). For example, the activity in the dorsal MPFC was shown to predict accurate judgment of the emotional states of other people (Zaki ). While social cognition involves a widespread set of brain regions, including temporal and posterior regions (Frith and Frith, 2006; Koster-Hale and Saxe, 2013), the social evaluative processes and the emotional inferential processes seem to converge specifically in the MPFC structures.

In sum, prior studies have shown that the low-level core affect systems and the high-level MPFC regions are implicated both in the intra-personal emotional processes and in mirroring and understanding other people. The main goal of the present study was to directly examine whether affective cues arriving from another individual who co-attends the same emotional input significantly shape the neural dynamics within these brain systems.

To that aim we employed an innovative social paradigm in which emotional feedback was continuously provided to participants whose attention was directed towards an engaging movie (Figure 1A). Participants in the experimental group (social) believed that the feedback was coming from another, co-present individual who was co-viewing the same movie simultaneously. To control for the intra-personal emotional processes and for the non-specific attentional and perceptual processes, we introduced a control group (non-social) in which participants received identical input, but did not think that the feedback was coming from another person.

Fig. 1.

(A) Emotional feedback was presented as a colorful frame, which changed color as the movie progressed. (B) To generate the feedback, dynamic ratings of the movie’s emotional intensity (black line) were transformed into a four-level color code (ranging from gray (low intensity) to warm (high intensity) colors) and embedded in the movie frame. (C) The resulting feedback approximately matched the natural emotionality of the movie (r = 0.49). (D) Timeline of the scanner session.

To investigate neural dynamics, we employed temporally extended stimuli, such that both the target stimulus (i.e. movie) and the social–emotional cues (i.e. feedback) were provided continuously. Recent research emphasizes the inherently dynamic nature of both intra- and inter-personal emotional processes, which constantly fluctuate over time (Butler, 2011, 2015; Frijda, 1988; Golland , 2015, Raz , 2016). The approach undertaken here specifically aims to capture the dynamic aspects of interpersonal neural processes, missed out when measures are aggregated across time.

To examine the effects of social–emotional feedback on neural dynamics we employed two types of analysis. First, we assessed response cohesion in the social and non-social group participants, using the inter-subject correlation analysis (ISC), which measures similarity of response patterns across individuals who are exposed to the same stimulus. This data-driven method, successfully used in numerous previous studies (see Hasson et al., 2010 for review), reveals where in the brain and to what extent the provided input reliably controls the neural dynamics. Critically, increased levels of ISC in functionally-specific brain regions have been shown to reflect enhanced functional recruitment (Ames ; Honey ; Lerner ; Schmälzle ) and enhanced emotional processing (Nummenmaa , 2014a,b; Trost ) in dynamic naturalistic paradigms. We asked whether continuous processing of social–emotional feedback controls the neural dynamics in the core affect insular and amygdala regions as well as in the high-level evaluative MPFC regions, leading to increased response similarity in the social compared with the non-social group.

In addition, we studied whether the neural dynamics in those regions were temporally aligned with the emotional timeline of the feedback. A series of previous studies have demonstrated that during verbal (Stephens ) and non-verbal (Dumas ; Konvalinka ; Müller and Lindenberger, 2011; Schippers ) communicative exchanges, the neural activity of interacting individuals becomes synchronized. These studies have suggested that neural synchrony between sender’s and receiver’s brains drives the transmission of information between them (Hasson and Frith, 2016; Hasson ). Here we studied neural synchrony between the receivers’ brains and the social–emotional feedback. Using feedback timeline as a predictor, we quantified neural alignment to the feedback timeline and compared it in the social and the non-social groups.

Materials and methods

Participants

Thirty-seven subjects participated in the experiment. Ten were discarded from the analysis: six due to excessive head motion > 4 mm, two due to corrupted functional data (e.g. contained spikes in the fMRI time series greater than a 5 s.d. change in image intensity), and two due to misunderstanding of experimental conditions, as revealed during debriefing. The final dataset included fourteen subjects in the experimental (social) group (age: 26.07 ± 2.99, seven males) and thirteen subjects in the control (non-social) group (age: 25.7 ± 3.37, six males). Procedures were approved by the Helsinki Committee on Activities Involving Human Subjects in the Tel Aviv Sourasky Medical Center. All subjects were native-Hebrew speakers and provided written informed consent.

Stimuli

Participants in both groups were presented with an episode from an Israeli TV series, ‘Yellow Peppers’ (directed by Keren Margalit, 2010), depicting an emotionally laden process in which a young couple’s 5-year-old son is diagnosed with autism. Participants freely viewed the episode, which was divided into two parts. In the first part (part1), participants watched the first 8 min of the episode. Then, a colorful frame appeared around the movie (Figure 1A). The frame dynamically changed color as the movie progressed (part2, ∼10 min). The color of the frame represented the emotional intensity of the movie. The scan was initiated with a 30 s blank screen, which was later discarded from analysis.

Generation of the feedback frame

To generate the emotional feedback timeline, an independent sample of participants (N = 12) provided continuous ratings of the emotional intensity of the movie, using an analog rating dial (Mindware, Ohio). These dynamic emotional ratings were averaged across participants and transformed into a four-level color-code (Figure 1B). Emotional intensity was decoded as color intensity, which ranged from a desaturated grey color (lowest intensity) to a fully saturated red color (highest intensity). This color code was adjusted to a smooth visual presentation and embedded into the movie’s frame, using video editing software (Premier, Adobe). The resulting feedback timeline approximately matched the natural emotional timeline of the movie (r = 0.49, Figure 1C).

Experimental procedure

Social group

Upon arriving at the experimental room, a participant (hereafter, P) met another ‘participant’, who was in fact a confederate (hereafter, C). The confederate was a female and participated in all sessions in this group. After a brief introduction, P and C were shown a two and a half minute preview of the movie and jointly answered a pre-scanning questionnaire that stimulated a discussion of possible scenario plots. P and C were explained that they would be positioned in different scanners, but would view the same movie simultaneously. In addition, during movie presentation, P would receive a dynamic feedback in the form of a color-changing frame which would represent the ongoing emotional responses of C to the movie. After the instructions, participants received explanation of the feedback colors and viewed a short demonstration of the feedback. After being positioned in the scanner, the experimenter asked P and C how they felt in the scanner (in fact, C was sitting in the control room). Afterwards, P had audio communication with C to enhance the feeling of co-presence, during which they played verbal games for ∼6 min. Then the scanning phase started and participants were shown the first (movie part1, no-feedback) and the second (movie part2 + feedback) parts of the movie. Figure 1D presents the scanning timeline. Following the scanning session participants rated the subjective feeling of social presence during the experiment (‘To what degree did you feel alone during the experiment?’) and the degree to which the feedback matched their own emotional experience, using nine-point Likert scale.

Non-social group

The control group participants viewed a preview of the movie, answered the pre-scanning questionnaire and received the feedback training alone. They were told that the feedback represents the emotional intensity of the movie, which was computed by an advanced computational algorithm. Upon entering the scanner, they had a brief conversation with the experimenter.

Critically, the scanning timeline was identical for both groups. Participants from both groups viewed an identical movie and received identical emotional feedback. But only for the experimental group participants this feedback was believed to be coming from a social source.

fMRI acquisition

MRI scanning was performed at Tel-Aviv Sourasky Medical Center with a 3 T head-only MRI scanner (GE Signa EXCITE, Milwaukee, WI, USA), using an eight-channel head coil. Blood oxygenation level dependent (BOLD) functional MRI was acquired with T2*-weighted imaging using the following parameters: repetition time (TR) = 3 s; echo time (TE) = 35 ms; flip angle (FA) = 90°; field of view (FOV) = 22 × 22 cm2; matrix size = 96 × 96; 39 slices of 3 mm thickness, 1 mm gap. The slices were positioned nearly horizontal to cover the entire temporal lobe and the parts of the frontal lobe, as well as nearly all of the occipital and parietal lobes. High-resolution, anatomical, T1-weighted, fast-spoiled gradient-echo images were acquired using the following parameters: FOV = 256 mm; matrix = 256 × 256; TR = 9.2 ms; TE = 3.5 ms; axial slices of 1 mm thickness, no gap. These anatomical volumes were used for cortical segmentation and surface reconstruction. To minimize head movements, participants’ heads were stabilized with foam padding. MRI-compatible headphones (OPTOACTIVE™) were used to attenuate the scanner noise and to present the audio stimuli (Presentation®, Neurobehavioral Systems).

fMRI data pre-processing

fMRI data were reconstructed and analyzed using the BrainVoyager QX software package (Brain Innovation, Maastricht, The Netherlands) and with in-house software written in MATLAB. Pre-processing of functional scans included 3D motion correction, slice scan time correction, linear trend removal and high-pass filtering (cut-off: 0.01 Hz). Spatial smoothing was applied using a Gaussian filter of 6 mm full-width at half-maximum value. The cortical surface was reconstructed from the 3D MPRAGE anatomical images using BrainVoyager software. The complete functional dataset was transformed to a common 3D Talairach space (Talairach and Tournoux, 1988) and projected onto a reconstruction of the cortical surface.

Initial 15 TRs (10 TRs blank + 5 TRs movie) were cropped from the functional data in the first part of the experiment to allow the hemodynamic responses to reach steady state and exclude general effects of audio-visual appearance. In addition, visual examination of the average BOLD time-courses in the pre-defined regions of interest (ROIs) revealed that initial appearance of the feedback frame elicited a distinctive neural event (∼132 s length), characterized by initial increase of response and followed by enhanced inter-subject variability. To avoid distortion of results by this one-time event and allow for an accommodation period to the feedback, 46 TRs (138 s) were cropped from the data at the beginning of the second part of the experiment. This procedure resulted in two functional sequences (part1, part2) of comparable length (444 s, 462 s, respectively).

ISC analysis

An ISC analysis evaluates the extent to which a given neural region responds similarly across individuals. In comparison to a standard general linear model (GLM) analysis, which typically assumes a canonical hemodynamic response function, the ISC method makes no a-priori assumptions concerning the specific timing of BOLD responses elicited by the stimulus. Rather, the ISC approach asks to what extent the same response time course is reliably observed across all participants (Golland ; Hasson ; Lerner ).

Computing voxel-by-voxel ISC

For each voxel, ISC was calculated as an average where the individual r are the Pearson product-moment correlations between that voxel’s BOLD time course in one individual and the average of that voxel’s BOLD time courses in the remaining individuals. ISC maps were produced separately for each part of the movie (part1: movie; part2: movie + feedback), both within each group (social, non-social) and for a unified pool of subjects.

Controlling for false positives

Because of the presence of long-range temporal autocorrelations in the BOLD signal (Zarahn ), the statistical likelihood of each observed correlation was assessed using a bootstrapping procedure based on phase randomization. The null hypothesis was that the BOLD signal in each voxel in each individual was independent of the BOLD signal values in the corresponding voxel in any other individual at any point in time (i.e. that there was no ISC between any pair of subjects). For both parts of the movie, a phase randomization of each voxel's time course was performed by applying a fast Fourier transform to the signal, randomizing the phase of each Fourier component and inverting the Fourier transformation. This procedure scrambles the phase of the BOLD time courses but leaves its power spectrum intact. For each randomly phase scrambled surrogate dataset, we computed the ISC coefficients (R) for all voxels in the exact same manner as the empirical correlation maps described above, i.e. by calculating the Pearson correlation between that voxel’s (scrambled) BOLD time course in one individual and the average of that voxel’s (scrambled) BOLD time courses in the remaining individuals. Performing this procedure for 5000 times allowed for an estimation of a null distribution of average correlations within this voxel. The statistical likelihood of the original voxel-wise ISC (R) was estimated directly from the control distribution. The resulting maps of ISCs (e.g. Figure 2) were corrected for multiple comparisons, using the Benjamini–Hochberg–Yekutieli false-discovery procedure, which controls the false discovery rate (FDR) under assumptions of positive dependence (Benjamini and Yekutieli, 2001; Genovese ).

Fig. 2.

Voxel-wise ISC maps in the social (A) and the non-social (B) groups. Maps demonstrate the cortical regions which showed significant ISC (corrected for multiple comparisons using FDR procedure). The color bar denotes the ISC statistical range. Dorsal medial pre-frontal cortex (dMPFC), ventral medial pre-frontal cortex (vMPFC), anterior cingulated cortex (ACC), posterior cingulated cortex (pACC), inferior frontal gyrus (IFG), anterior insula (AI), central sulcus (CS), inferior parietal sulcus (IPS).

Comparison between groups

To test the hypothesis that ISC is increased in the social as compared with the non-social group, a one-tailed t test was performed within each voxel that showed significant ISC in at least one of the inspected groups (Figure 2). The voxel-wised t test was done by comparing the Fisher-transformed correlation values of subjects from the social group to the Fisher-transformed correlation values of subjects from the non-social group, computed for each voxel (Figure 3A). The resulting maps were corrected for multiple comparisons, using FDR procedure.

Fig. 3.

(A) Brain regions showing statistically stronger ISC in the social comparing to the non-social group. (B) Comparison of ISC in a group of pre-defined ROIs, related to emotional processing. (C) Comparison of ISC in control regions, not related to emotional processing. (D) Comparison of ISC between the first (part1: movie only) and the second (part2: movie plus feedback) experimental parts, within social (orange) and non-social (green) groups.

Neural alignment analysis

Neural alignment analysis assesses the degree to which BOLD response time-courses are time-locked to the changes in the feedback time-line.

Computing voxel-by-voxel neural alignment

To identify brain regions which showed response alignment with the feedback timeline, we assessed cross-correlations between the BOLD time courses and the feedback timeline, convolved with a hemodynamic response function (Glover, 1999). Given the low temporal frequency of the feedback changes, we allowed for temporal shifts of up to 6 s (two TRs) between BOLD time-course and the feedback timeline. Accordingly, for each voxel and for each individual we computed a maximal correlation within 6 s lag (), between the response time-course of individual j, and the feedback timeline. These correlation scores were Fisher transformed and averaged across subjects, producing

The statistical significance of each voxel-wise R was assessed using non-parametric bootstrapping procedure as in ISC analysis. Neural alignment maps were computed for the whole experimental sample and for the social and non-social groups in separate (Figure 5). The resulting maps were corrected for multiple comparisons using FDR procedure (FDR, q = 0.05).

Fig. 5.

Whole-brain neural alignment. Upper panel. (A) Neural alignment map (grey) computed for the whole experimental sample (N = 27), overlapped (orange) with ISC map (red). While the ISC maps included both perceptual and high order regions, the neural alignment map was confined to regions previously linked with emotional processing. Lower panel presents neural alignment maps computed separately for the social (B) and non-social (C) groups.

To identify areas that showed enhanced alignment with the feedback in the social as compared with the non-social group, a one-tailed t test was performed on voxel-wise, Fisher transformed R scores. The neural alignment analysis was limited to voxels that showed significant ISC scores in at least one of the groups (Figure 6). The resulting map was FDR corrected (FDR, q = 0.05).

Fig. 6.

Regions showing significantly increased neural alignment in the social group participants. Maps are thresholded at P < 0.01, FDR corrected.

ROI analyses

A set of independent ROIs was defined based on an ISC map of subjects from both groups (N = 27), during the first part of the movie (without the feedback frame). ROIs were defined by sampling significant voxels, within a radius of 15 × 15 × 15 mm, in the vicinity of the following areas:

Bottom-up, core affect regions: left and right amygdala, left and right anterior insula (AI);

Top-down, evaluative regions: dorsal medial pre-frontal cortex (dMPFC), ventral medial pre-frontal cortex (vMPFC);

Control regions: Figure 3C associated with sensory and attentional processing: left and right frontal-eye-field (FEF), left and right inferior parietal sulcus (IPS), auditory cortex (A1+), visual cortex (V1+). See Table 1 for Talairach coordinates of the ROIs.

Table 1.

Talairach coordinates of the independently defined ROIs d/vMPFC, dorsomedial/ventromedial pre-frontal cortex; A1 + extended auditory cortex in the vicinity of temporal sulcus; V1 + extended visual cortex in the vicinity of calcarine sulcus; FEF, frontal eye field; IPL, inferior parietal lobule

Area	x	y	z
Core affect
Amygdala	−23	−6	14
	17	−7	−9
Insula	−39	7	−2
	37	13	0
MPFC
dMPFC	−1	51	36
vMPFC	4	44	1
Sensory
A1 +	−53	22	4
	54	26	3
V1 +	−21	−85	7
	15	−84	7
Dorsal Attention Network
IPL	−57	−41	22
	59	−39	24
FEF	−25	−12	54
	27	−10	52

ISC ROI analysis

To assess group differences in ISC, we computed for each ROI the regional ISC scores and performed a one-tailed t-test after applying Fisher transformation (Figure 3B–D).

Neural alignment ROI analysis

To assess group differences in neural alignment, we applied the cross-correlation analysis on the regional response time-courses. To assess whether neural alignment was higher in the social as compared with the non-social group, we performed a one-tailed t test on Fisher transformed individual scores within each ROI (Figure 4C).

Fig. 4.

Neural alignment with the emotional timeline of the feedback. (A) Average dMPFC response time-course in the social (orange) and non-social (green) groups. The alignment of the dMPFC responses to the feedback timeline (black line) is higher in the social than in the non-social group. (B) Cross-correlation plots for the individual dMPFC responses and the feedback timeline in the social and non-social groups. (C) Average cross-correlation between regional response time-course and the feedback timeline. Statistical comparisons are computed for max correlation values within 0–6 s lag.

Brain-behavior associations

To assess the association between the neural effects of the feedback and behavioral ratings (Figure 7), we computed Pearson correlations between the behavioral ratings and the neural scores (i.e. ISC, neural alignment) in each ROI for the social group participants.

Fig. 7.

Both ISC scores (A) and neural alignment scores (B) in the dMPFC and right amygdala were correlated with the degree of experienced social presence, reported by the participants.

Results

We examined the effects of social–emotional feedback on functional brain dynamics. Participants viewed an engaging emotional movie which was embedded in a feedback frame that gradually changed its color as the movie progressed (Figure 1A). We manipulated the meaning of the feedback, such that only the social group participants believed that the feedback represented real-time emotional responses to the movie coming from another, co-present individual. To examine the effects of the feedback on neural dynamics, we performed two types of analysis. First, we assessed within-group response similarity, using ISC analysis. We computed ISC maps in the social and non-social groups (Figure 2) and compared between them (Figure 3A). We repeated the social vs non-social ISC analysis in the pre-defined ROIs, comprising the core affect amygdala and insular regions, the top-down MPFC regions and a series of control regions (Figure 3B, C). Second, we asked whether the neural response time-courses in the emotion processing regions were temporally aligned to the dynamics of emotional feedback and whether such neural alignment was enhanced when the feedback comes from the social source (Figures 4–6). Finally, we assessed whether the neural effects of the feedback were modulated by the degree of social presence experienced by participants (Figure 7).

ISC in the social and non-social groups

Exploratory whole brain analysis: To begin with, we assessed neural response similarity in the social and non-social groups by computing group-level voxel-wise ISC (Figure 2). Consistent with previous studies (Guo ; Nummenmaa , 2014a), we found that emotionally laden naturalistic stimuli elicited robust neural response patterns, which were time-locked across subjects in a widespread set of anatomical regions. While the highest levels of ISC were observed in the auditory and visual primary cortices, significant correlations were also evident in other regions, including bilateral superior-temporal sulcus, anterior temporal poles, inferior and superior parietal regions, lateral pre-frontal regions, vMPFC and dMPFC regions, AI and subcortical structures.

Visual comparison of the ISC maps suggested that MPFC, insular and subcortical regions exhibited enhanced ISC in the social group. To directly map the neural regions showing the effects of social–emotional feedback on ISC, we statistically compared the ISC maps in the social and non-social groups (Figure 3A). This analysis confirmed the above observation, showing that participants who believed that the feedback comes from another individual exhibited increased ISC in the dMPFC, vMPFC, bilateral AI, extending dorsally towards the inferior frontal gyrus (IFG) on the right side, anterior portion of the temporal poles (aTP) and a subset of subcortical structures (see Table 2 for a complete set of regions and their coordinates). The reverse contrast (non-social vs social) did not show significant results. To examine whether the observed increase in ISC in the social group can be explained by the mere state of co-viewing, or by an affective state caused by communication with the confederate prior to the scanning, we also compared the social and the non-social ISC maps during the first part of the movie, which did not include feedback presentation. This analysis did not reveal significant effects, suggesting that the observed group differences (Figure 3A) were specifically driven by the social feedback.

Table 2.

Talairach coordinates of the statistical maps: the Talairach coordinates were derived from the following statistical maps: i) ISC within the social and the non-social group (S, nS; Figure 2A, B) and the comparison between them (S > nS, Figure 3A), ii) neural alignment within the social and non-social groups (S, nS; Figure 5B, C) and the comparison between them (S > nS, Figure 6). ACC, anterior cingulate cortex; SFG, superior frontal gyrus; AI, anterior insula; IFG, inferior frontal gyrus; pInsula, posterior insula; OFC, orbitorfrontal cortex; aTP, anterior temporal pole; STS, superior temporal sulcus; vLPFC, ventrolateral pre-frontal cortex; Supramarg. G, supramarginal gyrus

					ISC			n. alignment
	Area	x	y	z	S	nS	S > nS	S	nS	S > nS
Middle	Caudate head	0	7	3				v
	ACC	0	28	30	v			v
	dMPFC	−1	52	39	v		v	v		v
	vMPFC	0	48	4	v		v	v	v
	Precuneus	0	−74	37	v	v
	Mid. thalamus	2	−11	13				v		v
Left	SFG (SMA)	−5	11	64				v	v
	Caudate	−13	17	6				v	v
	AI	−39	15	−6	v		v	v		v
	IFG	−42	20	7	v	v		v	v
	pInsula	−40	−30	12	v	v		v
	OFC	−27	33	−6	v
	aTP	−28	17	−26				v		v
	STS	−51	−21	−5
	vLPFC	−46	25	5	v	v		v
	FEF	−49	−9	48	v	v		v
	IPL	−41	−38	43	v	v
	Supramarg. G.	−54	−43	23	v	v		v	v
	Amygdala	−16	−5	−10	v			v	v
	Cerebellum	−21	−73	−31	v	v		v
Right	SFG (SMA)	11	12	64				v	v
	Caudate	12	17	6	v		v	v	v
	AI	37	12	−2	v		v	v		v
	IFG	50	10	11	v		v	v	v
	pInsula	38	−27	13	v	v		v	v
	OFC	20	31	−6	v			v
	aTP	36	5	−27	v	v	v	v	v	v
	STS	49	−23	4	v	v		v	v
	vLPFC	45	34	12	v			v
	FEF	46	−5	50	v	v		v	v
	lSFG	45	22	34		v
	IPL	46	−34	39	v	v
	Supramarg. G.	59	−42	23	v	v		v	v
	Amygdala	17	−6	−9	v	v		v	v
	Cerebellum	20	−71	−31	v	v		v

ROI analysis

To test our hypothesis in the hypothesized brain regions associated with bottom-up and top-down emotional processing, we conducted the ISC analysis in a series of independently defined ROIs (see Methods). This analysis clearly showed (Figure 3B) that presentation of emotional feedback from a social as compared with a non-social source exerted higher control over cortical dynamics, yielding higher ISC scores in the MPFC regions, bilateral AI and right amygdala. Notably, although the amygdala did not pass the corrected threshold in the whole brain analysis, it did show significant effects in the right, but not the left hemisphere in the ROI analysis.

Control ROI analysis

To test alternative explanations for the enhanced ISC in the social compared with the non-social groups, we conducted several control comparisons. First, ISC in the sensory regions (visual: V1+, auditory: A1+) and in the dorsal attention regions (IPS, FEF) did not differ between the social and the non-social groups (Figure 3C), suggesting that the observed effects are specific to the social–emotional value of the feedback and cannot be explained by general attention or engagement effects.

To validate that the group effects in ISC specifically stem from the presentation of social–emotional feedback, we compared the regional ISCs between the two parts of the experiment: part 1, which included the movie presentation only and part 2 which included both the movie and feedback presentations (Figure 3D). Introduction of the social–emotional feedback (social group: part2 vs part1) significantly increased ISC in the dMPFC (P < 0.0005), vMPFC (P < 0.003), bilateral AI (P < 0.001) and right amygdala (P < 0.005) in the social group participants. When the feedback had no social meaning (non-social group) the regional ISC did not significantly differ between the two parts of the movie (all P's > 0.26). In other words, introduction of social (but not of non-social) emotional feedback exerted additional control over cortical dynamics in those regions.

Neural alignment with emotional feedback

To further examine whether social–emotional feedback directly shaped the neural dynamics in the MPFC and the core affect systems, we conducted a neural alignment analysis which assessed whether response time-courses were time-locked to the dynamic changes in the incoming emotional feedback.

Regional neural alignment analysis

Figure 4A illustrates the approach underlying the neural alignment analysis. It presents the average response time-course of the dMPFC in the social (orange) and the non-social (green) groups together with the timeline of the feedback (black line). The average dMPFC activity was clearly aligned with the emotional timeline of the feedback, and was more so in the social (r = 0.62) than in the non-social (r = 0.23) group. Figure 4B further demonstrates the increased neural alignment in the social group, by presenting cross-correlation plots between the dMPFC neural activity and the feedback timeline, computed for individual response time-courses.

We employed the neural alignment analysis in the pre-defined ROIs (Figure 4C) and found that dMPFC, bilateral AI, and right amygdala (but not vMPFC and left amygdala) exhibited enhanced correlation with the feedback timeline, when the feedback was coming from the social (orange) as compared with the non-social (green) source. A follow-up examination of the ROI response patterns demonstrated that amygdala responses were different from other ROIs. While the neural alignment in the social group was similar across regions, the degree of alignment in the non-social group was significantly larger in the right amygdala (max r = 0.25) compared with dMPFC, vMPFC and bilateral insula (all P's <0.05). Notably, the amygdala response alignment in the non-social group was more delayed (i.e. longer time-shift) (mean lag = 3.92 ± 2.56 s) as compared with social condition (mean lag = 1.7 ± 2.55 s, P < 0.01). This pattern of results suggested that the amygdala, more than any other examined region, was driven in a bottom-up manner by the emotional timeline of the feedback, even when it was coming from a non-social source.

Whole brain analysis

At the next step, we conducted a whole brain neural alignment analysis. First, we computed voxel-by-voxel alignment with the feedback for the unified set of participants (N = 27). This analysis showed that a distinctive set of brain regions exhibited robust correlations with the feedback timeline. We next compared the neural alignment map with the ISC map (Figure 5A). As evident from the figure, while the movie elicited consistent, time-locked responses across participants in many posterior and anterior brain regions (Figure 5A, ISC map), the alignment of these patterns with the emotional timeline of the feedback was limited to a subset of regions (Figure 5A, neural alignment map), previously linked with the intra-personal emotional processes (Kober ; Lindquist ). These regions included subcortical limbic structures, such as the amygdala and thalamus, insular-cingulate regions, IFG and inferior parietal regions as well as MPFC regions (See Table 2 for a complete set of regions and their coordinates).

To compare neural alignment to the social and non-social emotional feedback, we conducted the alignment analysis for each group separately (Figure 5B, C). Comparison of these two maps suggested that when the feedback was coming from another individual (social), the alignment of response time-courses with the feedback timeline was stronger and more anatomically widespread.

This was confirmed by a formal statistical test on voxel-wise neural alignment scores (Figure 6). Functional clusters that passed the correction threshold in this analysis were restricted to the dMPFC, bilateral AI, anterior temporal poles and medial thalamus. The opposite contrast did not show any significant clusters.

Modulation of neural alignment by the subjective feeling of social presence

As revealed by post-experiment questionnaires, participants differed in the degree to which they actually experienced the social presence of the confederate in the scanner. We therefore assessed whether subjective feelings of social presence in the social group participants (‘To what degree did you feel alone during the experiment?’) could explain the variability of the individual ISC and neural alignment scores obtained in the ROI analyses. Correlation analysis revealed positive association between the neural effects of the feedback and the reported degree of social presence (Figure 7). Specifically, those participants who felt less alone during the experiment exhibited higher ISC in the dMPFC and the right amygdala (r = -0.75, P < 0.002; r = -0.63, P < 0.01, respectively). Similar results were obtained for the neural alignment scores (r = -0.74, P < 0.002; r = -0.7, P < 0.005, respectively). Conversely, the neural effects of the feedback in the AI did not show significant association with experienced social presence (P's > 0.28).

Discussion

Our emotional experiences are vastly influenced by the emotions of other people in the environment. Given the prevalence of interpersonal emotional effects, it is important to understand the neural processes underlying emotion propagation between people. We approached this question by studying the effects of continuous emotional feedback from another individual on functional neural dynamics. We found that social–emotional cues (i.e. feedback) significantly shaped the neural dynamics in the main hubs of the brain’s emotional circuitry, including both stimulus-driven core affect regions and evaluative MPFC regions.

Social–emotional feedback affected the neural dynamics in the MPFC and AI

First, we found that social–emotional cues exerted significant control over the neural dynamics in the ventral and dorsal MPFC regions as well as in the bilateral AI. Participants who believed that the feedback had arrived from another person demonstrated greater ISC in those regions than participants who did not. Additional control analyses confirmed that these effects were driven by the introduction of social–emotional feedback and cannot be explained by general processes (e.g. engagement, attention) associated with the social condition.

MPFC is a high order region integrating the incoming sensory inputs with stored knowledge (Roy ). In the context of emotional processes it has been shown to play a central role in top-down emotion elicitation (Ochsner ), cognitive appraisals of emotional inputs (Etkin ) and generation of affective meaning (Barrett and Satpute, 2013; Lindquist ; Roy ). In addition, MPFC is consistently involved in understanding the minds of others (Frith and Frith, 2006; Mitchell, 2009). Finally, the MPFC seems to be the neural site in which the self-related and the other related representations are combined (Amodio and Frith, 2006; Mitchell ). Here we found that processing the social–emotional cues affected the neural dynamics in both vMPFC and dMPFC regions. Notably, an exploratory whole-brain analysis (Figure 3A) extended the socially-driven neural effects to the bilateral anterior temporal pole, similarly involved in evaluative emotional and social processes (Olson ). These results, taken together with previous models of MPFC function, suggest that social emotional cues are integrated at the higher, evaluative levels, which contribute to the generation of affective meaning of the concurrent situation.

An additional region to exhibit socially-driven increase in ISC in our study was the bilateral AI. A recent study, which investigated interpersonal emotional alignment, showed that activation in the AI/IFG was correlated with the emotional inputs from another person in an interactive task (Prehn ). Activity in AI has been previously linked with embodied emotional experiences (Brooks ; Craig, 2010; Harrison ; Zaki ), monitoring of salient cues (Menon and Uddin, 2010; Seeley ; Uddin, 2014) and interoceptive predictions (Barrett and Simmons, 2015; Singer ). AI is also consistently reported to respond to emotional cues from others, which is commonly interpreted as an automatic mapping of the others' affective and visceral states onto one's own brain and body (Rütgen ; Singer and Lamm, 2009; Zaki and Ochsner, 2012). In line with these studies, we suggest that the insular role in interpersonal emotional transmissions might involve continuous monitoring for self and other related emotional signals to form the most accurate prediction of the homeostatic consequences of the current input.

Neural alignment with social–emotional input

Previous studies have demonstrated that MPFC encodes social cues (Mitchell, 2009) and core affect systems respond to emotional displays of others (Brooks ; Prehn ). This study goes beyond these results by showing that the response time-courses in these regions were synchronized with the dynamic changes in the incoming social–emotional input. Strikingly, this neural alignment with the emotional arousal of another individual occurred not only in the bottom-up circuits monitoring for the emotional value of the input, but also in the high order dMPFC region, associated with cognitive evaluative functions (Figure 5). Whole brain analysis revealed that the neural alignment to the social–emotional feedback extended beyond the pre-defined regions, also including the medial thalamus and bilateral anterior temporal poles. Prior studies of social neural dynamics reported similar effects of brain-to-brain alignment during interaction (Anders ; Schippers ; Stephens ). For example, the activity in MPFC was aligned between speaker and listener during verbal exchanges (Stephens ), while the activity in anterior temporal and insular regions was synchronized between brains during non-verbal affective communication (Anders ). It has been suggested that interpersonal neural synchrony serves as a biological mechanism for sharing information between people (Hasson ). In line with this suggestion, the results of this study suggest that interpersonal emotional transmissions involve moment-by-moment neural alignment of the receivers with the emotional cues arriving from others.

Notably, while the activity in the AI showed robust feedback-driven effects, the social–emotional effects in the amygdala, a core element of emotional circuit, were less consistent. Regional ISC and alignment analyses showed significant, yet modest effects, which did not survive the whole-brain analysis. In this context, it should be mentioned that the emotional cues in the current study were represented by simple color code (Figure 1), thus diminishing the stimulus-driven component of amygdala response. In addition, further examination of amygdala response patterns (Figure 4C) revealed that, in contrast to other regions, the amygdala showed high neural alignment in the non-social group, suggesting that its activity reliably followed the incoming emotional cues, whether they were coming from the social or from the non-social sources. This is consistent with the role of this emotional hub in bottom-up monitoring for salient affective features in the environment (Adolphs, 2010). Yet another aspect of our findings suggests that the amygdala does play a role in interpersonal emotional processes. We found that the effect of social feedback on neural dynamics, as indexed by ISC and neural alignment scores, was correlated with the degree of experienced social presence in the dMPFC and in the right amygdala (Figure 7). In other words, individual differences in the experienced social saliency modulated the control of social emotional cues over the neural dynamics in those regions.

Limitations and future studies

Human beings frequently act as social chameleons, adjusting their responses to the inputs from others. It has been shown in numerous studies that people align their feelings with the feelings expressed by others (Butler, 2015; Hatfield and Cacioppo, 1994; Parkinson, 2011; Van Kleef, 2009). Using ISC analysis we showed that processing of social–emotional cues increased the similarity of neural response patterns across observers. It is tempting to interpret this increased similarity as evidence of emotional convergence with the provided emotional cues. This interpretation is further strengthened by the increased neural alignment with the social–emotional feedback, observed in these same regions. And yet, the design of the current study did not allow for direct investigation of emotional convergence, as the prolonged, naturalistic stimuli used in the current study, make it challenging to reliably probe the changes in subjective emotional responses. Consequently, monitoring and encoding processes cannot be clearly differentiated from convergence. Given the importance of emotional convergence between people, future studies can specifically tap into these processes by directly assessing whether and in what regions neural alignment predicts emotional convergence.