Behavioral and EEG responses to social evaluation: A two-generation family study on social anxiety.

Social anxiety disorder is an invalidating psychiatric disorder characterized by extreme fear and avoidance of one or more social situations in which patients might experience scrutiny by others. The goal of this two-generation family study was to delineate behavioral and electrocortical endophenotypes of social anxiety disorder related to social evaluation. Nine families of patients with social anxiety disorder (their spouse and children, and siblings of these patients with spouse and children) performed a social judgment paradigm in which they believed to be evaluated by peers. For each peer, participants indicated their expectation about the evaluative outcome, after which they received social acceptance or rejection feedback. Task behavior, as well as the feedback-related EEG brain potentials (N1, FRN, P3) and theta power were tested as candidate endophenotypes based on two criteria: co-segregation with social anxiety disorder within families and heritability. Results indicated that reaction time for indicating acceptance-expectations might be a candidate behavioral endophenotype of social anxiety disorder, possibly reflecting increased uncertainty or self-focused attention and vigilance during the social judgment paradigm. N1 in response to expected rejection feedback and P3 in response to acceptance feedback might be candidate electrocortical endophenotypes of social anxiety disorder, although the heritability analyses did not remain significant after correcting for multiple tests. Increased N1 possibly reflects hypervigilance to socially threatening stimuli, and increased P3 might reflect that positive feedback is more important for, and/or less expected by, participants with social anxiety disorder. Finally, increased feedback-related negativity and theta power in response to unexpected rejection feedback compared to the other conditions co-segregated with social anxiety disorder, but these EEG measures were not heritable. The candidate endophenotypes might play a new and promising role in future research on genetic mechanisms, early detection and/or prevention of social anxiety disorder.

Introduction

Social anxiety disorder (SAD) is a psychiatric disorder characterized by extreme anxiety and avoidance in one or more social situations (APA, 2013). SAD is a common and debilitating internalizing disorder (Furmark, 2002, Rapee and Spence, 2004), and a known precursor to other psychiatric disorders, such as depression and substance abuse disorders (Grant et al., 2005, Rapee and Spence, 2004, Spence and Rapee, 2016). The risk for developing SAD is higher for individuals with a close family member with SAD than for individuals without family members with SAD (Isomura et al., 2015), and heritability of SAD is estimated around 20–56% (Distel et al., 2008, Isomura et al., 2015, Kendler et al., 1992, Middeldorp et al., 2005, Nelson et al., 2000). The genetic basis of psychiatric disorders could be studied by delineating endophenotypes, which are heritable trait markers in between the genotype and phenotype (Glahn et al., 2007, Gottesman and Gould, 2003, Iacono et al., 2016, Miller and Rockstroh, 2013). Electrocortical endophenotypes are specifically useful because they are presumably more closely related to genes than behavioral endophenotypes (Cannon and Keller, 2006). This study aims to delineate candidate endophenotypes of SAD by examining both behavioral and electrocortical responses to social evaluation.

The social judgment paradigm (SJP) (Gunther Moor et al., 2010b, Somerville et al., 2006, Van der Molen et al., 2014) could be useful in delineating candidate endophenotypes of SAD because this task allows for examining behavioral and electrocortical responses to social evaluation. In this task, participants receive feedback that communicates social acceptance or rejection, which can either be congruent or incongruent with participants' expectancies (Van der Molen et al., 2014). At the behavioral level, a number of studies have shown an optimism bias in healthy participants, as they more often expect acceptance versus rejection feedback (Dekkers et al., 2015, Gunther Moor et al., 2010a, Van der Molen et al., 2017, Van der Molen et al., 2014, Van der Veen et al., 2016). Patients with SAD expected to be accepted less frequently than healthy controls before the ‘Island Getaway task’, a task in which participants received social feedback without indicating their expectation in each trial (Cao et al., 2015). This is in line with cognitive-behavioral studies showing that patients with SAD expect negative outcomes of social situations (Clark and McManus, 2002, Hirsch and Clark, 2004). In SAD, the SJP has not been studied yet. Fear of negative evaluation has been studied using the SJP in healthy females, and was not related to feedback expectations during the task (Van der Molen et al., 2014). Notably, fear of negative evaluation was positively correlated with reaction time for indicating feedback expectations in healthy females, suggesting increased self-focused attention and vigilance during the SJP (Van der Molen et al., 2014). So, both feedback expectations and reaction time to indicate these expectations might be candidate endophenotypes of SAD.

At the electrocortical level, two event-related potentials (ERPs) have been examined using the SJP: the feedback-related negativity (FRN) and P3. The FRN (a negative component around 250 ms after feedback) is typically increased for feedback that is unexpected or reflecting poor performance (Ferdinand et al., 2012, Oliveira et al., 2007, Van Noordt and Segalowitz, 2012). However, it is unknown whether the FRN in response to social feedback is modulated by social anxiety in the SJP. There was no relation between fear of negative evaluation and FRN in healthy females (Van der Molen et al., 2014). In the Island Getaway task, the FRN was increased after acceptance feedback in patients with SAD (Cao et al., 2015), whereas FRN was increased after rejection feedback in healthy children with higher levels of parent-reported social anxiety (Kujawa et al., 2014). The effect of social anxiety on feedback valence might be related to feedback expectancies during the task, but this was not assessed on a trial-by-trial basis in the Island Getaway task (Cao et al., 2015, Kujawa et al., 2014). Thus, using the SJP allows for delineating the (differential) effect of feedback valence (acceptance versus rejection) and congruency (expected versus unexpected) on electrocortical responses that might be related to SAD. If there is indeed an effect of valence of social evaluative feedback in social anxiety (Cao et al., 2015, Kujawa et al., 2014), this should be present on both expected and unexpected trials of the SJP.

The P3 (a positive component that peaks around 300–500 ms after stimulus onset) is known to be sensitive to emotionally motivational stimuli (Hajcak et al., 2013). P3 results for healthy participants in the SJP are mixed. Some have found that the P3 was largest in response to expected acceptance feedback, and suggested that this P3 response might be related to the level of reward communicated by expected acceptance feedback (Van der Veen et al., 2016, Van der Veen et al., 2014). However, other studies did not find this P3 effect (Dekkers et al., 2015, Van der Molen et al., 2014). Further, P3 amplitude was not associated with fear of negative evaluation in healthy participants in the SJP (Van der Molen et al., 2014), nor with SAD in the Island Getaway task (Cao et al., 2015). If the social feedback-related P3 indeed reflects reward processing (Van der Veen et al., 2016, Van der Veen et al., 2014), the P3 in response to expected acceptance feedback might be a candidate endophenotype of SAD, based on altered reward-system reactivity in social anxiety (Cremers et al., 2015, Lahat et al., 2016). But, if the social feedback-related P3 rather reflects the processing of emotionally motivational stimuli (Hajcak et al., 2013), the P3 in response to expected and unexpected acceptance feedback might be a candidate endophenotype of SAD, given the importance of positive social evaluation for patients with SAD (Rapee and Heimberg, 1997).

More recently, studies using the SJP have examined neural oscillatory power in response to social evaluation (Van der Molen et al., 2017, Van der Veen et al., 2016). In contrast to ERPs, time-frequency power represents neural activity that is not phase-locked to the onset of a stimulus and this can yield additional insights into the neural dynamics (Cohen, 2014, Makeig et al., 2004, Van der Molen et al., 2017, Van Noordt et al., 2016). Theta oscillatory power seems sensitive to social threat (Cristofori et al., 2013, Van Noordt et al., 2015), and recent SJP studies have reported higher theta power in response to unexpected rejection feedback in healthy participants (Van der Molen et al., 2017, Van der Veen et al., 2016). Although theta power has not yet been studied in social anxiety, increased theta power in response to unexpected rejection feedback might be a candidate endophenotype of SAD, reflecting increased sensitivity to negative feedback in SAD (Clark and McManus, 2002, Heinrichs and Hofmann, 2001, Hirsch and Clark, 2004).

It is argued that endophenotypes could play an important role in understanding the genetic mechanisms underlying SAD (Cannon and Keller, 2006, Glahn et al., 2007, Iacono et al., 2016, Miller and Rockstroh, 2013), because their genetic basis is proposed to be simpler than the genetic basis of complex psychiatric disorders (Cannon and Keller, 2006, Glahn et al., 2007). To meet the criteria of an endophenotype of SAD, behavioral and electrocortical responses to social evaluation should adhere to certain criteria: (1) association with SAD, (2) co-segregation with SAD within families, (3) heritability, (4) state-independence, and (5) increased in non-affected family members compared to the general population (Glahn et al., 2007, Gottesman and Gould, 2003). The first criterion could be studied by comparing patients with SAD and healthy controls (as in Cao et al., 2015). The second and third criterion are based on the observation that psychiatric disorders run in families (Glahn et al., 2007, Gottesman and Gould, 2003). Within these families, the endophenotype should be displayed by persons with the disorder (‘co-segregation’). Furthermore, the endophenotype should be heritable. The fourth criterion indicates that persons with the disorder should display the endophenotype whether or not the illness is active (Gottesman and Gould, 2003). The fifth criterion could be studied by comparing family members of patients with SAD with healthy controls.

Although various methods have been used to examine the endophenotype criteria, a family design seems particularly appropriate to assess both the ‘co-segregation’ and ‘heritability’ criteria of an endophenotype. Extended families (e.g. including partner and children of patient, and siblings of patient with their partner and children) provide the opportunity to compare family members with and witout SAD (‘co-segregation’). Furthermore, we examined extended families instead of twins or sib-pairs, to increase the power to identify genetic variability within the family (because of the many different genetic relations) and thus heritability (Gur et al., 2007, Williams and Blangero, 1999). Moreover, we selected families based on two probands (adult with SAD and child with (sub)clinical SAD) to ensure we focused on a genetic form of SAD and to increase the chance that endophenotypes were related to the genetic factors that influence SAD (Fears et al., 2014, Glahn et al., 2010).

The goal of the current study was to investigate for the first time whether behavioral and electrocortical responses to social evaluation are candidate endophenotypes of SAD. In our two-generation family study, patients with SAD and their family members performed the SJP to assess behavioral and electrocortical responses to social evaluation. For the behavioral data, we expected that the number of trials in which participants expected social acceptance, as well as the corresponding reaction time for indicating feedback expectations are candidate endophenotypes, because previous studies have confirmed the first criterion for endophenotypes (‘association’) (Cao et al., 2015, Van der Molen et al., 2014). Even though the SJP has not been studied in SAD before, we expected the following electrocortical endophenotypes of SAD: the FRN in response to valence regardless of expectations (Cao et al., 2015, Kujawa et al., 2014), altered P3 in response to expected acceptance feedback (Cremers et al., 2015, Lahat et al., 2016, Van der Veen et al., 2016, Van der Veen et al., 2014) or to expected and unexpected acceptance feedback (Hajcak et al., 2013, Rapee and Heimberg, 1997), and increased theta power in response to unexpected rejection feedback (Van der Molen et al., 2017, Van der Veen et al., 2016). We exploratively tested whether the N1 might be a candidate endophenotype of SAD, even though it has not been studied before in the SJP, because it was found during visual inspection of the data and might be related to early attentional processes such as hypervigilance to socially threatening stimuli (Bögels and Mansell, 2004, Clark and McManus, 2002, Heinrichs and Hofmann, 2001, Hirsch and Clark, 2004, Morrison and Heimberg, 2013).

Materials and methods

Participants

This was the first study intensively investigating patients with SAD and their family members. Families were recruited via media exposure (radio, tv, newspapers) and selected based on two probands: one adult with SAD (‘target participant’) and his/her child with clinical or subclinical SAD (further referred to as ‘(sub)clinical’). SAD was diagnosed by a psychiatrist using a clinical interview and the structured Mini-Plus International Neuropsychiatric Interview (Sheehan et al., 1998, Van Vliet and De Beurs, 2007), based on the DSM-IV-R criteria for SAD generalized subtype. The psychiatrist confirmed that these patients also met the DSM-5 criteria. Subclinical SAD was defined as meeting the criteria for SAD, without showing impairment in important areas of functioning (criterion G in the DSM-5 (APA, 2013)). In the target participant's child, (sub)clinical SAD was diagnosed by a licensed clinician using a clinical interview and the structured MINI Kid interview (Bauhuis et al., 2013, Sheehan et al., 2010).

Nine target participants were included with their spouse and children, and the siblings of the target participant with spouse and children (total n = 132). Fig. 1 depicts the inclusion criteria. Nine participants only filled out questionnaires at home, five participants only participated in EEG resting state, and data of one participant could not be collected due to technical problems. One participant was excluded because s/he did not believe the cover story and one because s/he fell asleep during the task. Analysis of the SJP was based on 115 participants (59 females, Mage = 30.29, SD = 15.57, range = 8–61 years).

Fig. 1

Flow-chart of the inclusion and assessment procedure of the Leiden Family Lab study on SAD. All family members participated in all parts of the assessment procedure in one or two days. The order of these parts differed between participants, depending on their preferences and availability of the labs. Mostly, participants came to the lab with their family.

Note: One target participant scored above the cutoff of the autism questionnaire, but the psychiatrist confirmed that s/he could not be diagnosed with autism spectrum disorder (the high score was probably caused by SAD symptoms). Results of the social performance task are reported in Harrewijn et al. (2017a). Participants did not know beforehand about the social performance task.

SAD = social anxiety disorder; MINI Plus = Mini-Plus International Neuropsychiatric Interview (MINI Plus version 5.0.0) (Sheehan et al., 1998, Van Vliet and De Beurs, 2007); MINI Kid = MINI Kid interview (Bauhuis et al., 2013, Sheehan et al., 2010); FNE = Fear of negative evaluation (Carleton et al., 2006); AQ = Autism-spectrum quotient questionnaire (Baron-Cohen et al., 2001); SRS = Social responsiveness scale (parent-rated) (Constantino et al., 2003); LSAS = Liebowitz Social Anxiety Scale (Liebowitz, 1987); SAS-A = Social Anxiety Scale – adolescents (La Greca and Lopez, 1998); BDI = Beck Depression Inventory (Beck et al., 1996); CDI = Child Depression Inventory (Kovacs, 1992); STAI = State-Trait Anxiety Inventory (Spielberger et al., 1983); EHI = Edinburgh handedness inventory (Oldfield, 1971); BisBas = Behavioral Inhibition and Behavioral Activation Scales (Carver and White, 1994); BisBas child version = Behavioral Inhibition and Behavioral Activation Scales, child version (Muris et al., 2005); PANAS = Positive and negative affect scale (Watson et al., 1988); WAIS IV = Wechsler Adult Intelligence Scale IV (Wechsler et al., 2008); WISC III = Wechsler Intelligence Scale for Children III (Wechsler, 1991). Reprinted from Journal of Affective Disorders, 227, Harrewijn, A., Van der Molen, M.J.W., Van Vliet, I.M., Houwing-Duistermaat, J.J., & Westenberg, P.M., Delta-beta correlation as a candidate endophenotype of social anxiety: A two-generation family study, 398–405, Copyright (2017), with permission from Elsevier.

A priori power calculations revealed that 12 families with 8 to 12 family members (on average 10 members per family) were required for sufficient power (minimally 80%). This was computed by simulating data of an endophenotype with heritability of 60% and a correlation of 70% with SAD, based on studies in behavioral inhibition and SAD (Muris et al., 2005, Smoller et al., 2008). We included somewhat fewer families, since the families we included were larger (on average 14.67 instead of 10 members per family) which results in more power than smaller families (Dolan et al., 1999, Gur et al., 2007, Rijsdijk et al., 2001, Williams and Blangero, 1999).

Experimental design

Fig. 1 shows a flow-chart of the inclusion and assessment procedure of the Leiden Family Lab study on SAD (Harrewijn et al., 2017a). All participants received €75 for their participation and we reimbursed travel expenses. All participants provided written informed consent, according to the Declaration of Helsinki. Both parents signed the informed consent form for their children from 8 to 18 years of age, children of 12 to 18 years also signed themselves. The procedure was approved by the medical ethics committee of the Leiden University Medical Center.

Social judgment paradigm

We used the SJP as described in Van der Molen et al., 2014, Van der Molen et al., 2017 (Fig. 2). When participants were contacted to make an appointment for the EEG session, we asked them to email us a portrait photograph of themselves for a task about first impressions. We told them a panel of peers would evaluate their photograph and indicate whether they liked or disliked the person on the picture. This was actually a cover story to elicit feelings of social evaluation during the task. Most participants sent their photograph at least one week before the EEG session (31 participants sent their photograph 2–6 days before the EEG session). Participants were reminded of this cover story right before the SJP.

Fig. 2

Trial sequence of the social judgment paradigm. Reprinted from NeuroImage, 146, Van der Molen, M.J.W., Dekkers, L.M.S., Westenberg, P.M., Van der Veen, F.M., & Van der Molen, M.W., Why don't you like me? Midfrontal theta power in response to unexpected peer rejection feedback, 474–783, Copyright (2017), with permission from Elsevier.

The SJP consisted of 10 practice trials and 150 experimental trials in three blocks with a short break in between. Each trial started with a fixation cross (jittered duration of 500–1000 ms). Then, the picture of a peer1 appeared and remained on the screen during the rest of the trial. Participants had to indicate whether this person would like or dislike them by pressing the button in the left or right arm rest (the meaning of the buttons was counterbalanced between participants). The response of the participants (yes or no) was immediately shown on the left side of the picture. If the participants did not respond within 3000 ms, the message ‘too slow’ appeared and these trials were excluded from analysis. After a delay (3000 ms), the feedback of the peer was shown on the right side of the picture for 2000 ms. Before and after the SJP, participants were asked to indicate on a visual analogue scale (VAS) from 0 (exclusively negative) to 100 (exclusively positive) how they expected to be evaluated (before), and how they were evaluated (after) (similar to Cao et al., 2015). Afterwards, we asked participants not to tell their family members about the SJP. All but one participant reported that they believed the cover story of the SJP (this participant was excluded).

EEG recording and signal processing

EEG was recorded at a sampling rate of 1024 Hz with the BioSemi Active Two system (Biosemi, Amsterdam, The Netherlands) from 64 Ag-AgCl electrodes mounted in an elastic electrode cap (10/20 placement). The ground electrode was replaced by a feedback loop consisting of the common mode sense and driven right leg electrode. The common mode sense was used as online reference. Eight external electrodes were used: two for horizontal electrooculography (placed at left and right canthus), two for vertical electrooculography (placed 1 cm above and below the left eye), two for offline re-referencing (placed at mastoids), and two for measuring heart rate (modified lead-2 placement on chest; results will be reported elsewhere).

During offline preprocessing in BrainVision Analyzer (Brain Products GmbH), the signal was down sampled to 512 Hz, re-referenced to the average of left and right mastoids, and band-pass filtered (0.5–40 Hz, 50 Hz notch). Valid response segments (4000 ms before and after feedback) were selected if there was no response in the first 100 ms after the picture appeared, and if there was no second response 500 ms after the first response (which might indicate uncertainty of the answer of the participant). These segments were manually inspected for artifacts and noisy channels were interpolated. Then, ocular artifacts were adjusted for by ocular independent component analysis. The segments were also automatically checked for artifacts.2 If an artifact was detected in one channel, the entire segment was removed during both manual and automatic artifact detection. Supplementary Table 1 shows the number of artifact-free trials per condition for participants with and without SAD, these trials were used for both ERP and time-frequency analyses. Participants with SAD had more artifact-free trials overall, β = 7.25, p = 0.02, and more artifact-free trials indicating they expected to be disliked, whereas participants without SAD had more clean trials indicating they expected to be liked, β = − 13.88, p < 0.001.

ERP analysis

We created ERP segments of 1200 ms (200 ms before feedback and 1000 ms after feedback), which were baseline corrected (−200–0 ms) and averaged across trials (for the four conditions separately). For each component, we manually selected three electrodes with the largest peak amplitude from the grand-grand average (including all participants and conditions, as recommended by Kappenman and Luck, 2016), and tested with a repeated-measures ANOVA which electrode showed the largest peak amplitude. We continued analysis with this single midline electrode.

The FRN was computed by subtracting P2 amplitude from the subsequent negative peak (peak-to-peak method) (Dekkers et al., 2015, Van der Molen et al., 2014). The automatic peak detection procedure (local maximum) was used to determine the P2 240–340 ms after feedback, and the subsequent negative peak 290–390 ms after feedback. The peaks were manually adjusted if the P2 did not precede the most negative peak. The FRN was maximal at AFz, Fz, and FCz. Since FRN amplitude did not differ between these channels, we used FRN amplitude from Fz for further analyses (Dekkers et al., 2015, Van der Veen et al., 2016). P3 amplitude was determined via an area measure (Luck, 2005) between 360 and 500 ms, based on the grand-grand average (Kappenman and Luck, 2016). P3 was largest on Pz, compared to Fz and Cz.

When visually inspecting the data, we encountered an early negative component that peaked around 180 ms after feedback and seemed to differ between participants with and without SAD. This was not in line with the unbiased method as proposed by Kappenman and Luck (2016), but we decided to exploratively study this component further. We termed this component the N1 and we used an area measure between 130 and 220 ms. N1 amplitude was most negative at FCz, compared to AFz and Fz.

Time-frequency analysis

We performed time-frequency analysis on the same artifact-free trials as the ERP analysis. We created segments of 8000 ms (4000 ms before and after feedback). We applied a current-source density transformation, and a baseline correction using a 2100–2400 ms post-feedback baseline interval (due to the jittered duration of the fixation cross it was not possible to use a consistent pre-feedback baseline). Time-frequency characteristics were extracted from the EEG data by convolution of the single trials with complex Morlet wavelets (Gaussian-windowed sine waves), which increased from 1 to 40 Hz in 40 logarithmically spaced steps. We applied a Morlet parameter of 5, and the unit energy normalization method. Trials were averaged for the four conditions separately. Data was normalized according to the ratio change compared to the 2100–2400 ms post-feedback baseline interval. Theta power (layer 16–23, 4.13–8.01 Hz) was extracted from a 300–500 ms window after feedback onset from the AFz, Fz and FCz electrodes (Van der Molen et al., 2017). We used data from Fz for further analyses as this electrode yielded largest theta values (as indicated by the grand-grand average (Kappenman and Luck, 2016)).

Statistical analysis

First, we validated the differences between participants with and without SAD by comparing their self-reported symptoms of social anxiety (La Greca and Lopez, 1998, Liebowitz, 1987), fear of negative evaluation (Carleton et al., 2006), and depression (Beck et al., 1996, Kovacs, 1992). For social anxiety and depressive symptoms, we computed z-scores based on normative samples (Fresco et al., 2001, Inderbitzen-Nolan and Walters, 2000, Miers et al., 2014, Roelofs et al., 2013) to enable comparison between adult and child questionnaires. Regression models were fitted in R (R Core Team, Vienna, Austria) with self-report questionnaires as dependent variable, and SAD, age (standardized), age (standardized)2 and sex as independent variables. A random effect was included to take into account the genetic correlations between family members.

Second, we tested whether behavioral and electrocortical measures during the SJP were candidate endophenotypes of SAD, using the two criteria ‘co-segregation with SAD within families’ and ‘heritability’ (Glahn et al., 2007). Like previous studies, we calculated a bias score indicating the percentage of trials on which participants expected to be accepted by peers (number of acceptance-expectations / (number of acceptance + rejection expectations) ∗ 100). Co-segregation analyses were performed by fitting regression models with the candidate endophenotype as dependent variable, and SAD, congruency, feedback, congruency ∗ feedback, SAD ∗ congruency, SAD ∗ feedback, SAD ∗ congruency ∗ feedback, age (standardized), age (standardized)2 and sex as independent variables.3 Random effects were included for the genetic correlations between family members and the correlations between conditions within a person. A significance level of α = 0.05 was used for statistical analyses.

Heritability analyses were performed using SOLAR (Almasy and Blangero, 1998). Briefly, SOLAR decomposes the total variance of the phenotype into genetic and environmental components. This is estimated using maximum likelihood techniques, based on a kinship matrix for the genetic component and an identity matrix for the unique environmental component (with ones on the diagonal and zeros everywhere else, implying that the environment is unique to every person). We did not include a shared environmental component to keep the model as simple as possible. Heritability is defined as the ratio of the additive genetic component and the total phenotypic variance (after removal of variance explained by covariates) (Almasy and Blangero, 2010). Age (standardized), age (standardized)2 and sex were included as covariates, and were removed from the final model if p > 0.05. We could not run bivariate analyses to calculate genetic correlations between the candidate endophenotypes and SAD, because too few non-target participants were diagnosed with SAD. Since the assumptions for SOLAR (trait standard deviation higher than 0.5, residual kurtosis normally distributed) were not met for most variables, an inverse normal transformation was applied to all EEG variables in this step, as implemented in SOLAR (Almasy and Blangero, 1998, Almasy and Blangero, 2010). We applied a Bonferroni correction for performing multiple (25) heritability tests (i.e. α = 0.002 as threshold for declaring statistical significance).

An important issue in analyzing the data from this family design is ascertainment. That is, we selected families based on a specific criterion (SAD) that is related to the candidate endophenotypes, which could influence the results. However, SAD was included as an independent variable in the co-segregation analyses, which is sufficient to correct for ascertainment (Monsees et al., 2009). In the heritability analyses, we corrected for ascertainment by using the proband correction available in the SOLAR software (Almasy and Blangero, 1998). Basically, SOLAR corrects for ascertainment by subtracting the likelihood of the probands from the likelihood of the rest of the sample (De Andrade and Amos, 2000, Hopper and Mathews, 1982). In this study, the target participant with SAD and his/her child with (sub)clinical SAD were indicated as ‘probands’ in SOLAR.

Results

Participant characteristics

Participants with SAD were older than participants without SAD, but there was no difference in estimated IQ between participants with and without SAD (Table 1). None of the participants with SAD were diagnosed with a current depressive episode, further descriptives of the groups in terms of clinical diagnoses can be found in Supplementary Table 2. We validated the differences between participants with and without SAD, by showing that participants with SAD reported more symptoms of social anxiety, fear of negative evaluation, and depression compared to participants without SAD (Table 1).

Table 1

Uncorrected means (and standard errors) of participants with and without SAD on the self-report questionnaires.

	Participants with SAD (12 females, 6 males)	Participants without SAD (47 females, 50 males)	β	p
Age	39.67 (3.24)	28.55 (1.56)	0.69	0.01
Estimated IQ	106.67 (2.82)	105.26 (1.13)	− 0.83	0.76
Social anxiety (z-score)	3.83 (0.49)	0.40 (0.14)	3.10	< 0.001
Fear of negative evaluation	31.89 (2.73)	13.49 (0.85)	18.69	< 0.001
Depression (z-score)	0.44 (0.20)	− 0.47 (0.07)	0.98	< 0.001

Note: SAD = social anxiety disorder.

Behavioral data

We tested whether the behavioral measures during the SJP were candidate endophenotypes of SAD (Fig. 3 and Supplementary Table 3). Co-segregation analysis with bias score revealed that participants with SAD expected rejection more often during the SJP than participants without SAD, β = − 14.39, p < 0.001. Co-segregation analysis with VAS ratings revealed that participants with SAD expected rejection more often before the SJP and had experienced rejection more often after the SJP than participants without SAD, β = − 12.05, p = 0.04. All participants had experienced rejection more often after the SJP, than they had expected before the SJP, β = − 12.88, p < 0.001.

Fig. 3

Means (1) and heritability analysis (2) for bias score (A), VAS ratings before and after the SJP (B), and reaction time for indicating acceptance and rejection-expectations for participants with and without SAD. Heritability results did not remain significant after correction for performing multiple tests.

Note: SAD = social anxiety disorder; h = heritability; SE = standard error; VAS = visual analogue scale; SJP = social judgment paradigm; RT = reaction time.

In addition, participants with SAD were overall slower than participants without SAD with indicating their expectations during the SJP, β = 260.66, p = 0.01. However, this was less the case for rejection-expectations, as indicated by the significant interaction between SAD and condition, β = − 135.78, p < 0.001. The main effect of condition, β = 31.25, p = 0.052, was probably driven by this interaction. Heritability analyses showed that none of the behavioral measures were heritable, the heritability estimate of reaction time for acceptance-expectations was only significant if we did not correct for multiple tests, h = 0.28, p = 0.02.

ERP data

N1

We exploratively studied the N1 component as a candidate endophenotype of SAD (Fig. 4 and Supplementary Table 3). N1 and SAD co-segregated within families, β = − 1.24, p = 0.01, with participants with SAD showing increased N1 across all conditions. The interaction between SAD and valence, β = 1.03, p = 0.02, was probably driven by the three-way interaction between SAD, congruency and feedback, β = 0.52, p < 0.001, showing that N1 was increased in all conditions except after expected acceptance feedback in participants with SAD. Heritability analyses revealed that N1 was not heritable, the heritability estimate of N1 after expected rejection feedback was only significant if we did not correct for multiple tests, h = 0.40, p = 0.03.

Fig. 4

ERP waves for the four conditions (negative values plotted upwards) (A), N1 amplitude for participants with and without SAD (B), and results of the heritability analysis (C). N1 was computed as an area measure in the time window from 130 to 220 ms at electrode FCz. Heritability results did not remain significant after correction for performing multiple tests.

Note: SAD = social anxiety disorder; NoNo = expected rejection; NoYes = unexpected acceptance; YesYes = expected acceptance; YesNo = unexpected rejection; h = heritability; SE = standard error. SE could not be computed if h = 0.

FRN

There was no co-segregation within families between SAD and FRN across conditions (Fig. 5 and Supplementary Table 3). The two-way interaction between congruency and valence, β = 0.60, p = 0.03, revealed that FRN was increased after unexpected rejection feedback compared to the other conditions. This effect was increased in SAD, as indicated by the three-way interaction between SAD, congruency and feedback, β = − 0.94, p < 0.001. Heritability analyses revealed that FRN was not heritable, the heritability estimates of FRN during expected and unexpected rejection feedback were only significant if we did not correct for multiple tests, respectively h = 0.48, p = 0.01 and h = 0.36, p = 0.02. FRN during unexpected rejection compared to the other conditions was not heritable, h = 0.002, p = 0.49.

Fig. 5

ERP waves for the four conditions (negative values plotted upwards) (A), FRN amplitude for participants with and without SAD (B), and results of the heritability analysis (C). FRN was computed using the peak-to-peak method in the time windows 240–340 and 290–390 ms at electrode Fz. Heritability results did not remain significant after correction for performing multiple tests.

Note: SAD = social anxiety disorder; NoNo = expected rejection; NoYes = unexpected acceptance; YesYes = expected acceptance; YesNo = unexpected rejection; h = heritability; SE = standard error. SE could not be computed if h = 0.

P3

There was no co-segregation within families between SAD and P3 across conditions (Fig. 6 and Supplementary Table 3). Overall, there was an effect of congruency, β = − 0.94, p = 0.001, and of valence, β = − 0.68, p = 0.01. The interaction between SAD and valence, β = − 1.02, p = 0.03, showed that P3 was increased after acceptance compared to rejection feedback for participants with SAD, but not for participants without SAD. Heritability analyses revealed that P3 was not heritable, the heritability estimates of P3 in response to expected and unexpected acceptance feedback were only significant if we did not correct for multiple tests, respectively h = 0.38, p = 0.01 and h = 0.41, p = 0.01.

Fig. 6

ERP waves for the four conditions (negative values plotted upwards) (A), P3 amplitude for participants with and without SAD (B), and results of the heritability analysis (C). P3 was computed as an area measure in the time window from 360 to 500 ms at electrode Pz. Heritability results did not remain significant after correction for performing multiple tests.

Note: SAD = social anxiety disorder; NoNo = expected rejection; NoYes = unexpected acceptance; YesYes = expected acceptance; YesNo = unexpected rejection; h = heritability; SE = standard error. SE could not be computed if h = 0.

Neural oscillatory power

Theta

There was no co-segregation within families between SAD and theta power across conditions (Fig. 7 and Supplementary Table 3). The two-way interaction between congruency and valence, β = 0.17, p < 0.001, revealed that theta power was increased after unexpected rejection feedback compared to the other conditions. This effect was increased in SAD, as indicated by the three-way interaction between SAD, congruency and feedback, β = 0.15, p < 0.001. Heritability analyses showed that theta power was not heritable, the heritability estimate of theta after expected acceptance feedback was only significant if we did not correct for multiple tests, h = 0.42, p = 0.03.

Fig. 7

Time-frequency plots for the four conditions (A), theta ratio change for participants with and without SAD (B), and results of the heritability analysis (C). Theta ratio change was computed within a time window from 300 to 500 ms at electrode Fz. Heritability results did not remain significant after correction for performing multiple tests. Note: SAD = social anxiety disorder; NoNo = expected rejection; NoYes = unexpected acceptance; YesYes = expected acceptance; YesNo = unexpected rejection; h = heritability; SE = standard error. SE could not be computed if h = 0.

Age and sex

Most variables (except P3 and theta) showed linear effect of age, and a quadratic effect of age (except bias score and VAS ratings). Most variables became stronger with increasing age and showed a peak between 20 and 40 years. Only the FRN became less strong with increasing age. There was no effect of sex on any of the behavioral, ERP or neural oscillatory findings.

Discussion

The goal of this study was to investigate whether behavioral and electrocortical responses to social evaluation are candidate endophenotypes of SAD. Using a validated paradigm (SJP) and a unique two-generation family design we tested two criteria for endophenotypes: co-segregation with SAD within families and heritability. Results revealed that participants with SAD expected rejection more often before and during the SJP, and had experienced rejection more often after the SJP. Reaction time associated with indicating their expectations during the SJP was longer in participants with SAD compared to participants without SAD. Electrocortical results revealed that increased N1 in response to all conditions, except after expected acceptance feedback, co-segregated with SAD. Increased FRN after unexpected rejection feedback compared to the other conditions co-segregated with SAD. P3 in response to acceptance versus rejection feedback co-segregated with SAD. Finally, increased theta power after unexpected rejection feedback compared to the other conditions co-segregated with SAD. The heritability estimates were not significant if we corrected for multiple tests. However, if we did not apply this correction, reaction time for acceptance expectations, N1 in response to expected rejection, and P3 in response to acceptance versus rejection feedback would be heritable.

As predicted, participants with SAD expected rejection more often before and during the SJP, and had experienced rejection more often after the SJP than participants without SAD. In general, people show an optimism bias in the SJP; they expect to be accepted more than rejected (Dekkers et al., 2015, Gunther Moor et al., 2010a, Van der Molen et al., 2017, Van der Molen et al., 2014, Van der Veen et al., 2016). The currently observed pessimism bias in SAD corroborates behavioral findings on the Island Getaway task (Cao et al., 2015), and cognitive-behavioral findings suggesting that patients with SAD predict future social events more negatively (Hirsch and Clark, 2004). This is an important focus of cognitive-behavioral therapy in SAD (Heimberg, 2002). In addition, the reaction time associated with indicating expectations co-segregated with SAD, extending findings on fear of negative evaluation in healthy females (Van der Molen et al., 2014). This might suggest that participants with SAD are less certain about their choices, or show more self-focused attention and vigilance during the SJP (Van der Molen et al., 2014). The heritability estimate for reaction time associated with indicating acceptance-expectations was only significant if we did not correct for multiple tests, suggesting that this might be a candidate behavioral endophenotype of SAD. This could mean that the other behavioral measures are symptoms of SAD instead of mechanisms underlying the development of SAD. Together, these behavioral findings showed that the SJP is a useful task to measure responses to social evaluation in SAD.

Patients with SAD showed an increased N1 in all conditions of the SJP, except after expected acceptance feedback. Although we had no a priori predictions regarding this component in the SJP, this finding is in accord with cognitive-behavioral studies showing hypervigilance to socially threatening stimuli (Bögels and Mansell, 2004, Clark and McManus, 2002, Heinrichs and Hofmann, 2001, Hirsch and Clark, 2004, Morrison and Heimberg, 2013), and ERP studies showing increased early attentional ERPs in SAD (Harrewijn et al., 2017b, Staugaard, 2010). Indeed, the N1 is related to attention (Luck, 2005, Luck and Kappenman, 2013), and increased for emotional compared to neutral stimuli (Hajcak et al., 2013). So, the N1 might reflect an early attentional bias towards socially threatening stimuli. The N1 in our study was not increased after expected acceptance feedback, probably because this condition is the least threatening. The heritability estimate of N1 in response to expected rejection feedback was only significant if we did not correct for multiple tests. Thus, N1 after expected rejection feedback might be a candidate endophenotype of SAD. Our study was the first to show that the N1 is an important component to study in the SJP and might be a candidate endophenotype of SAD.

Participants with SAD showed an increased FRN after unexpected rejection feedback compared to the other conditions. This finding shows that both congruency and valence of social feedback modulate the FRN in social anxiety, whereas in healthy participants the FRN is only sensitive to congruency in the SJP (Dekkers et al., 2015, Van der Molen et al., 2017, Van der Molen et al., 2014). Increased FRN after unexpected rejection feedback in SAD might reflect that the usual FRN response to incongruent feedback is intensified by a selective bias for negative evaluation (Clark and McManus, 2002, Heinrichs and Hofmann, 2001, Hirsch and Clark, 2004). We observed a similar effect in theta power: increased theta power after unexpected rejection feedback co-segregated with SAD. This result corroborates previous findings that theta power is increased after unexpected rejection feedback in the SJP (Van der Molen et al., 2017). Some have suggested that theta power is related to processing social threat (Cristofori et al., 2013, Van Noordt et al., 2015). This would indicate that unexpected rejection feedback is the most threatening condition in the SJP, a notion that is substantiated by heart rate studies using the SJP (Dekkers et al., 2015, Gunther Moor et al., 2014, Gunther Moor et al., 2010a, Van der Veen et al., 2014). Here we demonstrate that this effect is exaggerated in SAD, suggesting that unexpected rejection feedback is even more threatening for participants with SAD. This is in line with cognitive-behavioral studies showing that patients with SAD interpret mildly negative social events in a catastrophic way (Clark and McManus, 2002), and show extreme fear of negative evaluation (APA, 2013, Rapee and Heimberg, 1997). Interestingly, in SAD, both phase-locked (FRN) and induced oscillatory power (theta) are modulated by congruency and valence of social evaluative feedback. Cavanagh and Shackman (2015) argue that both the FRN and theta power are generated by the midcingulate cortex and might signal the need for adaptive control in uncertain situations. Receiving unexpected rejection feedback might reflect such a situation because there is uncertainty about the optimal course of action. Together these findings show that FRN and theta power are promising electrocortical markers of SAD, but did not meet the criteria of endophenotypes because they were not heritable. This might suggest that FRN and theta power are more influenced by environmental factors. Indeed, previous studies have found that neural correlates in response to rejection are related to environmental factors such as chronic rejection during childhood (Will et al., 2016), time spent with friends (Masten et al., 2012), attachment (White et al., 2013, White et al., 2012), early separation experiences (Puetz et al., 2014), and maltreatment (Puetz et al., 2016).

The P3 was larger for acceptance than rejection feedback, regardless of feedback congruency, in participants with SAD, but not in participants without SAD. This is in line with the interpretation of the P3 as an index of processing emotionally motivational stimuli (Hajcak et al., 2013). This would suggest that acceptance feedback is even more important for participants with SAD than participants without SAD, which is in line with cognitive theories emphasizing the importance of positive social evaluation for patients with SAD (Rapee and Heimberg, 1997). The P3 might also reflect reward processing (Van der Veen et al., 2016, Van der Veen et al., 2014), but this interpretation was based on an increased P3 only after expected acceptance. Our P3 results might be explained by subjective probability (Ferdinand et al., 2012, Johnson, 1986). That is, participants with SAD probably expect less acceptance by peers and due to this low subjective probability of acceptance feedback, P3 amplitudes in response to acceptance feedback are increased. This is supported by the behavioral data showing that participants with SAD expected to be accepted less often, and needed more time for indicating their expectations than participants without SAD. Heritability estimates of P3 in response to expected and unexpected acceptance feedback were only significant if we did not correct for multiple tests, suggesting that this might be a candidate endophenotype of SAD.

This unique two-generation family design has given us the opportunity to study two endophenotype criteria: co-segregation with SAD within families and heritability. Since this is the first study administering the SJP in participants with SAD, our findings should be confirmed in future research. Future research should also focus on specificity of these measures for SAD. A few limitations should be taken into account. First, since only few non-target family members were diagnosed with SAD, we could not calculate the genetic correlation between SAD and the candidate endophenotypes. Second, none of the heritability estimates survived corrections for performing multiple tests. Thus, although we found interesting results on behavioral and electrocortical responses to social evaluation in SAD, the robustness of these effects should be validated. Third, gene-environment interactions could also have played a role in these results, since we were not able to correct for shared environmental effects. Fourth, participants with and without SAD showed a different number of artifact-free trials, which is inherent to the behavioral finding that participants with SAD expected rejection more often. Fifth, the degree to which the currently reported endophenotypes are specific to SAD remains uncertain, particularly due to the co-occurrence of depressive symptoms in participants with SAD. Since SAD and depression are overlapping constructs (Cerda et al., 2010, Hettema et al., 2015, Mineka et al., 1998), controlling for depressive symptoms would have been invalid. That is, controlling for depression would remove important variance relevant to SAD (Miller and Chapman, 2001), as participants with SAD reported significantly higher levels of depressive symptoms than those without SAD. It should be noted, however, that the patients in the current study had SAD as primary diagnosis, and did not have a current depressive episode. Moreover, a recent study using the same paradigm has shown that the behavioral and electrophysiological responses to social evaluation were not related to depressive symptoms in healthy adults (Van der Veen et al., 2016). We acknowledge that specificity of these candidate endophenotypes should be an important focus in future studies. One approach would be to compare patients with SAD with and without comorbid depression. Another approach would be to cross syndrome boundaries and focus on traits shared across disorders (Levy and Ebstein, 2009). Such future endeavors would be of critical importance to determine specificity of these endophenotypes, or whether they could instead be conceptualized as transdiagnostic markers that will aid in understanding the etiology of psychopathology.

To conclude, in the present study reaction time for indicating acceptance-expectations might be a candidate behavioral endophenotype of SAD, possibly reflecting increased uncertainty or self-focused attention and vigilance during social evaluation. At the electrocortical level, this vigilance seems tracked by an increased N1 after rejection feedback and unexpected acceptance. At later processing stages, we observed increased P3 amplitudes to acceptance feedback as endophenotype of SAD. These behavioral and electrocortical endophenotypes might provide insight in genetic mechanisms underlying SAD (Cannon and Keller, 2006, Glahn et al., 2007, Iacono et al., 2016, Miller and Rockstroh, 2013). Future research should validate these findings, and investigate whether training these attentional biases might prevent the development of SAD in persons with a genetic vulnerability. Another interesting venue for future research is investigating how parents might influence this hypervigilance for socially threatening stimuli and/or focus on positive feedback in their children with a genetic vulnerability for SAD.

Conflicts of interest

None.