Seeing a Bayesian ghost: Sensorimotor activation leads to an illusory social perception.

Based on our prior experiences we form social expectations and anticipate another person's response. Under certain conditions, these expectations can be so strong that they lead to illusory perception of another person who is actually not there (i.e., seeing a Bayesian ghost). We used EEG to investigate the neural correlates of such illusory social perception. Our results showed that activation of the premotor cortex predicted the occurrence of the Bayesian ghost, whereas its actual appearance was later accompanied by activation in sensorimotor and adjacent parietal regions. These findings confirm that our perception of others is so strongly affected by prior expectations, in such a way they can prompt illusory social perceptions associated with activity change in brain regions relevant for action perception. They also contribute to a better understanding of social interaction in healthy individuals as well as persons with mental illnesses, which can be characterized by illusory perception and social interaction difficulties.

Introduction

To navigate through our complex world, we use prior experience to generate expectations and anticipate events. Our expectations then shape the way we perceive our environment, particularly in situations that require complex information processing such as social interactions. The ability to anticipate future events is crucial in social interactions, in which an action of one person may be predictive of a response by another. In addition, communicative actions by one person make it more likely to perceive another person to whom the action might be directed at (Manera et al., 2011b).

The use of prior experience to generate expectations about the presence and consequent action of another person has been described as interpersonal predictive coding (Manera et al., 2011b, 2011a; von der Lühe et al., 2016). Predictive coding accounts suggest that human beings possess Bayesian brains: The brain generates predictions and compares these to sensory input, which is especially advantageous in complex or noisy environments (Manera et al., 2011a). Interpersonal predictive coding applies this logic to the social domain and focuses on how prior knowledge and resulting expectations help us to understand social interactions, during which the behavior of one person can help to anticipate what the interaction partner will do (Bolis et al., 2018; von der Lühe et al., 2016).

However, predictive coding can also produce erroneous perception. Strong expectations can result in illusions, i.e., seeing a communicative action of one person increases the likelihood of expecting and actually seeing another person who, in fact, is not there. This special erroneous outcome of predictive coding has been referred to as seeing a Bayesian ghost (Manera et al., 2011b; von der Lühe et al., 2016). We have adopted this term to describe the instance in which one has an illusory perception of a person because of prior expectations generated by another’s communicative gesture. Thus, the Bayesian ghost is a metaphorical term and does not refer to Bayesian statistics or modeling in this paper.

The underlying neural mechanisms for this form of illusory social perception remain elusive. To address this, we recorded the brain’s electrical activity using electroencephalography (EEG) while neurotypical participants performed an established experimental task using two point-light agents (Figure 1) (Zillekens et al., 2019). The screen displayed a well recognizable point-light walker (agent A) and a second point-light walker individually masked or replaced by randomly moving noise dots (agent B). Participants were asked to passively watch agent A first, who either performed an individual (i.e., individual condition (IND)) or communicative (i.e., communicative condition (COM)) gesture (i.e., time segment (1) in Figure 1B). They were then required to passively observe agent B who responded to agent A’s gesture. Agent B was masked by noise dots (i.e., signal trials) or replaced completely by noise dots (i.e., noise trials) in half of the trials each (i.e., time segment (2) in Figure 1B). At the end of each trial, the participants’ task was to decide whether the masked agent B was present or absent (i.e., time segment (3) in Figure 1B). The response of the participant determined whether the trial was a false alarm (agent absent, response “yes”), correct rejection (agent absent, response “no”), hit (agent present, response “yes”), or a miss (agent present, response “no”) trial. In contrast to individual actions, communicative gestures have previously been shown to improve the visual detection of the presence of a second agent (Manera et al., 2011a) and to prompt the perception of a second agent, irrespectively of whether it was present or not (Manera et al., 2011b).

Figure 1

Experimental design (adapted from Zillekens et al. (2019))

(A) Experimental conditions. In the communicative condition (COM; top row), agent A performs a communicative action. In the left panel, agent B is present and reacts in accordance to it (signal trial). In the right panel, agent B is replaced by randomly moving dots (noise trial). In the individual condition (IND; bottom row), agent A performs an individual action. Agent B is again either present (left panel) or replaced by noise (right panel). The gray silhouettes serve illustrative purposes and were not visible for participants. The occurrence of a Bayesian ghost (indicated in blue) is defined by a false alarm (FA) in the communicative noise trial (i.e., the participant indicated that agent B was present when in fact noise dots were presented). The number of false alarms is illustrated in Figure S1.

(B) Structure of experimental trials. Jittered inter-trial-intervals (ITI) preceded a fixation cross appearing at the subsequent position of agent A. Following this, the video of the two point-light agents started. The video duration as well as agent B’s onset varied depending on the specific actions performed. The analyzed time segments are illustrated in color: (1) 1-s before agent B’s onset (red): Participants passively observed agent A, who either performed an individual or communicative gesture; (2) 1-s after agent B’s onset (gray): Participants passively watched the responding agent B or noise dots; and (3) 1-s before the response (blue): Participants decided whether the masked agent B was present or absent and indicated their choice with a button press.

In our experimental design, the presence of a Bayesian ghost would be indicated by the participant subjectively perceiving a second agent among randomly moving noise dots after seeing an actual first agent performing a communicative gesture (Manera et al., 2011b). Importantly, this can only occur in trials in which there was objectively no second agent present within the moving dots, i.e., participants’ responses represented a false alarm (in contrast to a correct rejection) in communicative noise trails (Figure 1A). The individual condition served as a control condition to ensure that the findings were specific to a social context. We expected that the communicative condition would lead to more false alarms than the individual one (Manera et al., 2011b).

On the neural level, we hypothesized that activation of the sensorimotor system first predicts the occurrence and then accompanies the appearance of the Bayesian ghost.

Firstly, we expected activation of the sensorimotor system in the time window in which agent A was performing a communicative action but agent B was not responding yet (i.e., time segment (1) before agent B’s onset). This increased sensorimotor activity should predict the occurrence of the Bayesian ghost later on in the trial. This hypothesis is based on previous studies, which showed that the expectation of an upcoming action from someone else in a predictable context, leads to the activation of one’s own sensorimotor system (Kilner et al., 2004; Krol et al., 2020; Maranesi et al., 2014). Activation of the sensorimotor system before the movement observation is usually advantageous, as it enables us to anticipate the behavior of others. However, in our case, this anticipation might lead to seeing a Bayesian ghost, i.e., to generate a false alarm in the communicative condition. In the individual condition, a pre-activation of the sensorimotor system was not expected, as the self-oriented, individual gesture of agent A did not allow for any predictions of agent B’s behavior.

Secondly, we hypothesized that the sensorimotor system should also be activated during the appearance of the Bayesian ghost in the communicative condition in the time segment (3) before the response. If participants really “see” the second agent (although it is not present), their own sensorimotor system should be facilitated by the illusory perception of the ghost’s action (Schütz-Bosbach et al., 2006). In addition, the sensorimotor system might be required to fill in the gaps between the point-lights, so the single points are perceived as biological motions; therefore, a second agent rather than noise is seen (Saygin et al., 2004; Ulloa and Pineda, 2007).

Activation of the sensorimotor system has previously been linked to a power decrease in alpha and beta frequency bands in the EEG (Neuper and Pfurtscheller, 2001; Sauseng et al., 2009). Therefore, we expected to find lower power in alpha and beta frequency bands as an indicator of more activation in the sensorimotor cortex in false alarm (i.e., having an illusion of observing a moving agent) than correct rejection trials (i.e., seeing noise dots) in the communicative condition. This is in contrast to the individual condition as only the communicative condition allows participants to generate expectations for agent B’s action in response to agent A’s communicative gestures.

Note that we did not contrast the communicative against the individual condition, but we compared false alarms to correct rejections within the same condition. This way, not only the task per se but also the visual input, the task load and difficulty, the context (i.e., individual or social situation), and the contingency (i.e., semantic match or mismatch between the two agents’ actions) were the same within our contrasts and cannot be confounding factors (Wurm and Caramazza, 2019). This is also the reason why we did not expect to find differences in occipitotemporal brain regions, which are typically activated in the visual processing of social interactions (Walbrin and Koldewyn, 2019; Wurm and Caramazza, 2019).

The only difference between false alarm (i.e., agent absent, response “yes”) and correct rejection (agent absent, response “no”) trials is the response itself. To investigate whether a general difference between a “yes” and a “no” response could account for our results, we compared the false alarm (i.e., agent absent, response “yes”) to the hit trials (agent present, response “yes”) in the communicative condition post hoc. This exploratory analysis should also shed light on the interesting question of how the brain activation during the occurrence and appearance of the Bayesian ghost (i.e., false alarm) differs from the actual display and correct detection of agent B (i.e., hit) in the communicative condition.

Results

Behavioral results

More false alarms in the communicative than individual condition

We calculated a repeated-measures ANOVA for the number of responses with the variables ‘type of response’ (i.e., false alarms or correct rejections) and ‘condition’ (i.e., communicative or individual). The main effect of ‘condition’ was not significant (F1,38 = 0.39, p = 0.54, partial η2 = 0.10), whereas the main effect of ‘type of response’ was significant (F1,38 = 60.40, p < 0.01, partial η2 = 0.61). Unsurprisingly, participants had more correct rejections (M = 42, SD = 11.70, N = 39) than false alarms (M = 18, SD = 9.57, N = 39). This is inherent to the design as the difficulty was individually adjusted for the participants to achieve an average performance of about 70%. Most importantly, however, the interaction of ‘condition’ and ‘type of response’ was also significant (F1,38 = 6.64, p < 0.05, partial η2 = 0.15). More false alarms were made in the communicative (M = 19, SD = 10, N = 39) than in the individual (M = 17, SD = 9, N = 39) condition. Furthermore, there were fewer correct rejections in the communicative (M = 41, SD = 12, N = 39) than in the individual (M = 43, SD = 12, N = 39) condition. For a graphical representation, please refer to the supplementary material (Figure S1).

Neural correlates

Increased sensorimotor activation in false alarm than correct rejection trials in the communicative condition

In the communicative condition, in time segment (1) before agent B’s onset, significantly lower power (indicating higher cortical excitability) was found in false alarm than correct rejection trials at alpha (8–12 Hz) and lower beta (13–17 Hz) frequency bands in the left premotor cortex (alpha: t = −3.72, p < 0.05, N = 20, dz = 0.83, Figures 2A and S2; beta: t = −3.91, p < 0.05, N = 20, dz = 0.87, Figures 2B and S3). In the lower beta band, significant voxels also spread to the primary motor cortex (t < −3.74, p < 0.05, N = 20, dz > 0.84, Figures 2B and S3).

Figure 2

Increased sensorimotor activation in false alarm than correct rejection trials in the communicative condition

For each of the three significant results (A–C), the significant voxels over participants are indicated in turquoise in the sagittal view of the brain (for other views, please see Figures S2–S4). For the most significant voxel, the power values for every participant as well as for the mean (turquoise points and line) are illustrated in the line chart for the false alarm (FA) and correct rejection (CR) trials. In the time segment (1) before agent B’s onset (in red), a power decrease in false alarm — compared to correct rejection trials — was found in (A) the alpha frequency band (8–12 Hz) in the left frontal lobe, Brodmann Area 6, premotor cortex (peak significance at MNI coordinates: −40/0/45; p < 0.05) and in (B) the lower beta band (13–17 Hz) in the left frontal lobe, Brodmann Area 6, premotor cortex (peak significance at MNI coordinates: −45/−10/60; p < 0.05) with significant activation spreading to voxels in the adjacent primary motor cortex, Brodmann Area 4.

(C) In the time segment (3) before the response (in blue), a power decrease in false alarm — compared to correct rejection trials — reached significance in the upper beta band (18–25 Hz) in the right postcentral gyrus, Brodmann Area 3, primary somatosensory cortex (peak significance at MNI coordinates: 20/−35/55; p < 0.05). Significant voxels were also present in the Brodmann Area 2 of the primary somatosensory cortex. In addition, the significant difference spread to the adjacent right precentral gyrus, Brodmann Area 4, primary motor cortex, and to the other following right parietal regions: Brodmann Area 5 (paracentral gyrus, postcentral gyrus), Brodmann Area 7 (postcentral gyrus, superior parietal lobule, precuneus), and Brodmann Area 40 (p < 0.05). For an illustration of the control contrasts false alarm versus correct rejection trials in the individual condition and false alarm versus hit trials in the communicative condition, please refer to the Figures S5–S7.

In the time segment (3) before the response, significantly lower power was shown in false alarm than correct rejection trials in the upper beta frequency band (18–25 Hz) in the right somatosensory cortex (t = −4.27, p < 0.05, N = 20, dz = 0.96, Figures 2C and S4). This significant difference also spread over the adjacent primary motor cortex and other adjacent regions of the parietal cortex (t < −3.64, p < 0.05, N = 20, dz > 0.81, for more details please see Figures 2C and S4).

No significant differences were found in the time segment (2) after agent B’s onset.

No difference in sensorimotor activation between false alarm and correct rejection trials in the individual condition

As a control condition, we calculated the same contrasts in the individual condition that became significant in the communicative condition. In the time segment (1) before agent B’s onset, false alarm trials showed lower power than correct rejection trials in the upper beta band in the bilateral anterior cingulate cortex (t = −4.81, p < 0.05, N = 17, dz = 1.17, Figure S5). We could not find any significant differences in the sensorimotor cortex or any other brain region in the individual condition.

Increased sensorimotor and parietal activation in false alarm than hit trials in the communicative condition

As another and exploratory control analysis, we compared the false alarm to hit trials in the same contrasts that became significant between false alarm and correct rejection trials in the communicative condition. In the time segment (1) before agent B’s onset, significantly lower power was found in false alarm than hit trials at lower beta frequency band in the left primary somatosensory cortex (t = −5.55, p < 0.05, N = 20, dz = 1.24). This significant difference also spread over the adjacent primary motor and premotor cortex (t < −4.07, p < 0.05, N = 20, dz > 0.91, Figure S6).

In the time segment (3) before the response, false alarm trials showed lower power than hit trials in the upper beta band in the right superior parietal lobule (t = −4.01, p < 0.05, N = 20, dz = 0.90, Figure S7).

Discussion

The Bayesian ghost is predicted and accompanied by sensorimotor and parietal activation

The goal of this study was to investigate the neural correlates of an illusory social perception. Because of our expectations, communicative gestures can affect our perception in such a way that we see a person who is actually not there. In other words we see a Bayesian ghost (Manera et al., 2011b; von der Lühe et al., 2016).

Our behavioral results confirmed that participants, indeed, made more false alarms (i.e., indicated that they saw an agent who was actually not present) in the communicative than in the individual condition (Manera et al., 2011b).

We also found evidence in favor of our hypothesis that an activated sensorimotor system in the communicative condition is predictive of whether a participant will see a Bayesian ghost or not. In the time segment (1) before agent B’s onset, agent A’s communicative gestures allowed participants to predict agent B’s action. We found that an increased activation of the premotor cortex (i.e., a power decrease in alpha and lower beta frequency bands) predicted the occurrence of the Bayesian ghost later in the trial (i.e., a false alarm). Our result offers further evidence that activation of the sensorimotor system is associated with the expectation of an upcoming action from someone else in a predictable context (Kilner et al., 2004; Krol et al., 2020; Maranesi et al., 2014). Moreover, it extends this to the case of dyadic social interaction, in which the actions of agents are typically linked in an interpersonal manner.

The involvement of the premotor cortex in anticipating observed actions within a predictable context has been interpreted as mirror neuron activity (Kilner et al., 2004; Krol et al., 2020; Maranesi et al., 2014; Rizzolatti et al., 2001). This is in line with a computational approach assuming that the mirror neuron system uses predictions to infer intentions from observed movements (Kilner et al., 2007). Especially, alpha but also beta oscillations were shown to convey top-down predictions (Michalareas et al., 2016; Tarasi et al., 2021) besides their involvement in tasks related to the observation of movement (Hari et al., 1998; Neuper et al., 2009; Pineda, 2005).

In the time segment (2) after agent B’s onset, the mere start of the movement of the noise dots did not elicit any specific brain activation. This indicates that the increased activity in the sensorimotor system during false alarm compared to correct rejection trials was not continuous. Instead, this activation seems specific to the formation of expectations about agent B’s action (i.e., time segment (1)) and then to the actual appearance of the Bayesian ghost (i.e., time segment (3)).

In the time segment (3) before the response, the appearance of the Bayesian ghost in the communicative condition was accompanied by increased activation (i.e., a power decrease in upper beta frequency band) in the primary somatosensory cortex as well as the adjacent primary motor cortex and further parietal regions. This is in line with our hypothesis that an activation of the sensorimotor system is required to perceive an (illusory) moving agent from single point-lights (Saygin et al., 2004; Schütz-Bosbach et al., 2006; Ulloa and Pineda, 2007).

In this later time segment, higher beta oscillations occur in association with the appearance of the Bayesian ghost than in time segment (1) before agent B’s onset. Slower brain oscillations (i.e., starting from theta frequencies) are indicative of higher cognitive processes, top-down control, and larger network processes, whereas faster oscillations (up to high gamma frequencies) are more indicative of local bottom-up processes (Michalareas et al., 2016; Von Stein and Sarnthein, 2000). This relationship might also apply to our finding: Generating strong predictions based on communicative gestures in time segment (1) is associated with slower brain oscillations than processing the sensory information by trying to find agent B in a cloud of noise dots in time segment (3).

With respect to the activation of the sensorimotor cortex in time segment (3) before the response, there is evidence that mirror neurons are also present in the primary motor cortex (Kilner and Lemon, 2013) and several studies emphasize the importance of somatosensory areas for action observation and mirror neuron activity (Caspers et al., 2010; Keysers et al., 2010; Pineda, 2008; Valchev et al., 2016). Moreover, alpha and beta rhythms over the sensorimotor cortex (i.e., mu rhythms) were suggested to serve as an indicator of mirror neuron activity in somatosensory, as well as premotor and inferior parietal cortices (Arnstein et al., 2011; Bimbi et al., 2018; Fox et al., 2016). The mu rhythm was shown to be modulated by action observation through a contribution of motor and mirror neurons (Bimbi et al., 2018) and involved in action inference by combining single point-lights to biological movements (Ulloa and Pineda, 2007). It is worth noting that there are also critical studies questioning the theory of the mirror neuron system to explain high-level action understanding in humans and the mu rhythm as a reliable marker for mirror system activity. Please see Heyes and Catmur (2021) for a recent perspective on this topic.

Besides the sensorimotor cortex, adjacent parietal regions were also activated during the appearance of the Bayesian ghost. Brodmann area 40 was found to be involved in mirror-like mechanisms during action observation (Del Vecchio et al., 2020) and together with Brodmann area 7 (superior parietal lobule) to be active during imitation tasks (Molenberghs et al., 2009). Moreover, the superior parietal lobule is associated with action observation (Caspers et al., 2010) and involved in biological motion perception (Pavlova, 2012). The precuneus is involved in motor and mental imagery, processing of visuospatial information, as well as processing and understanding intentions and actions (Cavanna and Trimble, 2006).

The superior temporal sulcus was not significantly more activated during false alarm than correct rejection trials within the communicative condition. The superior temporal sulcus was found to be involved in mirror neuron activity (Rizzolatti et al., 2001) and in perception of biological motion and social interaction (Caspers et al., 2010; Isik et al., 2017; Pavlova, 2012). Its role is the visual processing of socially-relevant features such as orientation of agents or directedness of actions (Wurm and Caramazza, 2019). As the visual features and social content were exactly the same between false alarm and correct rejection trials, we did not expect any difference in the superior temporal sulcus.

The only discernible difference is that in false alarm trials, participants form predictions that are so strong that they outweigh the bottom-up perception (which is the same between trials). In addition, this top-down control is realized by activation of the sensorimotor cortex. This interpretation is in line with the predictive coding framework of the mirror neuron system (Kilner et al., 2007). It is also supported by neurophysiological evidence that the sensorimotor cortex is not merely driven by visual input of an observed action but can generate a motor representation of another person’s action without visual input (Umiltà et al., 2001).

Nonetheless, other theories or models also have to be taken into account as potential alternative explanations of our findings. The activation of the sensorimotor and parietal cortex also plays an important role in the Comparator Model to understand voluntary actions and sense of agency (Haggard, 2008, 2017). It has been proposed that an action starts with an intention and a motor command that aims to fulfill a goal. Then, predictions of the probable sensory outcome of one’s motor action are formed and later compared to the actual sensory feedback. If the prediction error is zero, a sense of agency emerges. Although predicting and testing the hypothesis about ongoing actions are potentially related to our observed effect, false alarm and correct rejection trials do not differ in processes related to intention, volition, or sense of agency. Thus, this model does not seem to offer an alternative explanation for our results.

To sum up, our findings are in favor of our hypothesis and suggest that not only the sensorimotor cortex but a network of action-observation brain areas are contributing to the perception of the Bayesian ghost. As the activation of the sensorimotor cortex is indicative of movement observation, this finding suggests that participants really “saw” agent B moving in the communicative condition (although it was not present). Given our experimental design, agent B could only be perceived by its biological motion, which participants indicated in their behavioral response.

Importantly, the activation of the sensorimotor system predicted the occurrence and accompanied the appearance of the Bayesian ghost specifically in the communicative condition. This was in contrast to the individual condition, in which no predictions for agent B’s action could be made based on agent A’s gestures and in which no activation over the sensorimotor cortex or adjacent parietal regions was found.

Different responses are not confounding factors

As already mentioned in the introduction, the only difference between trials in our calculated contrast was the given answer that indicated whether the participant saw an agent (i.e., false alarm) or not (i.e., correct rejection). Thus, possible differences in performance (false or correct answer) and response bias (“yes” or “no” answer) could be the only possible alternative interpretation. Recent studies have indicated that pre-stimulus alpha power is associated with response bias rather than with performance (Benwell et al., 2021; Iemi et al., 2017). Although these studies investigated the pre-stimulus alpha power using simple visual experimental designs in which participants had to indicate if they perceived a simple masked or near-threshold stimuli e.g., (Benwell et al., 2021; Iemi et al., 2017), this hypothesis could theoretically also be applied to complex stimuli in a social context.

The decreased pre-stimulus alpha power in the communicative condition could facilitate stimulus perception and lead to a more liberal criterion (i.e., a tendency to say “yes”). Indeed, the communicative condition prompts the appearance of a second agent and thus biases participants more toward reporting the presence of a second agent (i.e., adopting a more liberal response strategy) in the communicative than in the individual condition (Manera et al., 2011b; Zillekens et al., 2019).

However, we argue that the decreased pre-stimulus alpha power found in the communicative condition was indicative of a prediction of the Bayesian ghost rather than the mere neural correlate of a “yes” answer or a more liberal criterion, because: (1) changes in the pre-stimulus alpha power associated with response bias in visual tasks were found in posterior rather than central regions (Limbach and Corballis, 2016; Romei et al., 2008); (2) the here-reported pre-stimulus effects in the sensorimotor cortex were not restricted to the alpha band but occurred even stronger in the lower beta band. Both frequency bands are indicative of tasks related to the observation of movement (Hari et al., 1998; Neuper et al., 2009; Pineda, 2005) and were shown to convey top-down predictions (Michalareas et al., 2016; Tarasi et al., 2021); (3) the comparison of false alarm versus correct rejection trials represents “yes” versus “no” answers rather than response tendencies, as hits and misses are not included in the main contrast analyses. If the pre-stimulus power decrease in the communicative condition would indeed illustrate a mere “yes” response, a similar effect should have been evident in the hit trials. In contrast, hit trials showed significantly less pre-activation of the sensorimotor cortex than false alarm trials, although they both are defined by a “yes”response.

Brain activation during social illusion compared to the presence of agent B might help to understand mental disorders

In the time segment (1) before agent B’s onset, the sensorimotor cortex was more activated in false alarm compared to hit trials in the communicative condition. This finding supports our claim that the activation of the sensorimotor system is a specific feature of the Bayesian ghost and not generally indicative for watching agent A performing a communicative action or preparing a “yes” response. This activation of the sensorimotor cortex in time segment (1) indicates that predictions were generated that are so strong that they outweighed the processing of the sensory information in time segment (3), which in turn led to a false alarm.

In time segment (3) before the response, the superior parietal cortex was more activated in false alarm than in hit trials in the communicative condition. This suggests that the superior parietal cortex was involved in creating the illusion of biological motion in absence of the actual agent B (while the sensorimotor cortex might be involved in both - the actual and the illusory observation of a moving agent). This result is especially interesting under a clinical aspect in terms of mental disorders in which social interaction is impaired and illusory perception can occur. As this analysis was exploratory, we did not have a priori hypotheses about the involved brain regions. However, our finding is supported by studies revealing that the superior parietal cortex showed structural brain alterations in patients with visual hallucinations (Cachia et al., 2015; Delli Pizzi et al., 2014) and aberrant functional connectivity in patients with schizophrenia (Schilbach et al., 2016; Tarasi et al., 2021).

A recent publication proposed that schizophrenia with positive symptoms and autism spectrum disorder can be understood within the Bayesian predictive coding framework as opposite poles of a continuum (Tarasi et al., 2021). Individuals with autism spectrum disorder underutilize predictions and contextual information and rely rather on external stimulus-related sensory information. In contrast, individuals with schizophrenia and positive symptoms are thought to base their perception too strongly on prior expectations (Fletcher and Frith, 2009).

This theory is supported by a previous study from our lab using a similar experimental design as the present one (von der Lühe et al., 2016): Here, it was shown that individuals with autism spectrum disorder did not use the contextual information of communicative gestures, although they did perceive the gestures themselves correctly. Thus, they did not have a performance advantage in the communicative as compared to the individual condition.

Following this framework and logic, individuals with schizophrenia should rely on their predictions based on the communicative gestures and thus make more false alarms in the communicative condition (i.e., have more illusions of the Bayesian ghost). That would be an interesting hypothesis to investigate in future studies. As a recent study showed that hallucination proneness is correlated with the detection of the presence of socially relevant signals in noisy and ambiguous stimuli (Stuke et al., 2021), this could even be tested first in a non-clinical population.

Activation differences in the anterior cingulate instead of the sensorimotor cortex in the individual condition

In line with our hypothesis, the activation of the sensorimotor cortex predicted the occurrence and accompanied the appearance of the Bayesian ghost specifically in the communicative condition. In contrast, in the individual condition, we found lower power in the time segment (1) before agent B’s onset in the beta band in the anterior cingulate cortex for false alarm compared to correct rejection trials. An activation of the anterior cingulate cortex was associated with decision-making in the presence of only little predictive information (Domenech and Dreher, 2010; Myers and Wyart, 2011). As no predictions could be made based on agent A’s action for both false alarm and correct rejection trials in the individual condition, we cannot entirely explain this difference between trials. We can only speculate that the higher activation of the anterior cingulate cortex reflects the failure to gather enough predictive information based on agent A’s individual gesture, which was more likely to lead to an incorrect response.

Conclusion

In this study, we investigated the neural correlates of seeing a Bayesian ghost. We used the term “Bayesian ghost” to describe the phenomenon in which one has an illusory perception of a person responding to a social cue because of prior expectations in social interactions. Our results demonstrated that activation of the premotor cortex predicted the occurrence of seeing a Bayesian ghost. This pre-activation of the sensorimotor cortex was indicative of the formation of predictions about agent B’s response to agent A’s social gesture. When these top-down predictions outweighed the processing of the sensory information in the later time segment, the Bayesian ghost occurred (i.e., a false alarm was made). The actual appearance of the Bayesian ghost was then accompanied by activation in the sensorimotor cortex and adjacent parietal regions. While the activation of the sensorimotor cortex was involved in action perception (no matter whether the observed movements were real or illusory), activation in the superior parietal cortex was associated with the creation of the illusion of biological motion in absence of the actual agent B.

This provides a striking confirmation for the idea that our perception of the presence of others is strongly influenced by our prior expectations. Importantly, we do not merely react to what we see, but rather use this information to predict the presence and social behavior of others. Such an intuitive and effortless anticipation of others’ actions, intentions, and goals is crucial for successful social interaction, yet, as demonstrated here, can also lead us to anticipate and actually see agents where there are none.

Limitations of the study

The limitation of this study was that out of the original sample of 41 participants, only the data of 20 participants could be included in the EEG analyses for the communicative and 17 for the individual condition. The reason is that it was difficult to find participants who achieved a performance of about 70% but still made enough false alarms to allow reliable EEG analyses.

STAR★METHODS

Key resources table

Resource availability

Lead contact

Further information and requests should be directed to and will be fulfilled by the lead contact, Elisabeth V. C. Friedrich (elisabeth.friedrich@psy.lmu.de).

Materials availability

This study did not generate new unique reagents.

Experimental model and subject details

Participants

This study included 41 volunteers. Two participants were excluded from all analyses, as their performance did not significantly exceed the chance level computed based on the formula of Mueller-Putz et al. (2008). The remaining 39 participants (21 female) were between 20 and 50 years old (M = 25.62, SD = 6.37), right-handed, had normal or corrected to normal vision, no diagnosis of neurological or psychiatric disorders, and no history of medication intake. The Autism Quotient values were representative for a neuro-typical control group (M = 15.64, SD = 5.55) (Baron-Cohen et al., 2001).

Participants were recruited at the Ludwig-Maximilians-Universität München via flyers and circular emails. Before the start of the study, all participants provided written informed consent. All participants received 10 € per hour as monetary compensation for their time and effort. The ethics committee of the Ludwig-Maximilians-Universität München approved the experimental procedures, which are in accordance with the guidelines of the Declaration of Helsinki.

Method details

Experimental design

The experimental design was based on the study of Zillekens et al. (2019) and implemented with the Psychophysics Toolbox (Version 3.0.14; Brainard, 1997; Kleiner et al., 2007; Pelli, 1997) in Matlab R2016a (The MathWorks, Inc., Natick, Massachusetts, United States).

Participants sat in front of a computer monitor (refresh rate = 60 Hz, resolution of 1280 × 1024) on which black moving dots on a gray background were presented (Figure 1). On one side of the computer monitor, the moving dots constituted a well recognizable point-walker (agent A), who performed either a communicative or an individual action out of six possible actions (communicative condition: asking to squat down, asking to look at the ceiling, asking to sit down; individual condition: turning around, sneezing, drinking). On the other side of the screen, the second point-walker (agent B) responded to agent A’s action in a cloud of temporally and spatially scrambled moving dots in only 50% of the trials (signal trials: squatting down, looking at the ceiling or sitting down). In the other half of the trials, agent B was replaced by a cloud of randomly moving noise dots (noise trials: agent B absent). Thus, the actions of agent A defined trials as the communicative or individual condition, whereas the presence or absence of agent B determined whether it was a signal or noise trial (Figure 1A).

Each trial started with a blank screen for a jittered inter-trial-interval between 1.5–3 s and a fixation cross for 1 s indicating the subsequent position of agent A on the left or right side of the screen (Figure 1B). Then, the video showing agent A was presented simultaneously with the cloud of dots on the other side of the screen (with or without agent B). The total duration of moving dots (i.e., the video) on the screen ranged from 3600 ms to 4300 ms, depending on the specific action. While dots constituting agent B were present and instable (i.e. presented using a limited life-time technique) from the beginning of each trial, the exact onset of agent B’s goal-directed response action was defined by a goal/spatially-directed dot movement exceeding a velocity of 10 angles/s for a minimal duration of 100 ms (Ansuini et al., 2015). Depending on the specific movement, this meant that agent B’s action started 1267–1567 ms after agent A’s action. In all trials, participants were asked to first watch agent A passively without any action required. Subsequently, participants should observe agent B’s responding action which was either masked or completely replaced by noise dots. They were then required to indicate as fast as possible whether agent B was present (signal) or absent (noise) with a button press. Responses had to be given before the dots disappeared. The position of the response buttons for yes and no (V-or N-key) on a standard German (QWERTZ) keyboard was counterbalanced across participants.

The position of the two point-walkers on the left or right side of the monitor was also counterbalanced. The actions of the agents were chosen from the Communicative Interaction Database (Manera et al., 2010) in the same way as in Zillekens et al. (2019). Manera et al. (2010) showed that participants could reliably identify, which of the actions from the Database were meant to be communicative or individual ones. In the experimental design, there were 13 possible dot positions to define agent B’s body. However, only 6 were occupied by signal dots at a given time. After 200 ms, a dot disappeared and reappeared at another position. The timing of dot appearance was desynchronized. This limited life-time technique was used in order to avoid participants from depending on simultaneous transitions of dots constituting agent B’s body. For more details, please see Manera et al. (2011b) and Zillekens et al. (2019).

Procedure and EEG data acquisition

Before the EEG experiment participants provided their age and gender, and filled out the short form of the Edinburgh Handedness Inventory (Veale, 2014), as well as an online version of the Autism Quotient (Autism Research Center; Baron-Cohen et al. (2001). Additionally, participants were asked to confirm the absence of neurological or psychiatric diagnosis, brain injuries, medication intake, and pregnancy in a short telephone interview.

During the experiment, EEG was recorded from 61 scalp electrodes (Ag/AgCl ring electrodes; Easycap ®) according to the international 10-10 system. Electrooculogram (EOG) was recorded from the left and right outer canthi and below the left eye. The reference was placed on the tip of the nose and the ground electrode was placed at Fpz. Signal was acquired at a sampling rate of 1000 Hz using a 64-channel amplifier system (BrainAmp, Brain Products ®), and impedances were kept below 10 kΩ.

To ensure that participants were able to solve the task with an average performance of about 70% correct responses, the level of difficulty (i.e., the number of noise dots masking agent B) was adapted individually using a pretest of 108 trials and an established procedure; A cloud of dots potentially containing agent B was presented with varying difficulty (i.e., a cloud of 5, 20, or 40 interfering dots), while participants indicated whether agent B was present or absent in the cloud via button press. A cumulative Gaussian function was fitted to the performance of the participants in order to obtain the number of dots corresponding to an average accuracy of 70%. The minimum number of dots used was five, even when participants’ estimated number of dots was lower (Manera et al., 2011a, 2011b; Zillekens et al., 2019). The individually calculated number of noise dots was employed in the main part of the experiment to mask agent B.

The participants could practice the task of the main experiment during twelve example trials. For the main experiment, 288 trials per participant were recorded (i.e., 72 trials per communicative/individual condition x signal/noise trials). The experiment was divided into four 9-min blocks with breaks in between.

Quantification and statistical analysis

As we could not find any other study that investigated the Bayesian ghost in the brain, we could not base our sample size on an a priori power analysis. A post-hoc analyses using G∗power (Faul et al., 2007) indicated that all our statistical analyses achieved a power of above 90%.

Behavioral data analysis and statistics

Behavioral data analysis was performed in MATLAB R2016a and statistical analysis in IBM® SPSS® Statistics version 24.0.0.0 and JASP version 0.9.2 (JASP Team, 2019). Two participants of the original 41 had to be excluded from all analyses because their performance was below chance level (Mueller-Putz et al., 2008). The behavioral results were computed over the remaining 39 participants. As intended, participants achieved an average performance of 71.65% (SD = 8.66) at an average difficulty level of 13.05 (SD = 10.07) interfering dots.

The onset of agent B’s goal-directed response action was defined by adding 1500 ms to the video onset of the action ‘squatting down’, 1267 ms to the video onset of the action ‘looking at ceiling’, and 1567 ms to the video onset of the action ‘sitting down’ (Figure 1B) (Ansuini et al., 2015). We excluded trials in which a response was given before agent B’s onset plus 200 ms or after the agents A and B had already disappeared.

For statistical analysis, we computed a 2×2 repeated-measures ANOVAs for the factors ‘condition’ (i.e., communicative or individual) and ‘type of response’ (i.e., false alarm or correct rejection trials). The dependent variable was the number of false alarms and correct rejections in the communicative and individual conditions. Normal distribution was confirmed for all variables using the Kolmogorov Smirnoff test (p < 0 .05). The statistical tests were performed two-tailed. Results were assumed as statistically significant if the probability of the results being random (i.e., showing no differences between groups) was less than 5% (p < 0.05). The descriptive and statistical details indicated by the mean (M), standard deviation (SD), the number of participants (N = 39), the effect size partial Eta squared (partial η2) and the F-value with the degree of freedom (Fx,x) can be found in the result section.

EEG data analysis and statistics

For pre-processing of the EEG data, BrainVision Analyzer 2.0 (Brain Products ®) was used. In a visual inspection, large artifacts were excluded, and bad channels were interpolated. The data was filtered between 0.1 and 120 Hz (48 db/oct) with a notch filter of 50 Hz. The EOG channels were re-referenced to a bipolar reference (i.e., right – left horizontal EOG channels, bottom vertical EOG channel – FP1), whereas the EEG channels were re-referenced to a common average reference. An Independent Component Analysis for ocular correction was computed for the bipolar EOG channels to detect blinks (vertical eye movements) and saccades (horizontal eye movements). After components that clearly represented artifacts were removed, the data was visually inspected for remaining artifacts again, which were then excluded. The data was re-sampled to 1024 Hz for the sLORETA analysis.

The data was not averaged and analyzed over the whole time course of the video because agent B’s onset and participants’ response times differed between trials. Thus, we extracted three time segments from every trial with respect to these two events (Figure 1B):

1 s before agent B’s onset (i.e., observation of agent A)

1 s after agent B’s onset (i.e., observation of agent B or noise dots)

1 s before the response (button press) occurred

Only trials in which a response occurred 1 s after agent B had started to move and before the stimuli had disappeared were included. Thus, the segments are free of responses, which would elicit motor response activation in the brain, and at the same time, all had a valid response. Time segment (2) can overlap with time segment (3). We did not have a specific hypothesis about time segment (2) but included it to investigate whether the effects of time segment (1) and (3) are continuously or very specifically tied to agent B’s onset and the response.

Segments without artifacts were exported for the following contrasts:

false alarm (i.e., noise trials with an incorrect “yes” answer) and correct rejection (i.e., noise trials with a correct “no” answer) trials in the communicative condition

false alarm and correct rejection trials in the individual condition as a control

hit trials (i.e., signal trials with a correct “yes” answer) in the communicative condition as a control (to be compared with false alarm trials)

The contrasts in the communicative condition included 20 participants who had a minimum of 15 artifact-free false alarm,15 correct rejection and 15 hit trials. A minimum of 15 trials was shown to produce stable EEG patterns (Hanslmayr et al., 2009). In the individual condition, 17 participants had a minimum of 15 artifact-free trials for each type of response. As the experimental task was designed in a way that participants performed with an accuracy of about 70% (Zillekens et al., 2019), numerous participants had to be excluded because they did not have enough false alarm trials. As it was especially difficult to find participants who had enough false alarm trials in the individual condition, the 17 participants can differ between time segments and to the communicative condition.

The EEG power analyses were performed according to a data-driven Whole Brain Analysis using Matlab R2015b and sLORETA v20190617 (Pascual-Marqui, 2002, 2007).

EEG crossspectra were computed over all single trials, separately for each subject and condition, in the frequency range between 1–40 Hz and global field power was extracted. The data across the whole brain was then transformed from scalp-level data into voxel-based Standardized Low Resolution Electromagnetic Tomography (sLORETA) data (i.e., source space).

For statistical analysis, the data was averaged into the following frequency bands: alpha (8–12 Hz), lower beta (13–17 Hz), and upper beta (18–25 Hz) for each of the three time segments: (1) 1-s before and (2) 1-s after agent B’s onset and (3) 1-s before the response. A two-tailed t-statistic was computed for paired groups using the sLORETA built-in voxelwise randomization test (5000 permutations). This correction for multiple comparisons is based on statistical non-parametric mapping (SnPM) to obtain corrected critical thresholds and p values and included all voxels across the whole brain. Significance was defined by the t-value exceeding the threshold based on the critical t-value on the 5% alpha level and indicated by p > 0.05. The descriptive and statistical details including the number of participants (N = 20 for the communicative and N = 17 for the individual condition) and the effect size Cohen’s dz (dz) can be found in the results section and in the legend of Figure 2. Indicated numbers in the manuscript were rounded to two decimal places although calculations are based on more decimal places.

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
EEG Segmented Data Single Trial	This paper	https://doi.org/10.17605/OSF.IO/9NQBW
EEG SLORETA Analyzed Data	This paper	https://doi.org/10.17605/OSF.IO/9NQBW
EEG MATLAB Code	This paper	https://doi.org/10.17605/OSF.IO/9NQBW
Behavioral Data Single Trial	This paper	https://doi.org/10.17605/OSF.IO/9NQBW
Behavioral Data MATLAB Code	This paper	https://doi.org/10.17605/OSF.IO/9NQBW
Software and algorithms
Psychophysics Toolbox (Version 3.0.14)	Brainard (1997); Kleiner et al. (2007); Pelli (1997)	http://psychtoolbox.org/docs/DownloadPsychtoolbox
MATLAB R2016a und R2015b	The MathWorks, Inc., Natick, Massachusetts, United States	https://de.mathworks.com/products/matlab.html
IBM® SPSS® Statistics version 24.0.0.0	IBM® SPSS®	https://www.ibm.com/de-de/analytics/spss-statistics-software
JASP version 0.9.2	JASP Team, 2019	https://jasp-stats.org/
sLORETA v20190617	Pascual-Marqui (2007, 2002)	http://www.uzh.ch/keyinst/loreta
BrainVision Analyzer 2.0	Brain Products®	https://www.brainproducts.com/downloads.php?kid=9