The influence of emotional facial expressions on gaze-following in grouped and solitary pedestrians.

The mechanisms contributing to collective attention in humans remain unclear. Research indicates that pedestrians utilise the gaze direction of others nearby to acquire environmentally relevant information, but it is not known which, if any, additional social cues influence this transmission. Extending upon previous field studies, we investigated whether gaze cues paired with emotional facial expressions (neutral, happy, suspicious and fearsome) of an oncoming walking confederate modulate gaze-following by pedestrians moving in a natural corridor. We found that pedestrians walking alone were not sensitive to this manipulation, while individuals traveling together in groups did reliably alter their response in relation to emotional cues. In particular, members of a collective were more likely to follow gaze cues indicative of a potential threat (i.e., suspicious or fearful facial expression). This modulation of visual attention dependent on whether pedestrians are in social aggregates may be important to drive adaptive exploitation of social information, and particularly emotional stimuli within natural contexts.

Group-living organisms often benefit from taking into account not only personally acquired information about their environment, but also the behaviour of and perceived decisions made by others123456. In this way individuals can benefit from the experience of others7, increase their capacity to detect and respond to threats such as predators89, and improve their decision-accuracy in contexts such as foraging10. Whereas among many social organisms individuals respond to relatively unambiguous cues, such as a change in direction or speed of others7, among humans some of the cues employed when mediating behaviour in a social environment can be relatively subtle. For example, individual pedestrians in crowded environments adjust visual attention to copy the gaze direction of others (so-called gaze-following). Recent studies of this behaviour in natural crowded environments suggest that social responsiveness to the gaze-direction of others can improve the acquisition of environmentally relevant information1112.

In particular, pedestrians show increased gaze-following in environments in which confederates performing ‘suspicious activity' have been placed11. This suggests that those who initially witness suspicious/irregular behaviour may exhibit additional social cues, coupled with gaze direction, which influence the attention of others. In other words, pedestrians may also be sensitive to the facial expressions of fellow passersby, processing these and other cues prior to, or during, their own gaze response. Research in the laboratory has shown that emotional expressions can modulate gaze-following [for an exception, see13], but that this effect is influenced by perceived emotional characteristics or context14151617, as well as the goal of the participant during the experiment18. For example, Holmes et al. (2006) provides evidence for stronger gaze-following effects when viewing fearful or angry, compared with happy or neutral, emotional expressions, but with high-state anxious participants showing greater shifts of attention15.

It is not known, however, how emotional cues influence gaze-following in natural environments, nor how access to social cues from other pedestrians, influence visual attention. For example by walking and interacting together pedestrian groups may show an overall increase in social attention, resulting in heightened gaze-following to cues provided by passerby. Furthermore recent laboratory research has shown that participants spend more time looking at images with negative compared to positive valence when they believe others are jointly viewing the same stimuli19. Therefore walking in groups may also alter perception to available cues, such as those associated with emotional expression. While walking alone, however, pedestrians may be less sensitive to social cues and attend primarily to external features of the environment to detect threats or localized disturbances. Hence, social context could be an important mediator of emotional gaze-following within crowds.

Here we investigate whether, and if so how, the emotional expression of a focal individual influences the propensity for oncoming pedestrians to alter their gaze-following behaviour in a natural and interactive environment (i.e., bi-directional pedestrian corridor). In particular, we ask whether walking as part of a group influences the propensity for pedestrians to respond to different gaze cues. We used four conditions, which included expressions of neutrality (control), happiness, suspicion and fear (Fig. 1). Similar to other social vertebrates [e.g.20], we hypothesise that human use of social cues may depend on social assemblage and correspondingly that the collective context may influence sensitivity to subtle social cues provided by facial expression.

Figure 1

Examples of the four emotional expressions from the confederate: (a) control/neutral; (b) happiness; (c) suspicion; (d) fear.

Results

A total of 1034 pedestrians were filmed during 270 neutral, 245 happy, 279 suspicious and 240 fear interactions. This sample consisted of 705 individuals walking alone and 329 pedestrians traveling in groups ranging from two to six members. Since the representation of group sizes greater than two varied across conditions (suppl. material), we report the comparisons between the gaze-following responses for solitary pedestrians and members of all group sizes. However, the overall results are consistent when excluding groups larger than two members (suppl. material).

A Generalized Linear Model was used to test for main effects of and interactions between emotional condition and group membership, while subsequent pairwise tests were performed to assess comparisons within and across conditions. Since previous research has shown a rearward transfer of gaze-following in the absence of experimental manipulations12, we investigated potential differences in gaze-following for pedestrians that could have been cued by members of their own group. Notably, previous in-group looks towards the stimulus did not significantly increase the gaze-following response of other members across or between conditions (p > 0.05). However, since group members may not respond independently, we treat groups as collective units whereby if any member in the group copied the gaze direction of the confederate the observation was treated as a single gaze-following event (looked: yes/no). The results are consistent when treating all pedestrians within groups independently (suppl. material).

Overall, the proportion of gaze-following pedestrians did not vary across the four emotional conditions (χ2(3) = 5.072, p = 0.167). There was, however, a significant main effect of group membership (χ2(1) = 23.188, p < 0.001), indicating a greater response by groups to copy the gaze of our confederate. There was also a significant interaction between emotional condition and group membership (χ2(3) = 10.289, p = 0.016; Fig. 2), indicating that the collective response varied as a function of the emotional expression displayed by our confederate. In particular, pairwise tests revealed that groups were more likely to copy the gaze direction when the confederate was displaying cues of suspicion and fear, compared to when exhibiting either neutral (p = 0.025; p = 0.026) or happy facial expressions (p = 0.034; p = 0.034). Furthermore, groups showed a heightened response to cues of suspicion and fear compared to individual pedestrians (p < 0.001; p < 0.001). There was no difference, however, when comparing the response of groups versus solitary pedestrians for neutral and happy cues.

Figure 2

The probability of gaze-following was independent of emotional expression for pedestrians walking alone (white bars), while those traveling in groups reliably altered their response to these cues (gray bars), showing a greater response to suspicion and fear than control and happiness.

The dotted line represents the baseline rate of gazing at the stimulus without previous gaze cues12. *p < 0.05; ***p < 0.001.

Discussion

The emotional facial expressions of the approaching confederate did not produce a detectable change in the gaze-following behaviour of individual pedestrians walking alone, but strongly modulated this response in those traveling in groups. Specifically pedestrians walking together were more likely to copy the gaze of suspicious and fearful expressions. This effect is not due to the enhanced transmission of cues among group members, but rather seems to result from those traveling together exhibiting a different perception of social cues, and thus a greater response to those associated with potential disturbance or threat.

These findings suggest that the information used to detect potentially relevant environmental factors is influenced by social aggregation (i.e., group affiliation). For solitary pedestrians the overall probability of gaze-following was independent of emotional expression, indicating a limited focus on the additional social cues potentially afforded by emotional state, or the ability to decipher between them, and perhaps a greater emphasis on external features (i.e., identifying the threat or disturbance) when monitoring the environment while traveling alone. When in a collective, however, responses to expressions of suspicion and fear were elevated, which is consistent with previous laboratory experiments showing a modulation of visual attention depending on social context19. Therefore, walking in a group appears to induce heightened behavioural monitoring of emotional cues from neighboring passersby. In other words, members of groups seem to follow cues of facial expression in a more informed way. To corroborate these findings, future research in the laboratory could test whether jointly viewing emotional gaze-following stimuli with others modulates the pattern of visual attention.

Our results could have implications for effective identification of, and response to, actual disturbances in public settings. When primed to monitor for threats and dangers it makes intuitive sense to prioritise fearful or suspicious faces, and thus we may expect a stronger cueing effect for these emotions within crowded transit locations or other high-risk sites. Laboratory research is consistent with these predictions: participants with heightened sensitivity and fear are more strongly cued by the gaze direction of faces with fearful expressions141516, and those instructed to search for a threatening target are more likely to follow the eye gaze on fearful compared to happy faces18. For purposes of stimulus repeatability we used the same female confederate for all trials in this study, but we recognise limitations to this approach and suggest future research investigate whether these results generalise to other pedestrian interactions17.

This study suggests that group membership is important in the context of social attention within natural contexts. Further field research of this nature is needed, as social orienting can vary substantially between real-life and approximated interactions21222324, but insights gained from this study could be paired with both laboratory and modeling approaches to better predict pedestrian social dynamics and ultimately improve security initiatives in crowded public settings during disturbances or threats.

Methods

Experiments occurred during four days between November 2011 and March 2012, including a total of 1034 pedestrian interactions. Using the same location and setup as Gallup et al.12, a hidden camcorder was placed to one side of a bi-directional corridor near an entranceway to a public building on a university campus. Concealed behind a one-way mirror of a larger apparatus, the camera was positioned close to eye-level for most people in order to track directed looks from passersby.

A professional female actor who was unaffiliated with the university and did not live in the surrounding area was hired as a confederate. She was instructed to wait at the opposite end of the corridor from the exit, and to approach pedestrians entering the building. She was trained to adapt her speed so that she entered the recording zone at the same time as the targeted oncoming pedestrians (approaching them from the left), at which point she continued walking but directed her head and gaze towards our hidden camera for a period of two-seconds while holding one of four emotional expressions; neutral, happiness, suspicion or fear (Fig. 1). This trajectory positioned the actor at the furthest point away from the apparatus within the corridor, and thus she terminated her gaze cue to face the exit just before she entered the view of the camera. The confederate then exited the building and looped around to a separate entranceway approximately 30 meters away (the building has nine entrances) where she approached the corridor from the original starting position to repeat the sequence (total loop took 40-seconds). This design ensured future pedestrians would not see our confederate walking back and forth within the corridor beforehand, and that she would not interact with the same pedestrian twice. Emotional expressions were chosen using a random number generator, and repeated for at least five consecutive trials, for a total of 115 in each condition. The order and sequence of conditions was recorded for purposes of pairing with the independent review of the footage.

The university ethics board approved this research. Following institutional guidelines, written scripts were posted on either end of the corridor notifying pedestrians that there was an “experiment” taking place and that a camcorder was recording the scene. Passersby were not informed of the rationale of the research nor the location of the camera.

Coding and reliability

Two independent reviewers, both blind to the experimental condition, scored the time entering the recording zone or scene, walking direction (towards/away from stimulus), looking behaviour (yes/no), sex (male/female), and group status (walking alone/with others) from all passersby. Groups were identified as walking closely together, often at the same velocity and with clear social interaction (talking, gesticulation)2526. Both reviewers scored roughly 5% of the same videos, obtaining high inter-rater reliabilities for looks and group status (Chronbach's α = .883; 1.00). Intra-rater reliability was also high (one reviewer: α = .968; 1.00). Gaze-following was defined by pedestrians altering their visual attention to a congruent direction of the confederate's gaze cue (i.e., towards our hidden camera). The lack of decay in the gaze-following response between the interactions occurring within 3-seconds and 10-seconds after the cue (p > 0.05 across and between conditions), paired with the strong cueing effect of our confederate at this temporal delay (compared to baseline rates of looking at the stimulus, p < 0.001)12, provided support for analyses to include the gaze behaviour of all pedestrians within a 10-second window following the initial cue. Each trial therefore included the behavioural reaction of at least one oncoming pedestrian, but in most cases we obtained interactions from multiple passersby.

To assess whether the emotional expressions displayed by our confederate were reliably interpreted as the emotions we intended to represent, we had 67 independent observers rate the four images in Fig. 1. These participants were first asked to respond to the following questions for each emotional expression: “Is the person in picture X displaying an emotion? If so, what?” These open responses were highly consistent with the intended expressions. For picture A, 51/67 (77.6%) reported the actor as a displaying no emotion, a neutral emotion or boredom, while 7/67 (10.4%) reported the actor as displaying a serious or annoyed expression. For picture B, 66/67 (98.5%) identified the actor as displaying happiness/joy. For picture C, 46/67 (68.7%) reported an emotion of suspicion, confusion or concern/anxiety, while 15/67 (22.4%) indicated an expression of anger/frustration/disapproval. For picture D, 66/67 participants (98.5%) reported an expression of fear or surprise/shock. Furthermore, there was 100% agreement for all expressions when participants were asked to distinguish between the four preset emotional responses in a multiple-choice format (i.e., picture A was unanimously identified as “neutral”; picture B was unanimously identified as “happy”; picture C was unanimously identified as “suspicion”; and picture D was unanimously identified as “fear”).

Author Contributions

A.C.G., A.C., A.K., J.R.K. and I.D.C. designed research; A.C.G. and A.C. performed research; A.C.G. analysed data; A.C.G., A.C., A.K., J.R.K. and I.D.C. wrote the paper. All authors reviewed the manuscript.