Spatial location in brief, free-viewing face encoding modulates contextual face recognition.

The effect of the spatial location of faces in the visual field during brief, free-viewing encoding in subsequent face recognition is not known. This study addressed this question by tagging three groups of faces with cheating, cooperating or neutral behaviours and presenting them for encoding in two visual hemifields (upper vs. lower or left vs. right). Participants then had to indicate if a centrally presented face had been seen before or not. Head and eye movements were free in all phases. Findings showed that the overall recognition of cooperators was significantly better than cheaters, and it was better for faces encoded in the upper hemifield than in the lower hemifield, both in terms of a higher d' and faster reaction time (RT). The d' for any given behaviour in the left and right hemifields was similar. The RT in the left hemifield did not vary with tagged behaviour, whereas the RT in the right hemifield was longer for cheaters than for cooperators. The results showed that memory biases in contextual face recognition were modulated by the spatial location of briefly encoded faces and are discussed in terms of scanning reading habits, top-left bias in lighting preference and peripersonal space.

Introduction

The ability to recognize faces is crucial because faces are highly salient and relevant visual stimuli in social interactions (Blais, Jack, Scheepers, Fiset, & Caldara, 2008; Bonner, Burton, & Bruce, 2003) and such ability is required across a broad array of exchanges. Information about the behaviour and appearance of individuals can be shared among people who have not interacted with each other directly (e.g. gossip) as face recognition involves the mental tagging of individuals with labels which are then used as guides in future encounters (Anderson, Siegel, Bliss-Moreau, & Barrett, 2011; Sommerfeld, Krambeck, Semmann, & Milinski, 2007).

Several studies have addressed the question as to whether we recognize faces of cooperators better than faces of cheaters or vice versa, since both types of responses are plausible from an evolutionary point of view: social animals rely on cooperation, but need to detect cheaters efficiently to avoid exploitation (Axelrod & Hamilton, 1981; Trivers, 1971). Such duality is reflected in the complexity of previous results. Some studies showed small biases towards faces of cooperators (Barclay, 2008; Brown & Moore, 2000; Felisberti & Pavey, 2010), whereas others argued that successful social exchanges require an enhanced recognition of cheaters for subsequent avoidance or punishment (Barclay & Lalumiere, 2006; Cosmides, 1989; Mehl & Buchner, 2008). In addition, a few studies either failed to find biases in the recognition of cheaters or cooperators (Buchner, Bell, Mehl, & Musch, 2009; Mehl & Buchner, 2008) or reported biases towards faces associated with threatening rather than cheating information (Bell & Buchner, 2012). The reasons underpinning the discrepancy in those results are not clear, but they may be related to the different experimental paradigms used in those studies.

Many social encounters and exchanges involve multiple and simultaneous interactions, whereby groups of faces are encoded rapidly (e.g. robberies). However, to date, remarkably few studies presented groups of tagged faces for encoding. Previous studies employed individual face encoding, since during free-viewing group encoding different parts of the visual field (i.e. fovea and periphery) are stimulated concomitantly, and perceptual anisotropies in the vertical and horizontal planes have been widely reported (Christman, 1993; Hillger & Koenig, 1991; Hines, Jordan-Brown, & Juzwin, 1987; Stone & Valentine, 2005; Thomas & Elias, 2011). For example, a left visual field (Luh, Rueckert, & Levy, 1991; Sergent, 1984; Yovel, Tambini, & Brandman, 2008) and an upper visual field (Carlson, Hogendoorn, Fonteijn, & Verstraten, 2011) advantage has been linked with face processing. Contextual face recognition biases in the vertical plane might be small and hard to tear apart, since such recognition involves widely distributed semantic, visual and even emotional processing.

In most of the previous recognition studies faces were not tagged with relevant social information and they were presented individually rather than in groups. Given the wide range of reported perceptual anisotropies, the question as to whether contextual face recognition after group encoding is affected by such anisotropies remains open.

This study investigated whether the spatial location of tagged faces in the visual field during brief encoding (10 s/group) affected subsequent face recognition. Due to the brief encoding time, it would be difficult for participants to split the time equally between the visual hemifields. Based on scanning studies during the reading of Western languages and biases in lighting preferences, it is predicted that the recognition of faces encoded in the upper hemifield would be better than faces encoded in the lower hemifield. On the other hand, if such biases do not modulate the encoding of groups of faces, memory biases in the horizontal plane should not be observed. Significant anisotropies related to the right or left visual hemifields were not expected during free-viewing encoding.

Methods

Participants

Participants (N = 80; 52 females, 28 males; M = 26 years old, SE = 1) were recruited via opportunity sampling at two university premises. Five participants were left-handers, and four participants did not specify their handedness. No financial compensation was given, but some participants received bonus course credits. Participants were informed that the experiment was about memory for faces, but no further experimental details were given. Written consent in accordance with the ethical guidelines of the British Psychological Society and as approved by the university ethics committee was given by participants prior to testing. Participants had normal or corrected-to-normal vision.

Materials

An equal number of photographs of male and female faces (N = 24) was selected from the XM2VTS database. The colour, frontal head-shot photographs (227 × 181 pixels) stood against a dark blue background and had neutral expressions. In a pilot study (N = 6), half of the faces were judged to belong to actors aged over 35 years old and the other half to actors aged between 18 and 30 years old. The faces were associated with one of three behaviours: cooperator, cheate, or neutral. Twelve faces had to be memorized in groups of four faces (Figure 1), which were presented in one of four positions in the visual field: upper left, upper right, lower left and lower right. Each face set contained two males and two females. The remaining 12 faces were used in the recognition test.

Figure 1.

Schematic illustration of the “old (yes)/new (no)” recognition test. Three sets of faces were memorized in conjunction with behaviours linked to descriptions of cheating, cooperation and neutral behaviours. A brief consolidation task (multiplications) followed the encoding phase. Afterwards, a memory test was presented whereby participants saw a face and had to answer whether they had seen that face before or not.

The faces in each of the three categories varied across some of the participants (e.g. the “cooperator” in one test could be the “cheater” in another one). Faces were allocated to one of the three behaviour groups using a simple randomization procedure (www.random.org).

The viewing angle was approximately 7 × 6 degrees at 60 cm from the centre of the monitor. E-Prime 2.0 (Psychology Software Tools, Pittsburgh, PA) was used to design and present the face recognition test.

Procedure

After reading and signing the consent forms, participants were told that the experiment would be testing their memory for faces. Participants were tested individually and according to the following core experimental protocol: (i) encoding of 12 faces with correspondent behaviours; (ii) distracter task during memory consolidation (10 multiplications); and (iii) “yes–no” face recognition test (48 trials: 24 faces—12 encoded and 12 new ones—in two cycles of trials).

In the encoding phase, a screen introduced the social scenario, followed by another screen with a behaviour. The information about people was obtained indirectly, i.e. participants did not suffer any direct financial loss. The template for the conditional rule was of the form: “If you take benefit P, then you must satisfy condition Q.” Note that the descriptors did not contain the words “cheater,” “cooperator” or “neutral.”

The introductory screen in the program read: “Before you continue, it is important to know that John is a successful businessman. Through his hard work, he has managed to build a very good life for himself and his family. He is also quite generous. He is willing to help out his long-time friends by offering them loans when needed. In the next three screens, you will meet John's friends. Press any key to continue.” The following screen read: “You will see three groups of people. The groups have different behaviours, which are specified in a screen before their photos. Press any key to continue.”

Before the participants saw the groups of faces, the tags for cooperators, cheater, or neutrals were introduced. The tags contained the following information:

“This group of friends borrowed £25,000 from John and paid it back with interest within a year;”

“This group of friends borrowed £25,000 from John and never paid it back;”

“This group of friends did not borrow money from John.”

The time allowed for reading the tags was not restricted; when participants finished reading them, they simply pressed a key or clicked the mouse to move to the next screen, which contained the four faces to be memorized in conjunction with the previous behaviour. The four faces in each of the groups (e.g. cheaters, cooperator, and neutrals) were displayed around a centred fixation point (two faces on top and two below it). The faces were spaced from each other by approximately 0.2 degree of visual angle. The duration of encoding for each group of four faces was restricted to 10 s and participants were free to move their eyes.

The order of encoding of the groups was randomized and counterbalanced as optimally as possible: 27 participants encoded cooperators first; 26 participants encoded cheaters first and 27 participants encoded neutrals first. The encoding order of the second group of participants was as follows: 25 encoded cooperators, 29 encoded cheaters and 26 encoded neutrals. Finally, the encoding order of the third group of participants was as follows: 27 encoded cooperators, 25 encoded cheaters and 28 encoded neutrals.

The encoding of faces and correspondent behaviour was followed by a memory consolidation phase, which consisted of a series of multiplications that lasted between 2 and 5 minutes. Answers were entered with the keyboard and feedback after each trial was provided.

The recognition test started with a black fixation cross presented on a blank screen for 1 s. It was followed by another screen with a face and the question “Have you seen this face before?” Half of the faces had been memorized in the encoding phase, and half of the faces were new faces, absent from the encoding phase. Participants answered by pressing a key (1 = yes; 2 = no). Feedback was provided after each trial. To continue to the next trial, participants just pressed any key or clicked the mouse.

Two cycles of 24 faces (12 tagged and memorized, 12 new) were presented in randomized order to each participant (N = 48 trials). The whole procedure lasted approximately 20 minutes.

Data analysis

Participants' performance was measured as response sensitivity (d′) and reaction time (RT) for correct responses (range: 100 ms and 10,000 ms); an adjustment was used when the hit rate was 100%, to avoid infinite d′ values (Green & Swets, 1974; Hautus, 1995; Macmillan & Creelman, 2004). The repeated-measures analysis of variance (ANOVA) had the descriptors tagged to faces (cooperator, cheater, neutral) and the visual hemifields (upper vs. lower or left vs. right) as the within-subject factors. The dependent variables were the d′ and the RT. The performance or quadrant was not analysed statistically due to the relatively small sample size or quadrant/behaviour. Greenhouse–Geisser adjustments to the degrees of freedom were performed when sphericity could not be assumed (Mauchly's sphericity test). All pair wise comparisons were carried out using Bonferroni adjustments. Partial eta-squared (ηp2) was used to refer to effect size.

Results

This experiment was designed to investigate whether the spatial location of faces during encoding affected the subsequent face recognition task. As mentioned previously, groups of four faces were tagged with one of three behaviours (cooperators, cheaters or neutrals). All faces in a group were presented simultaneously for encoding (10 s/group), but each face was presented in a different visual quadrant (upper left, upper right, lower left, or lower right).

The analysis of the d′ across all visual quadrants showed significant differences related to the behaviour tagged to the faces (F (2, 158) = 7.01, p = 0.001, ηp2 = 0.08), whereby the mean sensitivity for cheaters (d′ = 1.51, SE = 0.11) was lower than for cooperators (d′ = 1.79, SE = 0.10, p = 0.03) and neutrals (d′ = 1.86, SE = 0.17, p < 0.001).

The response latency also differed according to the behavioural tags. Six participants were removed from the analysis due to mean RT values outside the analysed range (100–10,000 ms). A significant interaction between behaviour and hemifield was observed (F (2, 146) = 5.13, p = 0.007, ηp2 = 0.07). Participants were faster at recognizing cooperators (M = 1,027 ms, SE = 29) than cheaters (M = 1,126 ms, SE = 35, p = 0.003), but not neutrals (M = 1,073 ms, SE = 38, p = 0.48). RT for cheaters and neutrals was statistically similar.

The d′ and RT for tagged faces encoded in the upper-left and lower-left quadrants were aggregated into a left (“Le”) hemifield, whereas data from the upper-right and lower-right quadrants were aggregated into a left (“Ri”) hemifield. Equally, data from the upper-right and upper-left quadrants were aggregated into an upper (“Up”) hemifield and the data encoded in the lower-right and lower-left quadrants were aggregated into a lower (“Lo”) hemifield.

Upper (Up) versus lower (Lo) hemifield

Response sensitivity (d′)

A significant interaction was observed between behavioural tag and the two encoding hemifields (F (2, 158) = 16.57, p < 0.001, ηp2 = 0.20). The d′ for cooperators encoded in the Up hemifield (d′ = 2.21, SE = 0.11) was higher than in the Lo hemifield (d′ = 1.42, SE = 0.14, p < 0.001). The inverse occurred with the d′ for cheaters, which was lower in the Up (d′ = 1.30, SE = 0.13) than in the Lo (d′ = 1.71, SE = 0.13, p < 0.001) hemifield. The d′ for neutral faces was statistically similar in both hemifields (d′Up = 1.79, SE = 0.11 and d′Lo = 1.99, SE = 0.12, p = 0.13) (Figure 2a).

Figure 2.

Response sensitivity (d′) (a, b), and reaction time (c, d) for the recognition of cheaters, cooperators and neutrals encoded in the four quadrants of the visual field. The quadrants were paired into two hemifields: upper vs. lower hemifield (a, c) and left vs. right hemifield (b, d). Line bars indicated the SE. Significant differences within the same category are shown by asterisks close to the bars, whereas differences between categories are shown by asterisks on a line connecting them. ∗p < 0.02, ∗∗p < 0.001.

Pairwise comparisons between faces encoded in the Up hemifield showed that participants were more sensitive to cooperators than cheaters (p < 0.001) and neutrals (p = 0.002). Such difference was not observed in the Lo hemifield, where d′ for cheaters was slightly higher than for cooperators, but not significantly (p = 0.06).

The effect of the encoding order on d′ was also investigated with a repeated measures ANOVA: 3 behavioural tags × 4 hemifields × 6 encoding orders (permutation of cooperators, cheaters and neutrals). No significant interaction between encoding order and tags was observed (F (9.32, 137.99) = 1.49, p = 0.15). Likewise, there was no significant interaction of encoding order with tags and visual hemifields (F (30, 444) = 0.94, p = 0.57).

RT

There was a significant interaction between behavioural tags and hemifield of encoding (F (2, 146) = 6.99, p = 0.001, ηp2 = 0.09), but only for neutral faces, which were recognized faster in the Lo (M = 977 ms, SE = 35) than in the Up hemifield (M = 1,168 ms, SE = 57, p = 0.001). There were no reliable differences between the Up and Lo hemifields for cheaters (MUp = 1,133 ms, SE = 41 and (MLo = 1,120 ms, SE = 45) or cooperators (MUp = 1,010 ms, SE = 45 and MLo = 1,047 ms, SE = 33) (Figure 2c).

Pairwise comparisons of RT between faces encoded in the Up hemifield showed that cooperators were recognized faster than cheaters (p = 0.006) and neutrals (p = 0.005). For faces encoded in the Lo hemifield, the RT for neutral faces was faster than for cheaters (p < 0.001) and cooperators (p = 0.03), but RT for cheaters and cooperators was similar.

Left (Le) versus right (Ri) hemifield

There were no significant interactions between the Le and Ri hemifields and behavioural tags (F < 1) (Figure 2b).

A significant interaction between tags and hemifield of encoding was recorded (F (2, 146) = 8.71, p < 0.001, ηp2 = 0.11). The RT in the Le hemifieds was similar for all behaviours: cheaters (MLe = 1,054 ms, SE = 41), cooperators (MLe = 1,057 ms, SE = 39) and neutrals (MLe = 1,068 ms, SE = 45). In contrast, the RT in the Ri hemifield varied with the tagged behaviour; the recognition of cooperators was faster than cheaters (p < 0.001) and neutrals (p = 0.02): cheaters (MRi = 1,203 ms, SE = 45), cooperators (MRi = 991 ms, SE = 38) and neutrals (MRi = 1,109 ms, SE = 49) (Figure 2d).

Discussion

This study investigated the effect of the spatial location of tagged faces during brief, free-viewing encoding in a subsequent face recognition task.

Participants were faster and more sensitive to faces tagged as cooperators or neutrals than faces tagged as cheaters, which is in line with previous studies showing biases towards cooperators when their proportion in the population was similar to the proportion of cheaters (Barclay, 2008; Felisberti, Cox, Wanli, & Dover, 2013; Felisberti & Pavey, 2010; Yarmey, 1993). In contrast, Chiappe and colleagues (2004) reported a better accuracy for cheaters than cooperators using a scenario similar to the one used in this study. However, like in this study they also reported that participants looked longer at cheaters than cooperators. Some differences in the two experimental designs make it difficult to explain some of the contradictory results. For example, in their study, faces were encoded individually, encoding time was not restricted, and the rules of the social contract were explicit, whereas in this study, no social contract rule was specified, encoding time was restricted to 10 s, faces were presented in groups and the tags did not contain explicit words related to moral aspects of the behaviours described.

Differences in d′ and RT linked to the hemifield of encoding were recorded for the first time. The recognition of cooperators was markedly better than of cheaters and neutrals when faces had been encoded in the upper hemifield, whereas no differences in d′ were observed for faces encoded in the lower hemifield. It is possible that participants' scanning patterns during reading (Heath, Rouhana, & Ghanem, 2005; Megreya & Havard, 2011; Sakhuja, Gupta, Singh, & Vaid, 1996) affected face encoding and subsequent recognition. In such case, faces in the upper hemifield would have had more encoding time than faces in the lower hemifield, since the group encoding was restricted to 10 s and it would have been difficult to allocate an equal share of time to each of the four faces in the group. In addition, the three groups of faces were interleaved with written behavioural descriptors, which could have acted as priming or masks. In pilot studies from this lab, contextual memory biases were attenuated or eliminated when faces were encoded individually rather than in groups and encoding time was raised to up to 9 s/face. Further studies are being carried out to confirm those preliminary results.

As mentioned previously, the order of face encoding was randomized and counterbalanced, but the overall advantage for cooperators over cheaters was observed independently of whether cooperators were the first, second or third group to be encoded. Furthermore, in a larger study (N = 153; under review), we showed a similar advantage for cooperators over cheaters even when the encoding order was fixed.

The findings in this study are also in line with the results showing a right-upper advantage in face matching tasks (Hagenbeek & Van Strien, 2002). They are also in line with the results from Raymond (2002), who reported an advantage for information encoded in the upper hemifield for branded images in relation to non-branded ones. Indeed, an advantage of the upper hemifield for visual encoding has been proposed by Previc (1990), whereby images viewed above the point of gaze would be preferentially associated with perceptual categorization and an “offline” analysis and storage of images, whereas images viewed below the point of gaze would be linked to the ongoing motor control of one's limbs (e.g. “online” processing).

In addition to the idea of anisotropies related to one's reading habits and peripersonal space, the advantage of visual processing in the upper hemifield may be also linked to prior visual knowledge about the environment (e.g. light comes from overhead). Such knowledge seems to be used to disambiguate scenes and leads to processing biases towards upper, left-lit stimuli (Gerardin, de Montalembert, & Mamassian, 2007; Mamassian & Goutcher, 2001).

The d′ for cooperators, cheaters and neutrals encoded in the left or the right hemifields was similar, since visual scanning in the horizontal plane is a more common task during reading than the scanning in the vertical plane. The lack of left versus right encoding differences in relation to the same behaviour may be related to the fact that both semantic and visual processing play a fundamental role in contextual face recognition, which activates different systems in the right and left hemispheres (Goel, Shuren, Sheesley, & Grafman, 2004). In addition, bilateral face processing has been shown to be more prominent in females (Bourne & Hole, 2006; Proverbio, Riva, Martin, & Zani, 2010), who were the majority of the participants in this study.

The spatial anisotropies in face recognition results did not explain why cheaters were not remembered as well as cooperators and neutrals, but they showed that contextual face recognition was flexible and finely tuned not only to changes in the social context but also to changes in the spatial location in which faces were encoded.

The findings indicate that the spatial location of faces during brief, free-viewing encoding modulated contextual face recognition significantly in the horizontal visual plane. The findings in this study are particularly relevant to cases where several faces are encoded in a brief period of time (e.g. eye witness cases). The results are likely to reflect participants' scanning patterns associated with reading, top-left bias in lighting preference and/or the perception of their peripersonal space.