The Influence of Emotional Visual Context on the Judgment of Face Trustworthiness.

BACKGROUND: Previous studies of face trustworthiness have often examined isolated face stimulus, ignoring the role of context.
PURPOSE: The current study used mouse-tracking technique and the seven-point Likert scale to examine the effect of emotional visual context on face trustworthiness judgment at the levels of the early evaluation process and final evaluation result.
METHODS: Experiment 1 used mouse-tracking technique to study the impact of different contexts on the judgment of face trustworthiness at the early evaluation process. Experiment 2 used the seven-point Likert scale to study the effect of different contexts on the judgment of face trustworthiness at the final evaluation result.
RESULTS: Experiment 1 found that when faces are embedded in threatening negative contexts, the mouse trajectories are more tortuous for trustworthy responses and straighter for untrustworthy responses than in neutral contexts. When faces are embedded in non-threatening negative contexts, the mouse trajectories are more tortuous for trustworthy responses but did not significantly differ for untrustworthy responses than in neutral contexts. When faces are embedded in positive contexts, the mouse trajectories are straighter for trustworthy responses and more tortuous for untrustworthy responses than in neutral contexts. Experiment 2 found that faces embedded in threatening and non-threatening negative contexts have lower scores and faces embedded in positive contexts have higher scores than in neutral contexts.
CONCLUSION: The results show that the emotional visual context significantly influences the judgment of face trustworthiness both at the levels of the early evaluation process and final evaluation result.

Introduction

People form impressions about one another in three ways: through second-hand information, direct interactions with individuals, and the appearance of individuals.1 Faces can provide rich information for impression formation.2–8 People can not only infer a global impression of others but also automatically infer others’ specific social trait impressions from their face.5,9 Studies have shown that the judgment of a specific trait can be made only after a brief presentation of the face.10–18 Inferencing specific social traits from facial appearance may be a rapid spontaneous and uncontrollable process.5,19

Studies have found that people have a consistent assessment of social traits based on faces, and the presentation of a face for only 50 milliseconds is sufficient for multiple perceivers to make highly relevant trait inferences.11,14,18 Even children aged 3 to 4 years old show a certain degree of consistency in the judgment of social traits based on faces, which is consistent with the findings in adults.20 These findings suggest that facial cues provide a reliable signal of the potential personality of others. Moreover, some studies have found that social trait judgments of others obtained from faces are significantly correlated with the measurements of these traits.21

Previous studies have shown that the assessment of traits based on faces is mainly dependent on two fundamental dimensions: trustworthiness and dominance. Principal component analysis of social traits shows that trustworthiness accounts for the largest proportion,22 and trustworthiness judgment based on the face approximated the evaluation of general face valence.23–25 A functional neuroimaging study found that the inference of trustworthiness trait is a spontaneous and automated process.25 The judgment of facial trustworthiness is very rapid,17 and even without awareness, individuals are sensitive to facial trustworthiness.17,26 Moreover, studies have found that trustworthiness judgments based on the face are highly consistent.18,27 People attach great importance to information on trustworthiness. Studies have shown that final impressions of strangers are more influenced by information on honesty and trustworthiness.28 A study using American students as sample found that the most desirable trait of an ideal person is trustworthiness,29 and similar results were found in German-sample studies.30,31 Evolutionary psychologists believe that the detection of trustworthiness is critical to human survival and development,32 and approaching positive stimuli and avoiding negative stimuli is a fundamental component of behavior.33–36 The judgment of individual trustworthiness can predict human proximity or evasive behavior —— trustworthy faces may herald opportunities for cooperation and belonging, while untrustworthy faces may indicate dangers to be avoided.21,32,37–42 Moreover, judgments on the trustworthiness of faces can affect many important social judgments,22,23,43 even though it may not make our social judgments more correct. For example, judgments of facial trustworthiness influence economic decision making: when a person’s face is judged to be trustworthy, he will receive more economic investment and return; on the contrary, when judged to be untrustworthy, he will receive less economic investment and return.44–46

In previous studies, researchers have usually presented isolated face stimuli to control the interference of additional variables. However, the faces that people encounter in daily life are often embedded in a rich background context that includes external scene. A large number of studies have found that contextual cues have a substantial impact on the focus stimulus, whether the stimulus is an object or a face. Studies have found that clues about the body or the scene around the face affect the perception of facial emotions,47–52 especially when the emotions expressed by the faces are ambiguous. In addition, studies have found that the context systematically affects the response and classification efficiency of facial race and has cross-cultural consistency.53,54 However, in the field of face perception, researches on contextual effect mainly focus on facial emotion and facial race, and only a few studies have explored the influence of context on facial traits. Recently, a study adopted mouse-tracking paradigm firstly tried to explore the impact of context on the judgment of face trustworthiness.55 The paradigm records the trajectory of the mouse movement when the participants conduct the experiment task and uses the deviation of the mouse trajectory over time as an indicator of the real-time change of the psychological process.56 The traditional view is that behavioral reaction can only passively reflect the results of cognitive process, and is the ultimate product of perception and cognition.57 However, recent studies have found that motion systems are not passive response to cognitive processes, but rather that continuous movement can reflect real-time changes in early cognitive states.58,59 The neural network model proposes that environmental cues sometimes have a strong effect that causes the final response to change, and sometimes environmental cues have a weak effect that only temporarily change early cognitive states.58 Therefore, the fact that the final response does not change does not mean that the early cognitive state is not affected by the context. Mouse-tracking paradigm is exactly a new way of observing the dynamics of psychological processes.

Although the mouse-tracking technique the study used gives us important enlightenment to research the context effect on face trustworthiness, the study can not provide enough evidence to conclude that context can impact the judgment of face trustworthiness. First, the context that the researchers set is too limited—the study only explored the influence of negative context (threatening scene and negative scene) and did not comprehensively explore the variable of context. From the perspective of “negative-positive”, the context includes negative information that threatens our security (threatening negative context), general negative information (non-threatening negative context), neutral information, and positive information. Therefore, only with a comprehensive examination of the context variable can we draw a conclusion with higher ecological validity. Second, the study used mouse-tracking technique to explore the influence of context on the judgment of facial trustworthiness at the level of earlier judgment process. However, a recent neural network model suggested that sometimes the influence of context cues weakly only briefly affect the evaluation process, but sometimes strongly lead to a shift of the final evaluation result.60 In addition, a study has shown that sometimes early cognitive states change, but ultimately behavior response does not necessarily change.57 Therefore, the judgment of the context on face trustworthiness may differ at the levels of earlier evaluation process and final evaluation result, and results-based measurement methods,54 such as the traditional seven-point Likert scale, should be used to further analyze the impact of context on the judgment of face trustworthiness from the perspective of final evaluation result. Finally, the face stimuli used in the study are faces with trustworthy or untrustworthy characters, so the study drew the conclusion that the context affects the judgment of face trustworthiness may be due to the conceptual consistency of the context and facial traits61 rather than the impact of the context on the trustworthiness of the face itself.

The current study used neutral faces without any traits as experimental materials to systematically study whether different emotional visual contexts (threatening negative context, non-threatening negative context and positive context) influence the judgment of facial trustworthiness at the levels of earlier evaluation process and final evaluation result. To verify the hypothesis of this study, at the level of earlier evaluation process, mouse-tracking technique which is sensitive to measuring earlier evaluation processes was adopted. At the level of final evaluation result, the seven-point Likert scale was adopted. Based on the results of previous studies on facial emotion,47,51 race,54 and trustworthiness,55 the hypothesis of this study is that in threatening negative and non-threatening negative contexts, faces tend to be judged as more untrustworthy than in a neutral context at the levels of the early evaluation process and final evaluation result, and threatening negative context has a greater impact than non-threatening negative context. Faces tend to be judged as more trustworthy in a positive context than in a neutral context at the levels of early evaluation process and final evaluation result.

Experiment 1

Experiment 1 aimed to explore whether different emotional visual contexts affect individuals’ judgment of the trustworthiness of other people’s faces at the level of early evaluation process. Using mouse-tracking technique, Experiments 1a, 1b, and 1c, respectively, examined the differences between the mouse trajectory for faces embedded in a neutral context and the mouse trajectory in threatening negative, non-threatening negative, and positive contexts. The hypothesis is that when a face is embedded in a threatening negative or non-threatening negative context, the mouse trajectory for the trustworthy response is more tortuous than in a neutral context, and the mouse trajectory for the untrustworthy response is straighter than in a neutral context. On the contrary, when a face is embedded in a positive context, the mouse trajectory for the trustworthy response is straighter than in a neutral context, and the mouse trajectory for the untrustworthy response is more tortuous than in a neutral context.

Experiment 1a

In Experiment 1a, the Mouse Tracker software56 was used to investigate the difference between the mouse trajectory for trustworthiness judgment of a face embedded in a threatening negative context and the mouse trajectory in a neutral context.

Participants

Before the beginning of the experiment, G-Power 3.0 software was used to calculate the sample size. Based on analysis, sample size of 36 can provide 95% power to detect a medium effect (f = 0.25) for a 2×2 within-subject ANOVA when alpha = 0.05. Therefore, the experiment recruited 40 college students to participate in the experiment. One participant was excluded because the data exceeded three standard deviations, and one participant was removed because the experimental equipment failed. Finally, 38 participants aged 18 to 25 years (M = 19.45, SD = 1.55, 12 males) were included in the analysis. These participants were right-handed, with normal or corrected-to-normal vision and no mental or cognitive impairment. The study passed the review of the Ethics Committee of South China Normal University. All participants signed an informed consent form before the experiment, and they could obtain 10 yuan as compensation after completing the experiment.

Stimuli

The faces used in the study are computer-generated neutral faces without any traits that were rated by Todorov et al.62 Due to concern about the other-race effect, the study recruited 26 scorers aged 18 to 25 years (M = 20.38, SD = 1.92, 4 males) to re-review the trustworthiness of 25 faces using a scale ranging from 1 (very untrustworthy) to 7 (very trustworthy). Twelve faces were selected as experimental faces. The scores of these faces (M = 4.20, SD = 0.70) were not significantly different from the midpoint of the scale (t(25) = 1.467, p = 0.155 > 0.05, d = 0.29).

The scene stimuli used in the study were from the Internet, and the materials were evaluated in advance for the purpose of the study. The scorer who evaluated the faces also rated the scene stimuli pictures. First, the raters were required to rate the valence of the scene pictures using a Likert scale ranging from 1 (very negative) to 7 (very positive) to select negative, neutral and positive scene pictures. The results showed that the score of the negative scene pictures (M = 2.79, SD = 0.66) was significantly lower than the score of the neutral scene pictures (M = 4.04, SD = 0.41), t(25) = −8.279, p < 0.001, d = 1.64, and the score of the positive scene pictures (M = 6.34, SD = 0.50) was significantly higher than that of the neutral scene pictures, t(25) = 18.525, p < 0.001, d = 3.66. Furthermore, the raters were required to rate the threat level of those scene pictures that have been evaluated as negative valence pictures using a scale ranging from 1 (not at all) to 7 (extremely) and then experiment select threatening negative scene pictures and non-threatening negative scene pictures based on the two dimensions of “valence” and “threat”. The paired-samples t-tests showed that threatening negative scene pictures (M = 5.63, SD = 0.98) were significantly higher in threat level than non-threatening negative scene pictures (M = 3.35, SD = 1.16), t(25) = 9.435, p < 0.001, d = 1.85; but the negative level of threatening negative scene pictures (M = 2.80, SD = 0.88) was not significantly different from non-threatening negative scene pictures (M = 2.79, SD = 0.68), (t(25) = 0.059, p = 0.954 > 0.05, d = 0.01). Finally, 4 threatening negative scene pictures, 4 non-threatening negative scene pictures, 4 neutral scene pictures, and 4 positive scene pictures were selected, each with a size of 650 × 488 pixels. See Figure 1 for sample stimulus. The stimulus materials used in this experiment were threatening negative scene and neutral scene pictures.

Figure 1

This is the sample of material stimulus. (A) is the the sample of threatening negative context, (B) is the the the sample of non-threatening negative context, (C) is the the sample of positive context, and (D) is the the sample of neutral context.

Procedure

The participants were seated comfortably in a chair positioned approximately 60 cm from the monitor from which they received instructions and were presented with the stimulus. After clearing the experimental requirements, the participants conducted 6 practice trials to familiarize the experimental procedure. First, they used the mouse to click the “Start” button located at the center of the bottom of the screen. And the mouse required to place and click the “start” button before each stimulus is presented, which is to ensure that the initial position of the mouse track is the same in each trial, and facilitate the subsequent data processing. Then, a face-scene pair was presented in the center of the screen. When participant responds, the face-scene pair will disappear. The order of the pairs was random. The participants’ task was to judge the trustworthiness of the face and use the mouse to click the corresponding “trustworthy” or “untrustworthy” buttons on the upper left or upper right of the screen to complete the reaction. The locations of the “trustworthy” and “untrustworthy” buttons were balanced between participants. Participants were asked to make judgments and responses with their first impression as quickly as possible. To record the real-time decision-making process, the computer screen will display a message (‘Respond as soon as possible!’) when the initial movement time of the mouse exceeded 250 ms. When the reaction time exceed 2000 ms, the participant’s reaction is not recorded by the computer, and a message (‘Time out!’) will appear on the screen. During the reaction process, the Mouse Tracker software56 recorded the X and Y coordinates of the mouse movement. The computer’s sampling rate was 60 Hz. The formal experiment had a total of 48 trials. Each face located in the center of the scene picture was presented 4 times (2 times paired with the threatening negative scene and 2 times paired with the neutral scene). Each scene picture appeared 6 times. The matching of faces and scenes was balanced between the participants. See Figure 2 for the sample procedure.

Figure 2

This is the procedure of Experiment 1a.

Data Analysis and Results

The participants had different reaction time for different trials and different trials recorded different numbers of X-Y coordinate pairs. Therefore, in order to facilitate averaging and comparison across trials, trajectories were time-normalized into 101 time-steps, and then the left trajectory was flipped to the right along the X-axis. Trials with trajectory values 3 standard deviations outside of the grand mean (accounting for 9.48% of the total trials) and more than 2000 ms (accounting for 0.44% of the total trials) were excluded.

To detect whether contexts affect the judgment of facial trustworthiness, we calculated the maximum deviation value (MD): the maximum vertical deviation of the trajectory to the ideal line (ie, straight line from the “Start” button to the reaction button).56 The larger the MD, the more tortuous the trajectory, and the more the participant was attracted by the other option. On the contrary, the smaller the MD, the straighter the mouse trajectory. The study used a 2 (Scene Context: Threatening Negative Context, Neutral Context) × 2 (Reaction Type: Trustworthy, Untrustworthy) within-subject ANOVA. The results showed that the main effect of the scene context was not significant (F(1, 37) = 0.868, p = 0.357 > 0.05, ηp2 = 0.023), and the main effect of the reaction type was not significant (F(1, 37) = 1.172, p = 0.286 > 0.05, ηp2 = 0.031). However, the interaction between the scene context and the reaction type was significant (F(1, 37) = 16.658, p < 0.001, ηp2 = 0.310). A further simple effect test found that for the trustworthy response, the MD of the threatening negative scene condition (M = 0.29, SD = 0.14) was significantly larger than the neutral scene condition (M=0.22, SD=0.13), F(1,37) = 7.800, p = 0.008<0.05, ηp2 = 0.174. However, for the untrustworthy response, the MD of the threatening negative scene condition (M = 0.23, SD = 0.14) was significantly smaller than the neutral scene condition (M = 0.33, SD = 0.22), F (1, 37) = 10.221, p = 0.003 < 0.05, ηp2 = 0.216.

Next, we calculated the area under the trajectory (AUC), which is the area surrounded by the mouse trajectory and the ideal straight line.56 Although AUC is related to MD, it reflects the overall judgment process more than MD.56 The larger the AUC, the more tortuous the trajectory, and the more the participant was attracted by the other option. On the contrary, the smaller the AUC, the straighter the mouse trajectory. The study used a 2 (Scene Context: Threatening Negative Context, Neutral Context) × 2 (Reaction Type: Trustworthy, Untrustworthy) within-subject ANOVA. The results showed that the main effect of the scene context was not significant (F(1, 37) = 0.266, p = 0.609 > 0.05, ηp2 = 0.007), and the main effect of the reaction type was not significant (F(1,37) = 0.203, p = 0.655 > 0.05, ηp2 = 0.005). However, the interaction between the scene context and the reaction type was significant (F(1,37) = 21.461, p < 0.001, ηp2 = 0.367). A further simple effect test found that for the trustworthy response, the AUC of the threatening negative scene condition (M = 0.43, SD = 0.26) was significantly larger than the neutral scene condition (M = 0.28, SD = 0.16), F(1,37) = 14.937, p < 0.001, ηp2 =0.288. However, for the untrustworthy response, the AUC of the threatening negative scene condition (M = 0.28, SD = 0.14) was significantly smaller than the neutral scene condition (M = 0.47, SD = 0.37), F(1,37) = 10.999, p = 0.002 < 0.05, ηp2 = 0.229.

Experiment 1b

In Experiment 1b, the Mouse Tracker software56 was used to investigate the difference between the mouse trajectory for the judgment of a face embedded in a non-threatening negative context and the mouse trajectory in a neutral context.

Before the beginning of the experiment, G-Power 3.0 software was used to calculate the sample size. Based on analysis, sample size of 36 can provide 95% power to detect a medium effect (f = 0.25) for a 2×2 within-subject ANOVA when alpha = 0.05. Therefore, the experiment recruited 40 college students who had no experience with Experiment 1a to participate in Experiment 1b. One participant was excluded because the data exceeded three standard deviations. Finally, 39 participants aged 18 to 25 years (M = 19.97, SD = 1.71, 13 males) were included in the analysis. These participants were right-handed, with normal or corrected-to-normal vision and no mental or cognitive impairment. The study passed the review of the Ethics Committee of South China Normal University. All participants signed an informed consent form before the experiment. After completing the experiment, they could obtain 10 yuan as compensation.

The stimulus materials used in this experiment were non-threatening negative scene and neutral scene pictures. For details, see Experiment 1a.

Same as Experiment 1a.

First, the trajectories were time-normalized into 101 time-steps, and then the left trajectory was flipped to the right along the X-axis. Trials with trajectory values 3 standard deviations outside of the grand mean (accounting for 6.94% of the total trials) and more than 2000 ms (accounting for 1.07% of the total trials) were excluded.

The calculation of MD was the same as in Experiment 1a. To detect whether context affects the judgment of facial trustworthiness, the study used a 2 (Scene Context: Non-threatening Negative Context, Neutral Context) × 2 (Reaction Type: Trustworthy, Untrustworthy) within-subject ANOVA. The results showed that the main effect of the scene context was significant (F(1, 38) = 4.87, p = 0.033 < 0.05, ηp2 = 0.114), and the main effect of the reaction type was not significant (F(1, 38) = 0.418, p = 0.522 > 0.05, ηp2 = 0.01). However, the interaction between the scene context and the reaction type was significant (F(1, 38) = 4.527, p = 0.04 < 0.05, ηp2 = 0.106). A further simple effect test found that for the trustworthy response, the MD of the non-threatening negative scene condition (M = 0.16, SD = 0.16) was significantly larger than the neutral scene condition (M = 0.10, SD = 0.11), F (1, 38) = 8.535, p = 0.006 < 0.05, ηp2 = 0.183; however, for the untrustworthy response, the MD of the non-threatening negative scene condition (M = 0.12, SD = 0.08) was not significantly different from the neutral scene condition (M = 0.11, SD = 0.11), F(1, 38) = 0.009, p = 0.923 > 0.05, ηp2 = 0.000.

As in Experiment 1a, we next calculated the AUC. The study used a 2 (Scene Context: Non-threatening Negative Context, Neutral Context) × 2 (Reaction Type: Trustworthy, Untrustworthy) within-subject ANOVA. The results showed that the main effect of the scene context was not significant (F(1, 38) = 3.369, p = 0.074 > 0.05, ηp2 = 0.081), and the main effect of the reaction type was not significant (F(1, 38) = 0.521, p = 0.475 > 0.05, ηp2 = 0.014). However, the interaction between the scene context and the reaction type was significant (F(1, 38) = 4.938, p = 0.032 < 0.05, ηp2 = 0.115). A further simple effect test found that for the trustworthy response, the AUC of the non-threatening negative scene condition (M = 0.18, SD = 0.20) was significantly larger than the neutral scene condition (M = 0.12, SD = 0.13), F(1, 38) = 7.011, p = 0.012 < 0.05, ηp2 = 0.156; however, for the untrustworthy response, the AUC of the non-threatening negative scene condition (M = 0.13, SD = 0.08) was not significantly different from the neutral scene condition (M = 0.13, SD = 0.11), F(1, 38) = 0.192, p = 0.664 > 0.05, ηp2 = 0.005.

Experiment 1c

In Experiment 1c, the Mouse Tracker software56 was used to investigate the difference between the mouse trajectory for the judgment of a face embedded in a positive context and the mouse trajectory in a neutral context.

Before the beginning of the experiment, G-Power 3.0 software was used to calculate the sample size. Based on analysis, sample size of 36 can provide 95% power to detect a medium effect (f = 0.25) for a 2×2 within-subject ANOVA when alpha = 0.05. Therefore, the experiment recruited 40 college students who had no experience with Experiments 1a and 1b to participate in Experiment 1c. One participant was excluded because the data exceeded three standard deviations, and one participant was removed because the number of overtime trials exceeded 12.5%. Finally, 38 participants aged 18 to 24 years (M = 19.58, SD=1.45, 12 males) were included in the analysis. These participants were right-handed, with normal or corrected-to-normal vision and no mental or cognitive impairment. The study passed the review of the Ethics Committee of South China Normal University. All participants signed an informed consent form before the experiment, and they could obtain 10 yuan as compensation after completing the experiment.

The stimulus materials used in this experiment were positive scene and neutral scene pictures. For details, see Experiment 1a.

Same as Experiment 1a.

First, the trajectories were time-normalized into 101 time-steps, and then the left trajectory was flipped to the right along the X-axis. Trials with trajectory values 3 standard deviations outside of the grand mean (accounting for 7.30% of the total trials) and more than 2000 ms (accounting for 1.6% of the total trials) were excluded.

The calculation of MD was the same as Experiment 1a. To detect whether contexts affect the judgment of facial trustworthiness, the study used a 2 (Scene Context: Positive Context, Neutral Context) × 2 (Reaction Type: Trustworthy, Untrustworthy) within-subject ANOVA. The results showed that the main effect of the scene context was not significant (F(1, 38) = 0.888, p = 0.352 > 0.05, ηp2 = 0.023), and the main effect of the reaction type was not significant (F(1, 38) = 5.373, p = 0.026 < 0.05, ηp2 = 0.127). However, the interaction between the scene context and the reaction type was significant (F(1, 38) = 20.191, p < 0.001, ηp2 =0.353). A further simple effect test found that for the trustworthy response, the MD of the positive scene condition (M = 0.07, SD = 0.10) was significantly smaller than the neutral scene condition (M = 0.17, SD = 0.17), F(1, 38) = 11.457, p = 0.002 < 0.05, ηp2 = 0.236; however, for the untrustworthy response, the MD of the positive scene condition (M = 0.21, SD = 0.14) was significantly larger than the neutral scene condition (M = 0.15, SD = 0.12), F(1, 38) = 7.938, p = 0.008 < 0.05, ηp2 = 0.177.

As in Experiment 1a, we next calculated the AUC. The study used a 2 (Scene Context: Positive Context, Neutral Context) × 2 (Reaction Type: Trustworthy, Untrustworthy) within-subject ANOVA. The results showed that the main effect of the scene context was not significant (F(1, 38) = 0.305, p = 0.584 > 0.05, ηp2 = 0.008), and the main effect of the reaction type was significant (F(1, 38) = 5.749, p = 0.022 < 0.05, ηp2 = 0.134). However, the interaction between the scene context and the reaction type was significant (F(1, 38) = 24.812, p < 0.001, ηp2 = 0.401). A further simple effect test found that for the trustworthy response, the AUC of the positive scene condition (M = 0.09, SD = 0.09) was significantly smaller than that of the neutral scene condition (M = 0.24, SD = 0.26), F(1, 38) = 13.058, p = 0.001 < 0.05, ηp2 = 0.261; however, for the untrustworthy response, the AUC of the positive scene condition (M = 0.29, SD = 0.23) was significantly larger than that of the neutral scene condition (M = 0.17, SD = 0.15), F(1, 38) = 11.912, p = 0.001 < 0.05, ηp2 = 0.244.

Experiment 2

Experiment 1 used mouse-tracking technique to explore the influence of context on the judgment of face trustworthiness. However, as mentioned above, the mouse trajectory mostly reflects the earlier evaluation process, and the context effect on the final evaluation result requires further proof. In Experiment 1, we limited the presentation time of the stimulus in order to allow subjects to respond as soon as possible and collect the information of judgment process, rather than the mouse trajectories after the subjects made the decision. On the contrary, in Experiment 2, we did not set the time for stimulus presentation in order to allow subjects to fully think before making a decision, which can largely ensure the data reflect the information of judgment result. Using the seven-point Likert scale, Experiment 2 studied whether the context affects the judgment of face trustworthiness at the level of final evaluation result. Experiments 2a, 2b, and 2c, respectively, examined the differences between the scores of faces embedded in threatening negative, non-threatening negative, and positive contexts and the scores of faces embedded in neutral contexts. It is predicted that when a face is embedded in a threatening negative or non-threatening negative context, the score would be lower than when the face is embedded in a neutral context. On the contrary, when a face is embedded in a positive context, the score would be higher than when the face is embedded in a neutral context.

Experiment 2a

Using the seven-point Likert scale, the difference in the facial trustworthiness scores between faces embedded in a threatening negative context and a neutral context was investigated.

Before the beginning of the experiment, G-Power 3.0 software was used to calculate the sample size. Based on analysis, sample size of 29 can provide 95% power to detect a medium to large effect (d = 0.7) for a paired-samples t-test when alpha = 0.05. Therefore, the experiment recruited 33 college students aged 18 to 25 years (M = 19.45, SD = 1.42, 11 males) who had no experience with Experiment 1 to participate in Experiment 2a. All of them were right-handed, with normal or corrected-to-normal vision and no mental or cognitive impairment. The study passed the review of the Ethics Committee of South China Normal University. All participants signed an informed consent form before the experiment, and they could obtain 10 yuan as compensation after completing the experiment.

The stimulus materials used in this experiment were threatening negative scene and neutral scene pictures. For details, see Experiment 1a.

The participants were seated comfortably in a chair positioned approximately 60 cm from the monitor from which they received instructions and were presented with the stimulus. After clearing the experimental requirements, the participants conducted 5 practice trials to familiarize the experimental procedure. First, a 400 ms fixation point was shown in the center of the screen. Then, a face-scene pair was presented randomly in the center of the screen. The participants’ task was to rate the trustworthiness of the faces using a scale ranging from 1 (very untrustworthy) to 7 (very trustworthy). There was no time limit for the participants’ reactions. After the practice, the participants were asked to conduct the formal experiment. The formal experiment had a total of 48 trials, and each face was presented at the center of the scene picture 4 times (2 times paired with a threatening negative scene and 2 times paired with a neutral scene). Each scene picture appeared 6 times. The matching of faces and scenes was balanced between the subjects. See Figure 3 for the sample procedure.

Figure 3

This is the procedure of Experiment 2a.

The paired-samples t-test was performed to compare the scores of facial trustworthiness in the condition of threatening negative context and the neutral context. The results showed that the score of the judgment of facial trustworthiness in the threatening negative context condition (M = 2.59, SD = 0.81) was significantly lower than in the neutral context condition (M = 4.59, SD = 0.63), t(32) = −10.806, p < 0.001, d = 1.88.

Experiment 2b

Using the seven-point Likert scale, the difference in the facial trustworthiness scores between the non-threatening negative context and the neutral context in which the face was placed was investigated.

Before the beginning of the experiment, G-Power 3.0 software was used to calculate the sample size. Based on analysis, sample size of 29 can provide 95% power to detect a medium to large effect (d = 0.7) for a paired-samples t-test when alpha = 0.05. Therefore, the experiment recruited 32 college students aged 18 to 24 years (M = 19.78, SD = 1.72, 10 males) who had no experience with Experiment 1 and Experiment 2a to participate in the experiment. All of them were right-handed, with normal or corrected-to-normal vision and no mental or cognitive impairment. The study passed the review of the Ethics Committee of South China Normal University. All participants signed an informed consent form before the experiment, and they could obtain 10 yuan as compensation after completing the experiment.

The stimulus materials used in this experiment were non-threatening negative scene and neutral scene pictures. For details, see Experiment 1a.

Same as Experiment 2a.

The paired-samples t-test was performed to compare the scores of facial trustworthiness in the condition of non-threatening negative context and the neutral context. The results showed that the score of facial trustworthiness in the non-threatening negative context condition (M = 3.06, SD = 0.53) was significantly lower than in the neutral context condition (M = 4.51, SD = 0.69), t(31) = −8.695, p < 0.001, d = 1.54.

Experiment 2c

Using the seven-point Likert scale, the difference in the facial trustworthiness scores between the positive context and the neutral context in which the face was placed was investigated.

Before the beginning of the experiment, G-Power 3.0 software was used to calculate the sample size. Based on analysis, sample size of 29 can provide 95% power to detect a medium to large effect (d = 0.7) for a paired-samples t-test when alpha = 0.05. Therefore, the experiment recruited 32 college students aged 18 to 24 years (M = 20.75, SD = 1.52, 10 males) who had no experience with Experiment 1 and Experiment 2a, 2b to participate in the experiment. All of them were right-handed, with normal or corrected-to-normal vision and no mental or cognitive impairment. The study passed the review of the Ethics Committee of South China Normal University. All participants signed an informed consent form before the experiment, and they could obtain 10 yuan as compensation after completing the experiment.

The stimulus materials used in this experiment were positive scene and neutral scene pictures. For details, see Experiment 1a.

Same as Experiment 2a.

The paired-samples t-test was performed to compare the scores of facial trustworthiness in the condition of positive context and the neutral context. The results showed that the score for facial trustworthiness in the positive context condition (M = 4.55, SD = 0.54) was significantly higher than in the neutral context condition (M = 3.27, SD = 0.85), t(31) = 8.044, p < 0.001, d = 1.41.

Discussion

The current study systematically explored the impact of the emotional visual context on the judgment of facial trustworthiness at the levels of earlier evaluation process and final evaluation result through two experiments. Experiment 1 used mouse-tracking technique which is sensitive to measure the psychology process to study whether different emotional visual contexts affect individuals’ judgment of face trustworthiness at the level of earlier evaluation process. It is found that when a face is embedded in a threatening negative context, the mouse trajectory of the trustworthy response is more tortuous than in the neutral context, and the mouse trajectory of the untrustworthy response is straighter than in the neutral context. When the face is embedded in a non-threatening negative context, the mouse trajectory of the trustworthy response is more tortuous than in the neutral context, but the mouse trajectory of the untrustworthy response is not significantly different from the neutral context. In contrast, when the face is embedded in a positive context, the mouse trajectory of the trustworthy response is straighter than in the neutral context, and the mouse trajectory of the untrustworthy response is more tortuous than in the neutral context. These results indicate that different emotional visual contexts can affect individuals’ earlier evaluation process of facial trustworthiness. The result of Experiment 1 is consistent with the results of previous studies on the influence of context on the judgment of face emotion and face race.54,63 These researches indicate that context can impact the perception and judgment of face at the early cognitive processing. At the level of early evaluation process, the context can influence the judgment of face trustworthiness may be caused by the individual’s early attention selection. The individual incorporates the prominent information in the background into the process of judging the trustworthiness of face, so that the background can affect the subject’s judgment in a short time.

However, the recent neural network model suggested that context cues have a significant impact on face judgment, but sometimes the influence of context cues weakly only briefly affect the judgment process and sometimes strongly lead to a shift of the final judgment result.56 Therefore, Experiment 2 used the seven-point Likert scale which is a results-based measurement method to examine whether different emotional visual contexts affect individuals’ judgment of face trustworthiness at the level of final evaluation result. It is found that when the face is embedded in a threatening negative or non-threatening negative context, the score of facial trustworthiness is significantly lower than in the neutral context. However, when the face is embedded in a positive context, the score is significantly higher than in the neutral context. These results indicate that different emotional visual contexts can affect individuals’ final evaluation result of facial trustworthiness. The result of Experiment 2 is consistent with the results of previous studies on the influence of context on the judgment of face emotion.51 These researches indicate that context can impact the perception and judgment of face at the final evaluation result. At the level of final judgment result, the context can influence the judgment of face trustworthiness may be caused by the subject’s top-down response choices. There is no time limit to judgment for the subjects. Therefore, they can combine with their experience accumulated in daily life and use more advanced cognitive processes to determine the ultimate behavioral response.

In summary, the current study showed that different contexts have a significant influence on the judgment of face trustworthiness. Specifically, perceivers are more likely to judge a face embedded in a threatening negative context or a non-threatening negative context as untrustworthy, and the phenomenon is more obvious in threatening negative contexts than in non-threatening negative contexts. On the contrast, perceivers are more likely to judge a face as trustworthy when it is embedded in a positive context. It should be noted that the previous study55 did not find that the negative context could influence the judgment of face trustworthiness but the study found it, which may be due to three reasons. First, the facial materials the previous study used with strong trustworthy or untrustworthy traits, causing the context effect to be weak and difficult to detect. Second, due to the existence of threatening salience effects,42 the impact of the threatening negative context is stronger than that of the non-threatening negative context. Third, the trait of trustworthiness is related to threat information, and an fMRI study has found that the amygdala has been activated when participants judge the trait of trustworthiness, which is a brain region sensitive to threat information.25

The study perfects and expands the previous research. First of all, previous researches on face traits only presented the single face as a stimulus and ignored the role of background on face perception. The current study explored the influence of the context on the judgment of face trustworthiness, and extended the context effect from the domain of face race and face emotion to the domain of face traits in the field of face perception. Secondly, for the first time, the study systematically examined the influence of different contexts on the face trustworthiness judgment from the levels of the early judgment process and the final judgment result. The results of this study show that although the non-threatening negative context has a weaker influence on the face trustworthiness judgment than the threatening negative context, it still has a significant negative impact on the face trustworthiness judgment; more importantly, the previous researches have not explored the influence of the positive context on the face trustworthiness, but the results of the study reveal that the positive context has a significant positive impact on the judgment of face trustworthiness. Therefore, the results of the study provide more sufficient and reliable supporting evidence for the conclusion that the context can influence the judgment of the face trustworthiness. Further, the results of the study reveal the relationship between perception and social stereotype. The faces used in the study are neutral faces without any traits, but it is found that when they appear in the threatening negative and non-threatening negative contexts, people are more likely to judge them as untrustworthy and when they appear in the positive contexts, they are more likely to be judged as trustworthiness. It is means that stereotype about the context can affect the perception of faces. Leyens et al think that due to the limitation of individual cognition, people often combine the surrounding environmental information to help themselves make judgments in order to obtain more information and make judgments more accurate, and when individuals process the surrounding context, they often combine their subjective life experience.64 Therefore, individuals’ perception of face trustworthiness is affected by stereotypes about the context. Moreover, the study used neutral faces as the research materials, excluding the influence of the conceptual consistency of the context and facial traits on the judgment of facial trustworthiness. Finally, previous studies on the impact of context on trustworthiness used Western participants,55 but the current study used Chinese students as participants and found context effects, indicating that there is cross-cultural consistency in the impact of context on the judgment of face trustworthiness.

The results of this research also have a guiding role in people’s daily life. The results indicate that context has plasticity in the judgment of face trustworthiness, which can prompt people to pay attention to the important role of context in their daily interactions with others, so that they can evaluate others more objective and accurate. Further, we can also use the effect to gain some benefits and achieve our own goals. For example, the salespersons can use the context to create a feeling that they are very trustworthy, thereby promoting product sales. Moreover, a large number of studies have found that judgments about other people’s social traits based on faces can affect subsequent social results, and even affect serious events such as court decisions. Therefore, the accurate judgment of face traits not only plays an important role in human social interaction, but also has a great impact on promoting social fairness and justice.

The current study also has some shortcomings and deficiencies. Given the within-subject design, each participant was treated in two contexts. This may have created a forced contrast effect between the contexts, resulting in a difference in the findings. Moreover, in the study, there are 4 background images of each type, which is a small number. Therefore, we need carefully to generalize the result in other circumstances. Further, the study only studied the influence of context on the face trustworthiness from the levels of the early evaluation process and the final evaluation result, and did not explore the influence of the early evaluation process on the final evaluation result and the underlying mechanism. This is an important and valuable question, and we will research it in the future. Besides that, the use of computer-generated faces controlled some extra variables, but it damaged the ecological validity of the faces. Future research can be improved in these aspects. In addition, the current study only explored the impact of the context on the judgment of facial trustworthiness. Previous studies have shown that assessments based on the face are based on two basic dimensions: trustworthiness and dominance.22 Is the dominance of the face affected by the context? And the current study only explored the effects of visual context. What is the impact of the auditory context and the audiovisual context on the judgment of face traits? Although the impact of the context on the trustworthiness of faces is cross-culturally consistent, studies have found that Eastern and Western participants have differences in the way of processing65,66 and processing time.54 Do these differences also occur in the influence of context on the judgment of face trustworthiness? These questions can be explored and expanded in the future.