Symmetries in a standardized set of emotional facial expressions (JACFEE).

OBJECTIVE: To investigate possible asymmetry in a widely used set of standardized emotional facial expressions (JACFEE, Japanese and Caucasian Facial Expressions of Emotion).
METHOD: Chimeric pictures with either two left (LL) or two right (RR) halves of the emotional face were produced from the original set and presented to healthy subjects who rated their intensity.
RESULTS: With the exception of "happiness", for all other emotions (sadness, disgust, anger, fear and surprise) LL and RR composites were given comparable intensities, speaking for high symmetry in the faces.
CONCLUSIONS: The JACFEE facial emotion pictures are mostly symmetrical, possibly due to their standardisation procedure when producing the pictures.

Introduction

Photographs of facial expressions of emotion constitute important stimuli for different fields of research, ranging from basic emotion studies to applied clinical investigations [1], [2]. When presenting facial emotions to subjects it is of utmost importance to have profound knowledge of the material used, i.e. to know basic components of the stimuli that influence their perception and recognisability. One of these components is the issue of facial asymmetry. It is assumed that faces generally are asymmetrical leading to differing portraits when composing “new” faces of two identical halves (so-called chimeras) [3], [4], [5]. Furthermore it is stated based on reviews that the left hemiface displays emotions more intensely than the right hemiface [6], [7], [8]. Although this general statement might hold true on the level of meta-analyses, there is surprising heterogeneity, when looking at single studies. There is empirical evidence speaking for greater left [9], [10] as well as right [11] hemiface expressiveness, and there are also reports of no differences [12], [13]. It is noticeable that studies differ greatly in their methods and stimuli which were used. Production methods range from giving clear instructions to actors to produce an emotional expression to idiosyncratic photographs of laymen portraying what they might think is an emotion. Some actors “feel” the emotion, others only pose their facial muscles to portray the emotion. Quality of photographs (lighting, viewing angle, background) and physiognomy of actors also vary greatly. Subtle facial cues like wrinkles and make-up are believed to inflate asymmetries and hence influence emotion recognition [14]. Before using facial stimuli to assess emotion recognition, the issue of asymmetry should thus be considered for the material at hand.

The aim of this methodological study was to thoroughly investigate possible asymmetry in a well-known and widely used stimulus set of facial affect, the JACFEE series (Japanese and Caucasian Facial Expressions of Emotion, [15]). This highly standardized and reliable colour image set [16] shows actors portraying one of six emotions (happiness, anger, fear, sadness, disgust and surprise). Actors have been instructed to move their facial muscles according to the Facial Action Coding System (FACS) [17], except for happiness, where pictures were taken when the actors smiled spontaneously.

Methods

To test for facial asymmetries so-called composites consisting of either two left or two right hemifaces were produced. Original JACFEE pictures were vertically divided in two halves to separate the left and right hemiface. Then the original hemiface and its respective mirror image were put together to create composites consisting either of two left (LL) or of two right hemifaces (RR). A considerable effort was made to eliminate technical artefacts (tilt, lighting, varying intensities or image size) during the stimulus production. Four images from this set (2 male & 2 female) were selected for each emotion, resulting in a total number of 48 images (6 emotions × 4 actors × 2 composites). Figure 1 (Fig. 1) shows examples of original and composite faces for happiness, fear and disgust.

Figure 1

Examples of chimeric faces showing the emotion happiness (top), fear (middle) and disgust (bottom): (a) original image taken from the JACFEE set, (b) LL-composite, (c) RR-composite

One widely used approach to test for facial asymmetries is to show LL- and RR-composites and let judges rate the intensity of the expressions. Differences between mean judged intensities of LL- and RR-composites argue for the respective left or right hemiface asymmetry. We tested 158 healthy volunteers (mean age 23.0 years, SD: 6.5 years, 66% female) who saw the composite faces based on the JACFEE set and asked them to rate their intensity. Subjects were recruited from the student population of the University of Ulm and the community. They all gave written informed consent. Half of the subjects (n=79) saw chimeric faces expressing anger, sadness and surprise. The other half judged faces displaying disgust, fear and happiness. Each image could be rated on a 5-point likert scale from 20 (low intensity) to 100 (high intensity), suggesting percentage of intensity in 20% steps. Each stimulus was presented for exactly 2500 ms. The order of presentation was completely randomized (emotions & images).

Results

Table 1 (Tab. 1) shows mean intensity ratings divided by emotion and composite. Only for happiness, LL composites were judged to be more intense (p<.001). For the other five emotions, there were no significant differences between LL and RR composites. For the four actors showing happiness, all pairs showed that the LL composite was judged as more intense. Intensity ratings followed a normal distribution across all emotions and varied between 28% and 100% (mean 72.9; SD 14.6). Differences in intensity rating among emotions were significant (ANOVA: F=43.030, p<.001).

Table 1

Intensity ratings (in percent) for LL and RR composites across emotions (n=79) and group comparison using Wilcoxon-Tests (Z)

Discussion

Our results show clearly that for anger, disgust, sadness, fear and surprise, the JACFEE pictures are symmetrical in terms of intensity judged by observers. For happiness there is a significant but rather small difference in absolute numbers speaking for greater expressiveness of the left hemiface. Therefore the high level of standardization applied when producing the JACFEE set seems to lead to symmetrical stimuli. Asymmetry effects observed for other facial affect stimuli should thus be irrelevant when using the JACFEE set in emotion recognition studies. It is interesting that happiness, the only expression that was displayed spontaneously by actors (“felt happiness”), is the only emotion with (moderate) asymmetry effects. Looking at Figure 1 (Fig. 1), more pronounced eye-wrinkles (“crow’s feet”) in the LL-composite could be one cause for the perceived greater expressiveness. Activity in the muscles orbiting the eye is part of the so-called Duchenne smile, which is believed to be genuine and reflect felt happiness [18]. Additionally, there have been previous reports of greater expressiveness of smiling in general on the left hemiface [19]. This strengthens the assumption that it might have been the standardisation procedure (FACS instructions) that caused the symmetry in all other emotions. When people express their emotions spontaneously, idiosyncrasies and lack of voluntary facial control might cause asymmetries. Consequently, this could have implications for the ecological validity of stimuli.

Notes

Conflicts of interest

None declared.