Comparison of response to Chinese and Western videos of mental-health-related emotions in a representative Chinese sample.

BACKGROUND: Emotion plays an important role in mental health. Studying the relationship between emotion and mental health requires effective emotion-eliciting materials. Most standardized emotional stimuli, however, were based on Western contents and have not been validated in other cultures. The present study compared the emotional response to standard Western videos with videos of Chinese contents in a large representative Chinese sample. The effects of content source (film vs. real-life) and delivery medium (online vs. offline), as well as the effects of demographic factors were investigated. Participants' depression level was assessed to test the potential use of the videos in mental health research.
METHODS: Top-ranked videos of basic emotions commonly implicated in mental health (happiness, sadness, anger, and fear) were chosen from a widely used Western video database. Twelve corresponding Chinese videos (film or real-life) were selected, with three clips for each emotion. In addition, three Chinese videos of the emotion "awe" were included because of the growing research attention to its role in promoting mental health. A large representative sample were recruited (N = 348) either online or offline and each participant viewed and rated his/her emotional reaction to all videos.
RESULTS: All Chinese and Western videos effectively elicited target emotions. The intensity of emotional response was generally higher for Chinese videos than for Western videos. Film and real-life videos provided mixed results in terms of the intensity of elicited emotions. There was a small difference in the delivery medium in which one video watched online were rated more intense than being watched in the laboratory. Older adults were more emotional reactive than young people in general, but the latter showed more differentiated response to Chinese versus Western videos. People with higher education levels responded less to happy videos. Finally, emotional reactivity of anger and awe were negatively related to depression level, which was partially consistent with the emotional-context-insensitivity (ECI) hypothesis of depression.
CONCLUSIONS: The results suggest that both Western and Chinese videos could reliably elicit emotion in Chinese people, but videos with local contents were generally more effective. The set of videos can be a useful tool for studying emotion and mental health in the Chinese cultural context. ©2021 Wang et al.

Introduction

A growing body of research has shown that emotion and emotion regulation play an important role in mental health, both in buffering the risk of mental disorders (e.g., Aldao, Nolenhoeksema & Schweizer, 2010; Berking & Wupperman, 2012; Gross & Jazaieri, 2014; Keltner & Kring, 1998; Rottenberg & Johnson, 2007; Sloan et al., 2017), and in promoting mental health and wellbeing (e.g., Gross & Munoz, 1995; Nyklíček, Vingerhoets & Zeelenberg, 2011). The study of emotion/emotion regulation and mental health calls for reliable and effective emotion-eliciting materials that could be applied to a wide range of populations.

Mental-health-related emotions

Among the basic emotions (Ekman, 1999), happiness, sadness, anger, and fear, are considered most closely related to mental health (Gross & Munoz, 1995; Gross & Jazaieri, 2014). According to the Diagnostic and Statistical Manual 5 (American Psychiatric Association, 2013), the cardinal diagnosis of major depression disorder is persistent sadness and/or lack of pleasurable feelings (anhedonia), while anxiety disorders are characterized by frequent and instantaneous fear-related experiences. In addition, dysregulation of anger is implicated in a myriad of disorders, such as post-traumatic stress disorder (PTSD), borderline personality disorder, bipolar disorder, and etc.

On the other hand, positive psychology emphasizes the role of positive emotions in buffering stress and promoting wellbeing (Fredrickson, 2001). Awe, a complex and distinct emotion (e.g., Valdesolo & Graham, 2014), has been found salutary for physical and psychological health (e.g., Rudd, Vohs & Aaker, 2012; Zhang & Keltner, 2016). Other health-promoting positive emotions, such as love and hope, are complex emotions usually occur in certain social contexts. For example, love is socially oriented and typically expressed in the presence of a close other (Gilbert, 2015), and hope is felt in the midst of a personally-significant stressor to signal positive adjustment (e.g., in the context of coping with trauma; Hassija et al., 2012). Thus, they might be harder to induce using unrealistic materials. On the contrary, the emotion of awe can be reliably elicited without particular social cues and easy to simulate in laboratory settings (e.g.,  Van Cappellen & Saroglou, 2012; Yang et al., 2016). It’s worth noting that some researcher suggest that awe could be both negatively-valenced and positively-valenced, which may differentially impact mental health (Gordon et al., 2017).

Cross-cultural comparison of standardized emotion videos

Among all the emotion elicitation methods, videos are the most commonly used and proven most effective (Lench, Flores & Bench, 2011). Furthermore, videos are easily implemented in the laboratory for eliciting both positive and negative emotions with high ecological validity (Gross & Levenson, 1995; Schaefer et al., 2010).

There are several widely used standardized video databases that include the four mental-health-related basic emotions (e.g., Gabert-Quillen et al., 2015; Gross & Levenson, 1995; Jenkins & Andrewes, 2012; Schaefer et al., 2010). However, the videos in these databases are based exclusively on Western contents (e.g., Gross & Levenson, 1995; Schaefer et al., 2010). Only a few studies have investigated the effectiveness of Western emotional videos in other cultures. In two studies, film clips from Gross and Levenson’s (Gross & Levenson, 1995) database (American films) were shown to Germans (Hagemann et al., 1999) and Japanese (Sato, Noguchi & Yoshikawa, 2007), respectively; and both studies showed that the American videos could elicit target emotions in participants from a different culture. Unfortunately, neither study compared the results with local videos, thus it was uncertain whether the videos were as effective as those with local contents. In another study, participants from Saudi Arabia were shown culturally-matched versus culturally unmatched emotional stimuli and reported higher emotional intensity to the former, suggesting that local emotional stimuli were more effective (Almutairi, Alghowinem & Alwabil, 2015; Alghowinem et al., 2019). Another study investigated the effectiveness of Western versus Chinese videos in basic-emotion elicitation in Chinese people, and also concluded that culturally-matched videos were more effective (Ge et al., 2019). However, the Western videos in the experiment were those with relatively low intensity of target emotions, which might have biased the results. Given the limited number of studies and the limitations of the studies, more research investigating the effect of culturally-matched and culturally-unmatched emotional videos are needed to gather more solid evidence on the cross-cultural use of standardized emotional stimuli.

The present study

The present study compared Chinese and Western videos of target emotions related to mental health (i.e., happiness, sadness, anger, fear, awe) in a Chinese sample. In order to enhance the validity of the study, we chose to select top-ranked videos from the most widely used Western database–FilmStim (Schaefer et al., 2010) as comparison stimuli. Although prior attempts have been made to establish Chinese film databases, most of them were based on small samples (Deng, Yang & Zhou, 2017; Xu, Huang & Luo, 2010), or included only college students (Ge et al., 2019). The present study therefore established a new set of Chinese videos based on a large representative sample, as an attempt to enhance the generalizability of the results. To reduce the probability of random errors and further improve the representativeness of the videos, participants were required to view all the videos instead of only a portion as in all previous studies (e.g., Gross & Levenson, 1995; Ge et al., 2019).

Most existing Western and Chinese databases only consist of fictional or deliberately produced materials (e.g., a clip from a full-length movie; Ge et al., 2019; Schaefer et al., 2010). Recently, researchers have begun using real-life materials in the study of emotion, such as user-generated videos and recordings of meetings (Jiang, Xu & Xue, 2014; Petridis & Pantic, 2009; Soleymani et al., 2012). Emotional response to films might be influenced by an individual’s ability to imagine a fictional reality, thus different from the response to reality-based videos (Lench, Flores & Bench, 2011). However, the similarity or differences of emotional response to films versus reality-based videos remain uninvestigated (e.g., Silvia, 2005). In the present study, we compared the response to two forms of emotional videos to address this gap in the literature.

The present study used both online and offline methods as an attempt to reach participants from a wider range of demographic background. The effects of the two delivery methods were compared. In addition, by recruiting a large representative sample, we were able to explore the impact of demographic factors on emotional response, and provide more nuanced information about the applicability of the videos to a wide range of populations.

Finally, we examined the association between participants’ emotional reactivity and their depression levels to test the potential utility of the video set in mental health research. Based on the emotional-context-insensitivity (ECI) hypothesis of depression (i.e., individuals with depression have lower emotional reactivity; Bylsma, Morris & Rottenberg, 2008), we predicted that people with higher depression levels would be less responsive to both positive and negative emotional videos (e.g., Rottenberg & Bylsma, 2014).

Materials & Methods

Clip selection

The first step was the selection of a large number of Chinese films and real-life clips that could elicit the five target emotions (happiness, sadness, anger, fear, and awe). Following the procedures in previous studies (Ge et al., 2019; Gross & Levenson, 1995; Schaefer et al., 2010), a subjective selection method was adopted. Ten experts (i.e., actors and directors in the film and television industry) were asked to fill in a brief questionnaire in which they recommended and rated three film clips for each emotional category. In addition to expert recommendations, five research assistants (two males and three females) were trained to select film clips and real-life videos from local movie websites and popular media sites, such as Tencent, Youku, iQiyi, Douyin, and Xiaomi Box. Third, films and real-life videos with target emotions were solicited from social media, such as WeChat groups, WeChat and Weibo public accounts. An initial set of 302 film and real-life clips was obtained using these methods. Next, the research team reviewed the set and selected videos according to the following criteria: (1) A complete storyline or event that elicited target emotion was captured within a time window from 1 to 3 min; (2) with 780p or above resolution and sufficient audio properties. The three most frequently cited videos were selected for each emotional category. The final materials include two film clips and one real-life video for each basic emotion, one film clip and two real-life videos for awe, and two neutral videos. For awe videos, we include two positively-valenced videos and one negatively-valence video to capture a wider range of awe experiences (see Table S1 for detailed descriptions of the videos). Because there were no standardized Western real-life videos, only videos of the four basic emotions from the most used Western film database, FlimStim (Schaefer et al., 2010), were selected as comparison materials. The four Western videos that have the highest rating of target emotions were chosen to enhance the validity of the comparison. Existing Western videos in eliciting awe were mostly tested in relatively small samples and used for specific research purposes, such as understanding the role of awe in religious rituals (Van Cappellen & Saroglou, 2012). Thus, we used only Chinese video clips for this emotion category. All Western videos were dubbed with simplified Chinese subtitles.

Measures

Emotional response to each video

We used the Differential Emotions Scale (Izard et al., 1974) to assess discrete emotions. The scale has shown high reliability and validity in distinguishing different emotions categories (Youngstrom & Green, 2003). Two emotional adjectives with identical or similar meaning in the Chinese language were used to capture each emotion: (1) joyful (yu kuai de), amused (hao xiao de) for happiness; (2) sad (nan guo de), heartbroken (bei shang de) for sadness; (3) angry (qi fen de), furious (fen nu de) for anger; (4) fearful (kong ju de), scared (hai pa de) for fear; (5) inspired (shou gu wu de), awestruck (jing wei de) for awe (see part one in the Results section for internal consistency of the word pairs). Participants rated the intensity of each emotional state on a 5-point scale (1 = “not at all”, 5 = “very much”)after they watched each video.

Previous viewing experience

After rating emotional states, participants answered a question about whether they had seen the video before (1 = “yes”, 2 = “no”).

Depression level

The Chinese version of the Symptom Checklist-90 depression subscale (SCL-90; Derogatis & Unger, 2010), translated by Wang (1984) and validated by Jin, Wu & Zhang (1986), was used to assess depression level (α = 0.92). The average score of the subscale was used in the final analysis.

Participants

The study was approved by the Ethics Committee of Shanghai Mental Health Center (IRB approval code: 2019-54R). Participants were recruited through social media groups, public billboards, and word of mouth. A total of 348 participants (61% female) with a mean age of 25.65 years (SD = 9.51; range: 16–74) participated in the study online (n = 320) or in the laboratory (n = 28). Each participant received 30 RMB as a compensation for their time.

Procedures

The online study was conducted via the largest online survey platform in China, Wenjuanxing (similar to Amazon Mechanical Turk). Before participating, all subjects were given instructions of the purpose and procedure of the study. They were informed that the study was voluntary and they gave consent by clicking the Start the experiment button or they could click the Know more button to read the complete consent form. Each participant watched all 21 videos that were randomly displayed. They were instructed to relax before each video and immerse themselves in each story or scenario in the videos, and honestly report their actual feelings. Participants rated emotional response immediately after watching each video. Participants completed the SCL-90 depression subscale after watching all the videos. At the end, each participant answered two randomly selected yes/no questions asking about simple and obvious facts in the videos as an attention check (e.g., “In one video you watched, a young man was going through airport security check and drinking up a large bottle of liquid. What did he drink?” 1 = milk; 2 = water.). The laboratory procedure was identical to that of the online study. Participants completed the experiment on a computer with a 17-inch color screen. The study took approximately one hour.

Three criteria were used to exclude participants and ensure data quality. First, participants who failed both attention check questions were excluded (n = 9). Second, participants were excluded if their scores of the two words describing the same target emotion were larger than 2 points apart (e.g., “1” for one word, but “4” for the other; n = 0). Third, participants were excluded if the difference between their scores of any word-pair describing the same emotion was larger than 3 SDs above/below the sample mean (n = 7 excluded). One participant met both criterion 1 and 3. In total, fifteen participants were excluded, leaving a final sample of 333 people. Demographics information of the final sample are presented in Table 1.

Table 1

Demographics of participants.

Variebles	Sample
	n = (274)	%	n = (333)	%
Gender
Male	95	34.7	124	37.2
Female	176	64.2	206	61.9
Missing	3	1.1	3	0.9
Age
16–17	6	2.2	6	1.8
18–25	211	77.0	226	67.9
26–45	47	17.2	74	22.2
46–74	9	3.3	23	6.9
Missing	1	0.4	4	1.2
Education
Middle School and Below	12	4.4	20	6.0
High School and Professional School	98	35.8	116	34.8
College	118	43.1	137	41.1
Graduate School and above	46	16.8	60	18.0
Medium
online	274	100.0	307	92.2
offline	0	0	26	7.8

Data analysis

The average score of each word-pair describing one emotion was used in the analysis. Descriptive statistics and t-tests were used to investigate whether the selected videos effectively elicit the target emotion. T-tests and Analyses of Variance (ANOVAs) were performed to compare emotional ratings between culturally-matched and culturally-unmatched videos, between film and real-life videos, between different delivery media and between multiple demographic groups. Bonferroni’s correction was used to all post-hoc multiple pairwise comparisons. Finally, correlation analyses on emotion reactivity and depression symptoms were used to test the ECI hypothesis of depression (Bylsma, Morris & Rottenberg, 2008). Emotional reactivity was operationalized as the differences between ratings of target emotion to the top-ranked videos and the corresponding emotional response to a neutral video (e.g., reactivity of anger = anger rating of the top-ranked anger video—anger rating of the neutral video). Due to a technical error at the initial stage of data collection, fifty-nine participants only viewed 15 videos. The change in degrees of freedom is denoted in the corresponding analyses. The significance level was set as 0.05. All analyses were completed using SPSS 20.0 (IBM Corp., Armonk, NY).

Results

Effectiveness of target emotion elicitation of each video

First, the reliability of each word-pair describing the same emotion was examined in all videos. The Cronbach’s alphas for word-pairs of sadness, fear, and anger were all high (range: α = 0.73–0.95). There was some discrepancy in the word pairs describing happiness and awe. The correlation for the word-pair of happiness for 12 videos was high (range: α = 0.70–0.89). Similarly, nine videos had high correlation for awe (range: α = 0.70–0.90). But for the other videos, the results were unsatisfactory (range: α = 0.05 –0.68). Especially, two awe videos showed very low correlation for the happy word-pair (α <  0.35). This was probably because we did not distinguish joy from amusement, while some researchers believe them to be two distinct emotional categories (Ge et al., 2019). In the majority of fear and anger videos, the correlation of the word-pair for awe was low. The result might be due to the fact that one word was positive, thus did not fit with the overall negative themes. Nevertheless, as the correlations of the happiness and awe ratings for videos intended to elicit happiness and awe were acceptable (range: α = 0.61–0.89), the average score of each word-pair was still used in the final analysis.

Paired-samples t-tests were performed to compare the highest with the second highest emotional scores for each video to investigate whether videos effectively elicited target emotion that was distinguishable from other emotions. Results showed that the intensity of the target emotion for each video was all significantly higher than the second-high emotions (ps <  .01; see Table 2), except for one awe video (Changing Sky; awe: M = 2.53, SD = 1.23; happiness M = 2.49, SD = 0.84, p = .482). It could be concluded that both culturally-matched and culturally-unmatched videos could effectively elicit intended target emotions.

Table 2

Descriptives of the ratings of target emotions for chinse and western videos.

		Target emotion
Video name		Happiness	Sadness	Anger	Fear	Awe
Lost on Journey	M SD	3.74a(1.14)	1.33 (0.67)	1.32 (0.65)	1.14 (0.44)	1.52 (0.88)
Crazy Alien	M SD	3.65a (1.06)	1.25 (0.63)	1.34 (0.74)	1.24 (0.64)	1.47 (0.87)
Chinese Baby	M SD	3.36a (1.08)	1.14 (0.50)	1.12 (0.44)	1.14 (0.52)	1.54 (0.85)
The Dinner Game (Western)	M SD	2.74a (1.17)	1.40 (0.67)	1.62 (0.87)	1.25 (0.62)	1.34 (0.67)
Tangshan Earthquake	M SD	1.12 (0.39)	4.24a (0.89)	2.13 (1.12)	2.62 (1.30)	2.01 (1.07)
Life Matters-A Mother with Cancer	M SD	1.54 (0.92)	4.19a (0.95)	1.32 (0.65)	1.81 (1.03)	3.21 (1.23)
Get Lost, Mr.Tumour	M SD	1.29 (0.65)	4.06a (0.96)	1.31 (0.67)	1.92 (1.07)	2.17 (1.20)
Schindler’s List (Western)	M SD	1.18 (0.54)	3.75a (1.14)	3.55 (1.26)	2.83 (1.28)	1.56 (0.91)
City of Life and Death	M SD	1.10 (0.38)	4.11 (1.05)	4.37a (0.96)	3.01 (1.38)	1.66 (0.97)
Indifferent Bystanders	M SD	1.14 (0.49)	3.48 (1.25)	3.83a (1.21)	2.89 (1.41)	1.33 (0.74)
Angels Wear White	M SD	1.20 (0.51)	3.13 (1.12)	3.29b (1.28)	1.89 (1.09)	1.24 (0.59)
American History X (Western)	M SD	1.16 (0.48)	3.11 (1.24)	4.13a (1.07)	2.93 (1.36)	1.33 (0.68)
The Great Hypnotist	M SD	1.22 (0.53)	1.78 (0.93)	1.71 (1.01)	3.84a (1.13)	1.31 (0.63)
Tornado	M SD	1.18 (0.47)	2.66 (1.34)	1.55 (0.95)	3.48a (1.26)	2.09 (0.90)
The Meg	M SD	1.44 (0.67)	1.80 (0.98)	1.65 (0.95)	3.05a (1.23)	1.61 (0.84)
Scream (Western)	M SD	1.21 (0.53)	2.41 (1.27)	2.49 (1.33)	3.69a (1.21)	1.35 (0.70)
Parade	M SD	2.51 (0.65)	1.13 (0.48)	1.14 (0.55)	1.13 (0.50)	4.27a (0.92)
Dong Cunrui	M SD	1.30 (0.56)	3.30 (1.20)	2.18 (1.23)	1.74 (1.01)	3.66a (1.17)
Changing Sky	M SD	2.49 (0.84)	1.20 (0.54)	1.16 (0.56)	1.18 (0.54)	2.53 (1.23)
Driver’s View	M SD	1.59 (0.72)	1.32 (0.67)	1.33 (0.70)	1.47 (0.77)	1.43 (0.75)
Pot Manufacturing	M SD	1.57 (0.74)	1.24 (0.62)	1.22 (0.55)	1.26 (0.63)	1.77 (1.15)

Notes.

Top-ranked and second-ranked emotion ratings for each video are in bold. Significant differences between top-ranked and second-ranked emotions are denoted as p < .001. and p < .01.

As we were interested in distinguishing positively-valenced versus negatively valenced awe videos, we compared other emotional responses among the three awe videos. The results showed that the happiness ratings of the videos of Parade and Changing Sky were significantly higher than three negative emotion ratings (i.e., fear, sadness, anger; ps <  .001), whereas for the video of Dong Cunrui, the negative emotion ratings were higher than that of happiness (p < .001). Thus, it confirmed previous research suggesting the emotion of awe could be either positive or negative.

Comparison between Chinese and western videos

Using culture as a within-subject variable, one-way ANOVA revealed significant main effects in each emotion category (Fs >  5.79, ps <  .002, ŋ2 s >  .02). The intensity ratings of target emotions were then compared between the Chinese videos and the corresponding Western video. Post hoc results showed that all Chinese videos were rated significantly more intense than Western videos for both happiness and sadness (ps <  .001). One Chinese angry video was rated more intense than the Western counterpart (p < .001; see Fig. 1). Although the top-ranked Chinese fear video (M = 3.84, SD = 1.13) was rated more intense than its Western counterpart (M = 3.69, SD = 1.21), the difference was not significant statistically (p = .833). We selected relatively low-intensity fear-eliciting materials due to ethical concerns that high-intensity fear response might be traumatic for psychologically vulnerable individuals. This might be especially problematic when the study was conducted online, where no clinical assistance was available in case of emergency. However, as for most emotional categories, Chinese videos generally elicited higher levels of target emotions, it was reasonable to conclude that videos with local contents were more effective emotional stimuli.

Figure 1

Ratings of target emotion of all Chinese and Western videos.

Apart from awe, the column on the far right in each emotion category represents the target emotion intensity of the corresponding Western video, and the other three represent the target emotion intensity of three Chinese videos. No Western videos were tested for awe, leaving three columns for the three selected Chinese awed videos. The order of the videos matches the list in Table 2.

Comparison between films and real-life videos

Using film-real-life category as a within-subject variable, one-way ANOVA revealed significant main effects in each emotion category (Fs >  6.66, ps <  .002, ŋ2 s >  .02). Post hoc results were mixed as to whether films or real-life videos elicited more intense target emotion. Five film videos elicited target emotion of significantly higher intensity than real-life videos (ps <  .01; see Table 3), whereas four of the real-life videos elicited more intense emotion than films (ps <  .001; see Table 3). No significant differences emerged between the real-life video and one film intended to elicit sadness (p = 1.000; see Table 3).

Table 3

Correlations between depression level, target emotional response and emotional reactivity of top-ranked Chinese videos and Western videos.

Target emotion	Film-real-life video pair	Mean difference (SE)	p	95% CI
Happiness	Lost on Journey -	0.40 (0.08)	.000	[0.21, 0.59]
	Chinese Baby
	Crazy Alien - Chinese Baby	0.20 (0.07)	.018	[0.03, 0.37]
Sadness	Tangshan Earthquake -	0.05 (0.06)	1.000	[−0.08, 0.19]
	Life Matters
	Get Lost, Mr.Tumor -	−0.15 (0.06)	.037	[−0.29, −0.01]
	Life Matters
Anger	City of Life and Death - Indifferent Bystanders	0.52 (0.07)	.000	[0.34, 0.69]
	Angels Wear White - Indifferent Bystanders	−0.61 (0.09)	.000	[−0.82, −0.41]
Fear	The Great Hypnotist - Tornado	0.36 (0.07)	.000	[0.19, 0.54]
	The Meg - Tornado	−0.40 (0.08)	.000	[−0.59, −0.20]
Awe	Dong Cunrui - Parade	−0.55 (0.07)	.000	[−0.71, −0.39]
	Dong Cunrui - Changing Sky	1.14 (0.09)	.000	[0.92, 1.36]

The effects of delivery medium and previous viewing experience

T-tests showed that, in general, whether the experiment was completed online or in the laboratory did not influence emotional response in general (ps >  .05). However, the Western fear video was rated as of higher intensity by online participants (online: n = 307, M = 3.73, SD = 1.21; offline: n = 20, M = 3.03, SD = 1.04, t (325) = 2.57, p = .011, d = 0.58, 95% CI [0.17–1.25]). Analyses of previous viewing experience showed that participants who had watched the video before reported higher intensity of the target emotional response for all happy videos, all awe videos, and one sad video (ps <  .05). It appeared that familiarity could boost both positive and negative emotional response.

Demographic differences

Female participants reported greater levels of fear and sadness than male participants for four Chinese videos (fear Video S1: female: M = 3.21, SD = 1.25; male: M = 2.79, SD = 1.16, t (328) = 3.06, p = .002, d = 0.34, 95% CI [0.15, 0.69]; fear Video S2: female: M = 4.08, SD = 1.04; male: M = 3.41, SD = 1.17, t (269) = 4.85, p < .001, d = 0.57, 95% CI [0.40, 0.94]; sad Video S1: female: M = 4.21, SD = 0.90; male: M = 3.76, SD = 1.01, t (269) = 3.73, p < .001, d = 0.45, 95% CI [0.21, 0.68]). Differences were also significant for one happy video (F (3, 239) = 5.02, p = .002, ŋ2 = .04) across education groups. Post-hoc comparisons revealed that subjects with higher educational levels reported less intensity of happiness (middle school and below: M = 4.23, SD = 0.98; high school/professional school: M = 3.31, SD = 1.14, p = .003, d = 0.81, 95% CI [0.23, 1.59]; college: M = 3.35, SD = 0.95, p = .004, d = 0.90, 95% CI [0.20, 1.55]; graduate school and above: M = 3.19, SD = 1.16, p = .001, d = 0.90, 95% CI [0.31, 1.76]). We also explored whether age showed differential effects for Chinese versus Western videos. Correlation analyses revealed that older participants experienced more sadness, anger and awe (sadness: r (326) = 0.13, p = .017; anger: r (271) = 0.16, p = .007; awe: r (271) = 0.16, p = .009). Furthermore, using age as a covariate, the results of variance analysis showed an interaction effect between culture and age on target emotion ratings (F = 2.48, p = .003, ŋ2 = .01). Estimated marginal means were analyzed at the high age level (i.e., one SD above the mean, age = 36.55) and at the low age level (i.e., one SD below the mean, age = 16.18). Pairwise comparisons between emotional ratings for different videos eliciting the same target emotion indicated that all the Chinese sad videos elicited significant higher intensity of sadness than the Western sad video only in the low age group (Mean differences >0.51 , SEs >  0.10, ps <  .001, 95% CIs [0.31, 0.85]). Younger participants responded more emotionally to the Western fear video than one Chinese fear video (i.e., Tornado; Mean difference = −0.51 , SE = 0.11, p < .001, 95% CI [−0.73, −0.29]). The other Chinese versus Western videos intended to elicit angry and fear revealed no difference between the two age groups. For happiness, both groups showed more sensitivity to all Chinese happy videos than the Western one (Mean differences >0.50 , SE s >  0.12, ps <  .001, 95% CIs [0.24, 1.46]). Results suggest that age had a significant impact on emotional responding to culturally-matched versus culturally-unmatched materials for most emotions. Compared with younger subjects, it seemed that older subjects were less sensitive to culture difference.

Emotional reactivity and depression

The correlation analyses of depression level and emotion reactivity are presented in Table 4. As can be seen, participants with higher depression levels showed lower emotional response to anger and awe (positively-valenced; ps <  .01; see Table 4). There was no relation between emotional reactivity scores and depression levels for happiness, sadness and fear (ps >  .05). As some researcher suggest that the impact of awe experience on mental health might be dependent on its valence (Gordon et al., 2017), we examined the relation between depression and the reactivity to the other two awe videos. The results showed that higher depression levels were associated with lower reactivity of the two videos, one positively-valenced and one negatively-valenced (r (274) = −0.17, r (274) = −0.16, respectively, ps <  .01).We also analyzed whether the response to Western videos differentiated individuals with different depression levels. No significant results were found for any emotion category (ps >  .05; see Table 4). Thus, culturally-matched videos seemed to better distinguish individuals with lower depression from those who showed more symptoms.

Table 4

Correlations between depression level, target emotional response and emotional reactivity of top-ranked Chinese videos and Western videos.

Variable	1	2	3	4	5	6	7	8	9	10
Chinese videos
1. Depression	—
2. Happiness	.17**	—
3. Sadness	.05	.37**	—
4. Anger	.04	.30**	.35**	—
5. Fear	.15*	.21**	.39**	.32**	—
6. Awe	- .07	.35**	.43**	.31**	.40**	—
7. R. Happiness	.04	.83**	.25**	.29**	.15*	.31**	—
8. R. Sadness	−.11	.28**	.80**	.38**	.37**	.36**	.36**	—
9. R. Anger	-.18**	.21**	.27**	.83**	.26**	.34**	.37**	.54**	—
10. R. Fear	.01	.17**	.33**	.34**	.82**	.26**	.23**	.55**	.45**	—
11. R. Awe	-.17**	.26**	.34**	.40**	.28**	.78**	.45**	.55**	.56**	.44**
Western videos
Variable	1	2	3	4	5	—	7	8	9	10
1. Depression	—
2. Happiness	.13*	—
3. Sadness	.09	.16**	—
4. Anger	.10	.16**	.49**	—
5. Fear	.09	.09	.42**	.43**	—
7. R. Happiness	.04	.83**	.14*	.17**	.07	—
8. R. Sadness	- .06	.07	.88**	.46**	.40**	—	.20**	—
9. R. Anger	- .11	.07	.44**	.85**	.38**	—	.24**	.60**	—
10. R. Fear	- .02	.04	.40**	.44**	.84**	—	.11	.55**	.52**	—

Notes.

total score of the SCL-90 depression sub-scale

raw rating minus the corresponding emotion rating of the neutral video

No Western videos were tested for awe.

p < .05, two-tailed.

p < .01, two-tailed.

Discussion

The present study aimed to compare the effectiveness of Chinese videos with Western videos among a large representative Chinese sample and compiled a set of standardized videos that could elicit emotions relevant to mental health. Four basic emotions (i.e., happiness, sadness, anger, and fear), and one complex emotion, awe, were defined as target emotions. The results showed that all Chinese and Western videos effectively elicited intended target emotion, suggesting that most emotional videos could be effectively used across cultures (see Sato, Noguchi & Yoshikawa, 2007). However, our results also showed that culturally-matched videos tended to elicit stronger emotions across a variety of emotions, suggesting that videos with local contents might be better tools for the study of emotion in a certain culture (see Zhu, Ho & Bonanno, 2013).

It is noted that not all Chinese videos elicited stronger emotions than Western videos in the current sample. The video clips of anger and fear from FilmStim (Schaefer et al., 2010) elicited more intense emotion than some Chinese videos. This suggests that even though culturally-match emotional videos were generally more effective, emotional response to videos may also depend on factors beyond the cultural boundary, such as specific content of the story or scenario. Therefore, instead of being exclusively reliant on culturally-congruent materials, emotional research could benefit from a pretest of materials from both indigenous and foreign cultures to select the more reliable stimuli.

The present set of emotional videos was based on a relatively large and representative sample, with age ranging from 16 to 74 years old and education level ranging from elementary school to post-doctorate. This has largely enhanced the generalizability of the video set, as compared to the existing similar databases (e.g., Deng, Yang & Zhou, 2017; Xu, Huang & Luo, 2010; Ge et al., 2019). Analyses on the demographic variables suggest that women responded stronger for fear and sadness than men, which was consistent with previous studies showing that women tended to report higher negative affect regardless of cultural background (Ge et al., 2019; Gross & Levenson, 1995; Schaefer et al., 2010). The results also showed that people with higher education levels tended to response less to happiness. The emotion response for people from different age groups were also significant different. Specifically, response to sadness, anger, and awe were stronger for older adults. Furthermore, older adults were also less sensitive to culture differences. The results ran counter with some research which found older people to be less emotionally reactive (Tsai, Levenson & Carstensen, 2000) and more emotionally stable (Carstensen et al., 2011), especially for Asians (Soto, Levenson & Ebling, 2005). It should be noted that age and educational differences were not observed across all videos, which suggests that the impact of age and education on emotional response might be more complicated than previous thought. Our results highlight the importance of including people from different demographic background when building standardized experimental tools, and call for more research to investigate the nuanced relationships between demographic factors and emotional response.

The results of previous viewing experience were consistent with prior findings that experience of having seen the video before was associated with greater intensity of both positive and negative emotions. This could be due to the sensitization effect, such that knowledge of the complete story primed participants with expectations and elicited greater emotional response (Gross & Levenson, 1995). Difference between online versus offline was found in only one video: participants who viewed the videos online experienced more fear than in the laboratory for the Western fear film (Scream). As the effect was limited to one video, we were unable to make any sound conclusion based on the current data. More study is needed to elucidate the effect of delivery methods on emotional response.

Comparison of film versus real-life videos showed that the form of the video was not a main determinant for emotional intensity. Some videos were rated more intense than real-life videos, whereas the opposite results were found for other videos. Some researchers have suggested the cognitive process of responding emotionally to a film are more taxing than to a documentary or user-generated clip, thus films would elicit lower response (Lench, Flores & Bench, 2011).This hypothesis, however, was not supported by our results. It seemed that emotional response was dependent more on the specific content of the video rather than its form.

A notable strength of current work was to establish and validate videos eliciting awe. Awe is elicited by something vast and information-rich that is beyond the person’s existing cognitive frameworks and requires mental accommodation (Keltner & Haidt, 2003). A number of studies has shown the salutary role of awe on mental and social health (e.g., Chirico & Yaden, 2018; Rudd, Vohs & Aaker, 2012; Zhang & Keltner, 2016), but some researcher suggested that this effect may only be present for positively-valenced awe (Gordon et al., 2017). Our results showed that reactivity to awe was consistently associated with lower depression levels, regardless of its valence. The discrepancy was probably due to the difference in the measurement frame: we assessed a longer period of mental health (the average of past four weeks), and the previous study assessed momentary wellbeing (Gordon et al., 2017). Nevertheless, our results suggest the overarching benefits of awe on mental health, and that the present composition of awe-eliciting videos could be a valuable tool for the growing research investigating this complex emotion and its impact on mental health (e.g., Capaldi et al., 2015; Piedmont, 2009).

Finally, the results of the relationship between lower emotional reactivity and depression provided preliminary evidence for the ECI hypothesis in the Chinese population (Bylsma, Morris & Rottenberg, 2008; Jin, Steding & Webb, 2015), and showed the sensitivity of the video set in distinguishing different mental health status. Furthermore, as such effects were only present for Chinese videos, but not the Western ones, it again highlights the importance of being sensitive to cultural impact in the research of emotion and mental health.

The study has several limitations that warrant discussion. First, although the use of the online method enabled us to recruit a representative sample, and offered an opportunity to compare the effects of delivery means, there were inherent limitations of internet-based research. Probably the most prominent one was the issue of lack of experiment control (e.g., Nosek, Banaji & Greenwald, 2002). We have adopted measures to control the quality of the data (e.g., attention check, eliminate aberrant data), however, more study comparing online and offline results are needed to validate the use of internet for this type of research. Second, the present set of videos consisted a relatively small number of videos. This choice was motivated by taking advantage of a within-subjects design and shortening the time to complete the whole experiment as a way to control random errors. However, the small number of videos may not be able to meet all requirements of relevant studies. Third, we only used subjective selection method to select the Chinese videos, as well as self-reports to measure subjective emotional states. Future research could benefit from incorporating behavioral indicators and physiological measures to capture emotional response more accurately. Finally, the present video set was based on non-clinical samples. The utility of this video set in mental health research awaits further investigation.

Conclusions

We compared the effectiveness of emotion elicitation between culturally-matched and culturally-unmatched videos. Results showed that both culturally-matched and culturally-unmatched videos were reliable in eliciting target emotions, but videos with local contents were generally more effective. The present study compiled and validated a video set that could elicit emotions most relevant to mental health, including the complex emotion of awe. Data was collected from a large representative Chinese sample through different medium. This has largely enhanced the generalizability of the videos. Meaningful differences in demographic factors were observed, which provides useful information for further research. Taken together, by compiling a mental-health-related emotion video set that comprises both Chinese and Western videos, film and real-life videos, the present study expands the repertoire of standard stimuli for emotion research and could be a practical tool for studying emotion and mental health among Chinese people.

Click here for additional data file.

Click here for additional data file.