Cockroaches are scarier than snakes and spiders: Validation of an affective standardized set of animal images (ASSAI).

Research on emotional processes has been closely related to the use of emotional stimuli, promoting the development of different standardized sets of images. However, some kinds of images that would be relevant in clinical psychology research are not available, especially for small animal phobias. The aim of the present study is to validate a set of animal images that includes images of cockroaches, which are not present in the current sets of images. Two hundred and forty images depicting five types of animals (cockroaches, spiders, snakes, cats, and butterflies) were collected from online public domains. Four hundred and twenty-four participants (72.9% women) took part in the study rating the images in two affective dimensions (i.e., valence and arousal). Cockroach pictures were rated as significantly more unpleasant than pictures of spiders, snakes, butterflies, and cats. Moreover, results revealed that women rated cockroach, spider, and snake pictures as more negative than men did. Also, women in comparison with men rated cockroach images as more arousing. The results highlight the importance of using images of cockroaches, due to their high negative valence, which even exceeds that of snakes and spiders, the unpleasant animals typically used in phobic research. This set of images can be useful in research on small-animal phobias.

Introduction

Research on behavior and emotions has increased exponentially in recent years. One reason could be the greater availability of standardized tools that allow researchers to evoke emotions in controlled laboratory settings (Davidson & Cacioppo, 1992). Several different emotion induction methods have been used, such as imagining specific situations, hypnosis, listening to affective music, telling a story, or giving an unexpected gift (see Gerdes et al., 2014; Gerrards-Hesse et al., 1994, for a review). However, despite this wide variety of procedures, the most commonly used method in emotion research is the presentation of emotionally salient stimuli. According to classical appraisal theories (Scherer, 1999), when a salient stimulus appears in the environment, the appraisal is automatically made, leading to reactions that can be measured as physiological responses, motor expressions, and feeling states (Scherer, 1987, 2004). The assessment of these reactions is an essential part of the study of the underlying mechanisms involved in emotional processes.

Specifically, the use of visual stimuli (i.e., pictures) has become the most prevalent method to induce emotions. This method offers the possibility of evoking emotional responses, in terms of physiological, behavioral, cognitive, and subjective changes (Lang et al., 1993). In addition, the use of images offers some advantages, such as the fact that pictures are not tied to language restrictions, and so they can be used to induce emotions in different countries and in people with language limitations. Another advantage is that pictures are static stimuli, which, in comparison with dynamic stimuli (e.g., movies or stories), facilitates the process of interpreting and measuring the emotional responses. Finally, static images allow researchers to control and manipulate some physical parameters (e.g., size, brightness, color, or display duration), which aids in selection and experimental implementation processes (Lang & Bradley, 2007).

In order to facilitate, improve, and increase psychological research with images, Lang et al. (2008) created the International Affective Picture System (IAPS), a large set of standardized images that are available worldwide. The IAPS includes more than 1000 color photographs grouped in different categories, such as objects, landscapes, and social and daily life scenes. It has been implemented in a wide variety of research topics, especially in cognitive (e.g., Neiss et al., 2009; Steinmetz et al., 2010) and clinical research fields (e. g., MacNamara & Hajcak, 2010; Sloan & Sandt, 2010). The widespread use of the IAPS in psychology research highlights the significance of having access to this type of stimuli database.

Nevertheless, the IAPS has several limitations such as its copyright restrictions or the fact that there are not enough different pictures of a specific type of stimulus, which is especially relevant in phobic research. In an attempt to overcome these issues, several researchers have worked on developing new picture sets, some focused on offering specialized emotional pictures, such as the Geneva Affective Picture Database (GAPED; Dan-Glauser & Scherer, 2011) or the Open Library of Affective Foods (OLAF; Miccoli et al., 2016), and others focused on offering free image databases with no copyright restrictions, such as the Open Affective Standardized Image Set (OASIS; Kurdi et al., 2017).

However, there is still a lack of specific sets of images that are relevant in many clinical and psychopathological research topics. Phobic research is a field where the use of animal pictures is extremely prevalent, and numerous pictures of a specific animal are needed in the experimental protocols. According to the prevalence rates, specific phobia is the most common disorder in the general population (Kessler et al., 2009). Specifically, this disorder affects 7.4% of the world population, whereas in Spain, specific phobia is found in 4.8% of the population (Wardenaar et al., 2017). Among all phobic stimuli included in the specific phobia subtypes, animals are the most feared (Becker et al., 2007). In addition, animal phobia presents a high life-time prevalence, ranging between 3.3 to 5.7% (LeBeau et al., 2010). Despite the large number of people affected, only 7.8% seek treatment (Mackenzie et al., 2012) because 25% reject traditional in vivo exposure treatment when they receive information about the treatment procedure (Garcia-Palacios et al., 2007). Thus, further research about specific phobias is necessary in order to expand the current knowledge about the underlying cognitive and psychophysiological mechanisms, as well as available treatments.

One of the problems that arise when researchers try to conduct studies about specific phobias is the small number of phobic stimuli available in the two largest databases of standardized images (i.e., IAPS and OASIS). Therefore, the GAPED could be a useful tool in this field because it includes numerous pictures of snakes and spiders, the most prevalent stimuli used in phobia research. However, there is still no specific standardized set of images of other different animals that could be relevant in phobic research, such as cockroaches.

Cockroach phobia is a highly prevalent disorder in several parts of the world, especially in cities where these animals are common, such as Mediterranean zones (Bueno-Marí et al., 2013). We believe that cockroach phobia could be relevant in the phobic research field due to the lack of knowledge about the cognitive and behavioral mechanisms involved in its etiology and maintenance, in comparison with other widely studied animal phobias (Öhman, 1993; Öhman & Mineka, 2001). In this regard, an increasing number of studies are focusing on cockroach phobia treatments with the aim of developing new, more accepted treatments as potential alternatives to applying the exposure technique supported by ICTs, as in virtual reality or augmented reality (e.g., Botella et al., 2005; Botella et al., 2008; Botella et al., 2010). From the neurocognitive research field, there is also recent empirical evidence from an fMRI study focused on the neural processes involved in cockroach phobia (Rivero et al., 2017).

Due to the growing interest in this field, this study aimed to develop and validate a set of animal images that includes pictures of cockroaches, which are not included (or insufficiently represented) in other image databases. Following previous research (i.e., Kurdi et al., 2017), we used the circumplex model of affect (Posner, Russell, & Peterson, 2005) to conceptualize the affective responses and their measurement. Thus, the images were assessed on two subjective scales: (i) valence, referring to the degree of negativity or positivity of the image; and (ii) arousal, referring to the level of emotional excitement the image evokes. Considering prior findings in this field, our first hypothesis was that cockroach scenes would be at least as unpleasant as snake and spider pictures, leading to more negative valence ratings and higher levels of arousal. Second, based on previous studies of affective ratings (Lang & Bradley, 2007), we expected to find gender differences, especially for unpleasant images, in that women would rate cockroach, snake, and spider scenes as more unpleasant and arousing, whereas cat images would be rated as more pleasant in comparison with men. Similar ratings by males and females were expected for butterfly pictures.

Method

The entire process of this research occurred over two separated phases due to the global pandemic situation derived from the COVID-19. In the first phase, a validation of 80 animal images was conducted, in which the experimental runs were performed in-person. Hereinafter, a second phase was conducted in order to enlarge the number of images including also an additional category of animals to improve the scope of our database. Unfortunately, this second phase occurred when the healthcare crisis was already affecting our country, and therefore the new experimental runs had to be slightly adapted in accordance with the sanitary restrictions and research protocols at our university. Nevertheless, an attempt was made to maintain the experimental conditions as similar as possible across phases to ensure the reliability of the overall results. Hereunder, methodology and results are described separately for both phases.

Participants

Phase 1

A total of 221 participants (147 women and 74 men) took part voluntarily in the first phase of the study after providing informed consent. All participants were between the ages of 18 and 59, with a mean age of 22.08 (SD = 6.92). They were undergraduate students recruited at the Universitat Jaume I (Spain). All the participants understood the study instructions perfectly.

Phase 2

A total of 203 participants (162 women and 42 men) took part voluntarily during the second phase of the study after signing the informed consent. In order to optimize the sample recruitment, which was more complicated due to the pandemic situation derived from the COVID-19, the overall sample was divided into four groups. Each group rated a block containing 40 different pictures in accordance with the methodology followed by similar studies (Kurdi et al., 2017). The number of participants was 48 in the first and second blocks, 55 in the third block and 52 in the fourth block. There were no differences between the number of participants nor the gender distribution across the blocks, χ2 (12) = .136, p = 1.000 and χ2 (3) = 4.114, p = .249, respectively. All participants were undergraduate students recruited at the Universitat Jaume I (Spain) with ages ranging from 19 to 55, with a mean age of 22.50 (SD = 4.833).

Sample size was calculated previously using G*Power 3.1.9.4 software. In order to obtain a medium effect size (0.25), considering an alpha level of 0.05, four repeated measures (one per each type of picture), and expecting a statistical power of 0.95, results indicated a minimum sample of 36 participants to obtain significant differences. Despite this, we decided to increase the number of participants per session according to the methodology implemented in previous validation studies (Kurdi et al., 2017; Lang et al., 2008).

Materials

Pictures were collected from different public domains using the Google Images web browser (https://images.google.com). Only images labeled for noncommercial reuse were selected in order to ensure that the set could be used without copyright limitations. Images of snakes, cats, and butterflies were selected as negative, positive, and neutral stimuli in order to compare them with the ratings of images of cockroaches. We chose these animals (snakes, cats, and butterflies) based on their ratings in previous databases (i.e., OASIS; Kurdi et al., 2017; IAPS; Lang et al., 2008). A total of 80 color images depicting the four types of animals were selected: cockroaches (N = 20), snakes (N = 20), cats (N = 20), and butterflies (N = 20). Subsequently, the images were resized to a panoramic width (between 1024 x 574 to 1280 x 720 pixels) to favor their visualization at the current standardized screen size ratio (i.e., 16:9). Furthermore, only those with a resolution close to or greater than 1024 x 574 pixels were selected, in order to guarantee a high resolution. Thus, all the images could be displayed on the full screen without losing resolution quality. Pictures were rated after they were resized.

A total of 160 new animal images were included in the second phase of the study. These pictures were selected following the same methodology regarding web browsing procedure, copyright restrictions and resolution size applied in the previous phase. The original four picture categories (i.e., cockroaches, snakes, butterflies, and cats) were maintained but a new category of spiders was added. In total, 60 images of spiders were selected, as well as 40 new images of cockroaches and 20 additional images for each category (snakes, cats, and butterflies).

The pictures were randomly divided into four blocks of 40 images each, controlling that the number of pictures per category remained constant across blocks: spiders (N = 15), cockroaches (N = 10), snakes (N = 5), butterflies (N = 5), and cats (N = 5). In addition, pictures from each block were randomly presented.

Fear of cockroaches and disgust measures

In order to control for a priori possible influences related to specific fear of cockroaches, as well as disgust sensitivity and propensity of the participants, two self-report instruments were administered in both phases:

Fear of Cockroaches Questionnaire (FCQ; translated and adapted by Nebot et al., 2012) is a cockroach version of the Fear of Spiders Questionnaire (FSQ; Szymanski & O'Donohue, 1995). This questionnaire contains 18 items referred to cockroaches designed to evaluate the severity of the phobia. Items range from 0 (“I strongly disagree”) to 7 (“I strongly agree”) and the scores can range from 0 to 126. The FSQ showed excellent psychometric properties. A randomized controlled trial focused on comparing different exposure treatment versions for small animal phobia (cockroaches and spiders) that used both versions (FCQ and FSQ) in Spanish population (Botella et al., 2016), showed that means score in phobic patients before and after treatment were 95.81 (SD = 13.96) and 44.47 (SD = 21.38), respectively.

Disgust Propensity and Sensitivity Scale - Revised (DPSS-R-12; van Overveld et al., 2006, adapted and validated in Spanish population by Sandín et al., 2008). This scale is composed of two subscales that measure propensity to disgust (six items) and sensitivity to disgust (six items). The items of both scales range between 1 (“never”) and 5 (“always”) and the total score in each subscale can range from 6 to 30. Regarding the psychometric properties reported in the Spanish adaptation (Sandín et al., 2008), the DPSS-R-12 showed good reliability (internal consistency) and validity (convergent, divergent, and predictive) results. Mean scores reported in normative values were 15.3 (SD = 3.5) for the propensity to disgust subscale and 12.2 (SD = 4.0) for the sensitivity to disgust subscale.

Procedure

The presentation of the images took place in a university classroom. The participants were distributed in groups ranging from 5 to 35 subjects. The images were displayed on a projector screen measuring 300 x 300 centimeters. Environmental conditions were controlled by regulating the artificial light for the optimal visualization of the screen. Participants were asked to fill out the valence and arousal measures for each image. Valence and arousal scales ranged from 1 to 7; the valence scale ranged from 1 = very negative to 7 = very positive; and the arousal scale ranged from 1 = very low to 7 = very high.

The 80 images were randomly presented using Microsoft Power-Point 2016. Each trial started with the display of one image on the screen for 5 s. Immediately after that, a black screen with the sentence “rate the image on the scale based on its valence and arousal” was displayed for 7 s. To avoid fatigue in the participants during the task, the presentation was divided into two blocks (40 images), with a rest time of 2 min. The whole presentation lasted 16 min, and the complete session (including instructions and image rating presentation) lasted approximately 25 min.

In order to maintain similar conditions between the two phases of the study, the environmental conditions for each run (university classroom), instructions and procedure were highly similar. In order to ensure the safety of participants and researchers and according to the sanitary protocols, participants used their smartphones instead of responding in a paper and pencil format. Rating questions for each image and questionnaires were filled out online using Qualtrics (www.qualtrics.com). Before each run, a QR code associated with the corresponding block was presented on the projector screen located in the classroom and participants captured these codes using their smartphone cameras. Once all participants had signed the informed consent and introduced their demographic data, images were displayed on the projector screen and participants rated each image using their smartphones. The duration of the task was shorter in comparison with phase 1 due that blocks contained 40 images (instead of 80). Participants were randomly assigned to each block.

Data analyses

Descriptive statistics (mean and standard deviation) were obtained for each picture both for the overall sample, as well as for women and men, separately. Moreover, in order to study the shape of the ratings’ distribution along the bidimensional space of affective valence and arousal, a scatterplot of the data was obtained. Additionally, Pearson’s correlations between arousal and valence ratings were calculated on the key points observed in the distribution (i.e., neutral, positive, and negative ratings) in order to assess the association of these variables.

To analyze the emotional valence and arousal ratings for the pictures corresponding to Phase 1, a 2 x 4 mixed repeated-measures ANOVA with gender (men, women) as between-subjects factor and image category (cockroaches, snakes, butterflies, cats) as within-subjects factor was performed both for valence and for arousal ratings. Regarding the Phase 2, a 2 x 5 mixed repeated-measures ANOVA with gender (men, women) as between-subjects factor and image category (spiders, cockroaches, snakes, butterflies, cats) as within-subjects factor was performed for both valence and arousal ratings. Bonferroni follow-up tests were used to further analyze significant differences. Furthermore, multivariate multiple regression analyses were conducted to explore the influence of gender, fear, and disgust scores on the valence and arousal ratings.

In order to explore the smallest effect that we could detect given our experimental design and sample size, sensitivity analysis (Cohen, 1988) was calculated separately for valence and arousal ratings. To detect the minimum effect size to which the tests conducted were sufficiently sensitive, we considered an alpha level of 0.05, a power of at least 0.80, and a sample size of 221 participants for subjective ratings.

Statistical analyses were performed using SPSS 24 and G*Power 3.1.9.4. Alpha level was set at 5%, using Greenhouse–Geisser corrections where appropriate.

Results

Univariate distributions

Missing data (0.28%) were handled using the mean imputation method. Tables 1 and 2 show mean and standard deviations for each image category included in Phases 1 and 2, separately.

Table 1

Means and standard deviations broken down by gender for valence and arousal ratings, for images included in Phase 1

	Valence			Arousal
	Men(N = 74)	Women (N = 147)	Total sample (N = 221)	Men(N = 74)	Women (N = 147)	Total sample (N = 221)
	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)
Cockroaches	2.44 (1.09) (2.236–2.655)	1.79 (0.80) (1.643–1.941)	2.01 (0.96) (1.99–2.248)	4.19 (1.41) (3.846–4.540)	4.92 (1.56) (4.675–5.168)	4.67 (1.55) (4.344–4.770)
Snakes	3.87 (1.35) (3.563–4.189)	3.08 (1.37) (2.860–3.304)	3.34 (1.41) (3.287–3.671)	4.59 (1.11) (4.317–4.869)	4.66 (1.24) (4.474–4.865)	4.64 (1.20) (4.462–4.800)
Butterflies	4.91 (0.89) (4.690–5.138)	4.92 (1.01) (4.762–5.079)	4.91 (0.97) (4.780–5.055)	4.05 (1.24) (3.779–4.326)	3.87 (1.67) (3.678–4.067)	3.93 (1.19) (3.795–4.130)
Cats	5.40 (0.94) (5.195–5.607)	5.72 (0.87) (5.577–5.870)	5.61 (0.91) (5.436–5.689)	4.56 (1.31) (4.268–4.867)	4.77 (1.30) (4.563–4.987)	4.70 (1.30) (4.488–4.855)

Table 2

Means and standard deviations broken down by gender for valence and arousal ratings, for images included in Phase 2

	Valence			Arousal
	Men(N = 42)	Women (N = 157–161)	Total sample (N = 199–203)	Men(N = 42)	Women (N = 157–161)	Total sample (N = 199–203)
	Mean (SD) CI (95%)	Mean (SD) CI (95%)	Mean (SD) CI (95%)	Mean (SD) CI (95%)	Mean (SD) CI (95%)	Mean (SD) CI (95%)
Cockroaches	2.21 (0.80) (1.96–2.46)	2.01 (1,09) (1.84 – 2.19)	2.05 (1.04) (1.91–2.20)	4.30 (1.24) (3.919–4.695)	4.84 (1.48) (4.609–5.078)	4.73 (1.45) (4.527–4.933)
Spiders	3.05 (0.87) (2.777–3.32)	2.26 (1.01) (2.475–2.796)	2.72 (1.00) (1.99–2.86)	4.11 (0.89) (3.837–4.391)	4.22 (1.34) (4.011–4.434)	4.20 (1.25) (4.024–4.376)
Snakes	3.57 (1.41) (3.137–4.015)	2.82 (1.36) (2.612–3.040)	2.98 (1.40) (2.788–3.179)	4.86 (0.99) (4.551–5.172)	4.76 (1.45) (4.539–4.996)	4.78 (1.36) (4.597–4.978)
Butterflies	5.55 (1.12) (5.205–5.908)	5.87 (1.30) (5.667–6.073)	5.80 (1.27) (5.629–5.982)	3.28 (1.51) (2.807–3.754)	3.38 (1.41) (3.158–3.601)	3.35 (1.43) (3.16–3.558)
Cats	5.57 (1.12) (5.205–5.908)	5.87 (1.30) (5.667–6.073)	5.80 (1.27) (5.629–5.982)	3.81 (1.54) (3.333–4.295)	4.11 (1.69) (3.851–4.379)	4.05 (1.66) (3.822–4.283)

As there were no differences regarding the number of ratings and gender distribution between the four blocks of images included in the second phase (χ2 (12) = .136, p = 1.000 and χ2 (3) = 4.114, p = .249, respectively), the overall results are presented. Regarding the distribution for valence ratings, the subjective estimates ranged between 1.75 and 6.18 for images included in Phase 1, and between 1.27 and 6.60 for those included in Phase 2, evidencing a wide use of the full range scale values regardless of the phase of this study. The mean value was 3.97 for Phase 1 and 3.29 for Phase 2, both of them close to the central value of the scale, whereas the median standard deviation was 1.32 and 1.49, respectively. Normality tests were carried out to analyze the distribution of the total of images’ ratings (combining both phases). Kolmogorov–Smirnov normality test revealed that valence ratings were not uniformly distributed, D = 0.15, p < 0.001. However, separated Kolmogorov–Smirnov tests calculated for each picture category showed a uniform distribution (D = 0.16, p = 0.16 for cockroaches; D = 0.07, p = 0.2 for spiders; D = 0.10, p = 0.2 for snakes; D = 0.98, p = 0.2 for butterflies; and D = 0.11, p = 0.2 for cats’ pictures).

As regards the distribution for arousal ratings, estimates ranged between 3.68 and 5.35 for images included in Phase 1 and between 2.88 and 5.71 for those included in Phase 2, revealing a restricted use of the complete rating scale in both phases. The mean value was 4.48 for Phase 1 and 4.27 for Phase 2, slightly higher than the central value of the scale. The median standard deviation was 1.63 for pictures validated in Phase 1 and 1.72 for those in Phase 2, being both values higher than the median standard deviation for valence ratings. Finally, overall Kolmogorov–Smirnov test showed that arousal ratings were not uniformly distributed, D = 0.058, p = 0.046. However, separated Kolmogorov–Smirnov tests calculated again for each picture category showed a uniform distribution (D = 0.09, p = 0.2 for cockroaches; D = 0.09, p = 0.2 for spiders; D = 0.11, p = 0.2 for snakes; D = 0.09, p = 0.2 for butterflies; and D = 0.08, p = 0.2 for cats’ pictures).

Fear of cockroaches and disgust analyses

Means and standard deviations obtained for FCQ (fear) and DPSS-R-12 (propensity and sensitivity to disgust) for the overall sample are presented in Table 3. Unpaired samples t tests showed significant differences between women and men in all measures: FCQ (t (361.94) = 13.15; p = .000) and both DPSS-12-R subscales; propensity to disgust subscale (t (312.9) = 5.98; p = .000), and sensitivity to disgust subscale (t (268) = 4.54; p = .000).

Table 3

Means and standard deviations broken down by gender for FCQ and DPSS-12-R scores for the overall sample

Instrument	TotalMean (SD)	MenMean (SD)	WomenMean (SD)
FCQ	44.93 (32.99)	20.80 (17.93)	54.10 (32.80)
DPSS-12-R
Propensity to disgust	13.72 (5.06)	11.77 (3.53)	14.50 (5.35)
Sensitivity to disgust	11.55 (4.19)	10.23 (3.37)	12.07 (4.37)

Analysis of valence

Figure 1 shows a summary of the main findings for valence and arousal means by picture category and gender. The results revealed a significant main effect of image category F (3, 217) = 491.47, p = .000, η = .87. Bonferroni post hoc tests revealed that cockroaches were rated as significatively more negative than snakes, butterflies, and cats (all p´s = .000). Snake ratings were more negative than those assigned to butterflies and cats (all p´s = .000). Finally, cats were rated as more positive than butterflies (p = .000).

Fig. 1

Representation of mean ratings for each image category in valence and arousal and broken down by gender. * = p < .05 with each other categories. # = p < .05 within the same category in gender (Phase 1)

Moreover, there was a significant gender x image category interaction, F (3, 217) = 12.19, p = .000, η = .14. Bonferroni analyses indicated that women rated both cockroaches and snakes more negatively than men did (all p´s = .000). In addition, analyses revealed that women rated cats more positively than men did (p = .01). Finally, no gender differences were found for butterfly scenes (p = .96).

Figure 2 shows a summary of the main results of valence and arousal means in each category for the pictures of the Phase 2, broken down by gender.

Fig. 2

Representation of mean ratings for each image category in valence and arousal and broken down by gender (only Phase 2 images). * = p < .05 with each other categories. a = p < .05 between spiders and cats. b = p < .05 between cockroaches and snakes. # = p < .05 within the same category in gender (Phase 2)

The results revealed a significant main effect of image category F (4, 193) = 115,93, p = .000, η = .71. Bonferroni post hoc tests revealed that cockroaches were rated as more unpleasant than spiders, snakes, butterflies, and cats (all p´s = .000). Spider images were rated as more negative than snake (p = .003), butterfly (p = .000) and cat images (p = .000). Finally, cats were rated as more pleasant than butterflies (p = .000).

An interaction effect was also found between gender x image category, F (4, 193) = 3.97, p = .004, η = .076. Post hoc analyses indicated that women rated spiders (p = .017) and snakes (p = .001) more negatively than men. No differences by gender were found for cockroaches (p = .31), cats (p = .09), and butterflies (p = .46).

Analysis of arousal

The results showed a significant main effect for image category, F (3, 217) = 30.86, p = .000, η = .29. Bonferroni analyses indicated that butterflies were rated as less arousing, compared to cockroaches, snakes, and cats (all p´s = .000). No differences were found between cockroach, snake, and cat images (all p´s = 1.000).

Furthermore, results showed a significant gender x image category interaction, F (3, 217) = 5.80, p = .001, η = .74. Bonferroni tests indicated that women rated the pictures depicting cockroaches as more arousing than men did (p = .001). No significant differences were found between men’s and women’s arousal ratings of snake (p = .65), cat (p = .26), or butterfly images (p = .29).

Post hoc sensitivity analysis showed that the smallest effect detectable for subjective ratings given this design and sample size was calculated to η2p = 0.99 for valence and arousal.

A significant main effect was found for image category, F (4, 193) = 28.31, p = .000, η = .037. Bonferroni analyses showed that cockroaches were more arousing than spiders (p = .000), cats (p = .018), and butterflies (p = .000). No differences were found between cockroaches and snakes (p = .35). However, snakes were rated as more arousing than spiders, cats and butterflies (all p´s = .000). Spiders and cats did not differ in their arousal ratings (p = 1.000). Finally, butterflies were rated as less arousing than cockroaches, spiders, snakes, and cats (all p´s = .000).

Regarding gender, a marginal significant interaction gender x image category was observed, F (4, 193) = 2.28, p = .062, η = .045. Bonferroni tests revealed a trend that showed that women rated cockroaches as more arousing in comparison with men (p = .036). No other differences were found between men and women (all p´s > .35).

General results

In order to offer a final and general overview of this research, we re-conducted the statistical analyses including data from the Phase 1 and Phase 2 together. Table 4 shows mean and standard deviations for each image category for the overall sample combining both phases.

Table 4

Means and standard deviations for valence and arousal ratings for the overall sample and broken down by gender. Values are calculated for all the images included in the dataset by combining the two phases of the study

	Valence			Arousal
	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)	Mean (SD)CI (95%)
	Men (N = 116)	Women (N = 304–308)	Total sample (N = 420–424)	Men (N = 116)	Women (N = 304-308)	Total sample (N = 420–424)
Cockroaches	2.36 (1.00) (2.175–2.545)	1.90 (0.97) (1.799–2.019)	2.03 (1.00) (1.938–2.130)	4.24 (1.34) (3.986–4.482)	4.88 (1.52) (4.709–5.053)	4.70 (1.50) (4.558–4.846)
Snakes	3.76 (1.373) (3.514–4.020)	2.94 (1.37) (2.795–3.104)	3.17 (1.41) (3.039–3.310)	4.69 (1.07) (4.492–4.888)	4.72 (1.35) (4.568–4.873)	4.71 (1.28) (4.589–4.835)
Butterflies	5.04 (0.98) (4.866–5.230)	5.18 (1.21) (5.047–5.319)	5.14 (1.15) (5.036–5.257)	3.77 (1.39) (3.516–4.03)	3.61 (1.32) (3.467–3.764)	3.65 (1.34) (3.53–3.787)
Cats	5.45 (1.01) (5.271–5.643)	5.80 (1.12) (5.674–5.926)	5.70 (1.10) (5.601–5.811)	4.29 (1.44) (4.029–4.559)	4.43 (1.55) (4.256–4.604)	4.39 (1.52) (4.247–4.538)
	Men (N = 42)	Women (N = 157)	Total sample (N = 199)	Men (N = 42)	Women (N = 157)	Total sample (N = 199)
Spiders	3.05 (0.87) (2.777–3.32)	2.63 (1.01) (2.475–2.796)	2.72 (1.00) (2.58–2.86)	4.11 (0.89) (3.837–4.391)	4.22 (1.34) (4.011–4.434)	4.20 (1.25) (4.024–4.376)

Relationship between valence and arousal dimensions

The distribution of the mean values for valence and arousal ratings for each picture divided by category, both overall and broken down by gender, is represented in Fig. 3. The bidimensional space is formed by valence – represented on the X-axis (ranging from 1 = very negative, to 7 = very positive) – and arousal – on the Y-axis (ranging from 1 = very low, to 7 = very high).

Fig. 3

Bidimensional distribution of valence and arousal ratings for each image by category, for the overall sample (above), and for men and women, separately (below)

As shown in Fig. 3, the distribution of the subjective ratings was similar to an “U-shape”, where the center of the distribution coincides with neutral affectivity with moderate-low intensity, and the positive and negative branches are moving toward the arousal extremes. This typical distribution shape is similar to those reported in previous studies with affective pictures, sounds, or words (Fernandez-Abascal et al., 2008; Kurdi et al., 2017; Moltó et al., 2013; Redondo et al., 2007). Furthermore, Pearson analyses confirmed this distribution, showing that the strongest correlations between arousal and valence ratings were observed for neutral (i.e., butterflies) (r = .124, p = .011), positive (i.e., cats) (r = .265, p = .000), and negative pictures (i.e., cockroaches) (r = – .360, p = .000). These strong correlations on the three key points of the affective scale support the fact that neutral images were rated as moderate-low activating, and as the affect became more pleasant or unpleasant, the images clearly tended to be rated as more arousing.

Analysis of valence and arousal

Ratings of valence and arousal for cockroaches, snakes, cats, and butterfly images were analyzed. Ratings (valence and arousal) for scenes depicting spiders were not included in the analyses because they were only collected in the Phase 2 of the study. Figure 4 presents a summary of the main results of valence and arousal means in each category for all pictures and broken down by gender.

Fig. 4

Representation of mean ratings for each image category in valence and arousal and broken down by gender (total images). Only images depicting cockroaches, snakes, butterflies, and cats were included in the analyses. * = p < .05 with each other categories. c = p < .05 between cats and snakes. # = p < .05 within the same category in gender

The 2 (men, women) x 4 (cockroaches, snakes, butterflies, cats) mixed repeated-measures ANOVA for valence showed a main effect for image category, F (3, 416) = 516.96, p = .000, η = .79. Post hoc analyses showed significant differences between all pairwise comparisons (all p´s =.000): 1) cockroaches were rated as more unpleasant than snakes, cats, and butterflies, 2) snakes were rated as more negative than cats and butterflies, and 3) cats were rated as more positive than butterflies.

A main effect for gender was also found, F (1, 418) = 11.90, p = .006, η = .018. This effect showed that, in general, men rated all types of images more pleasant than women. An interaction effect between gender and image category was found, F (3, 416) = 13.27, p = .000, η = .087. Post hoc analyses showed that, in comparison with men, women rated cockroach and snake images as more negative (all p´s = .000) and cat images as more positive (p =.002). No differences were found for butterfly images (p =.24).

Regarding arousal, a main effect for image category was observed, F (3, 416) = 60.09, p = .000, η = .30, from the 2 x 4 mixed repeated-measures ANOVA. Bonferroni post hoc analyses revealed that butterfly images were rated as less arousing than cockroaches, snakes and cats’ images (all p´s = .000). Moreover, snakes were rated as more arousing than cats (p =.006), and cockroaches’ ratings did not differ from scenes depicting snakes (p =.33) and cats (p =.58).

Finally, the interaction between gender and image category reached a significant level, F (3, 416) = 7.47, p = .000, η = .051. Post-hoc pairwise comparisons showed that women rated cockroaches significantly more arousing than men (p =.000). No differences were found between men and women in the arousal ratings for snakes (p =.83), cats (p =.42) and butterfly images (p =.29).

In order to analyze the effects of gender, fear, and disgust scores on the valence and arousal ratings, a multivariate multiple regression using Gender, FCQ and DPSS-12-R (propensity and sensitivity subscales) as predictors was conducted for each image category.

The analyses showed that FCQ scores (fear) significantly predicted the valence ratings for cockroaches (R =.15, F(4, 397) = 16.91, p =.000; β = – .011, p =.000) and spiders’ images (R =.07, F(4, 187) = 3.71, p =.006; β = – .007, p =.008). However, Gender and DPSS-12-R scores (both propensity and sensitivity to disgust) did not reach the significant level to be considered as potential predictors. Furthermore, Gender was revealed as the only significant predictor of valence ratings for snakes (R =.07, F(4, 397) = 7.12, p =.000; β = – .689, p =.000) and cats’ images (R =.04, F(4, 397) = 3.92, p =.004; β = .386, p =.002). Finally, valence ratings for butterfly images were significantly predicted by Gender (R = .03, F(4, 397) = 3.14, p =.015; β = .336, p =.017) and Sensitivity to disgust (β = – .035, p = .027).

The multivariate multiple regressions conducted to analyze arousal ratings revealed that FCQ scores (fear) and DPSS-12-R scores (propensity to disgust) were significant predictors for cockroaches (R =.18, F(4, 397) = 21.30, p = .000; β = .017, p = .000 and β = .036, p =.027), spiders (R = .08, F(4, 187) = 4.35, p = .002; β = .007, p = .022 and β = .059, p = .009) and butterflies images (R = .04, F(4, 397) = 4.47, p = .002; β = – .008, p = .001 and β = .048, p = .003). The arousal ratings for cat images were only predicted by propensity to disgust (R = .03, F(4, 397) = 2.91, p = .022; β = .044, p = .016). Arousal ratings for snakes were not significantly predicted by any predictor variable.

Discussion

The main goal of the present work was to develop and validate a set of animal images that included numerous pictures of cockroaches. We gathered and evaluated a total of 240 animal pictures, all from online public domains in order to allow their use in future empirical research, especially in online studies. Pictures were assessed by a large sample of Spanish adults according to two affective dimensions, valence (i.e., emotional value of the picture, from positive to negative) and arousal (i.e., the degree of intensity of the emotional response evoked by the picture), following the methodology used in previous studies (i.e., Kurdi et al., 2017). This two-dimensional affective assessing scale is based on the circumplex model of affect (Posner et al., 2005), which is supported by relevant empirical evidence from neurocognitive approaches (e.g., Heilman, 2000; Lang et al., 1993). From this integrative approach, affective states are generated from cognitive responses resulting from two independent neurophysiological systems: valence and arousal neural circuitry. In the circumplex space defined by these two basic affective circuits (pleasant-unpleasant and activation-deactivation), the discrete emotions such as fear, joy or disgust arise as the product of subsequent cognitive interpretations of physiological experiences. Furthermore, in terms of parsimony, using exclusively this two-dimensional rating scale simplifies the evaluation process as adding more properties to rate (such as the discrete emotions mentioned above) could lead to confusion in the participants’ subjective ratings and less reliable results.

Regarding our first hypothesis (i.e., we expected that cockroach pictures would obtain similar negative affective ratings to those of images depicting snakes and spiders), the results obtained in the present study revealed that cockroaches were rated as the most negative animals presented in the study, even more negative than snakes and spiders. These findings confirm our hypothesis and highlight the importance of including cockroaches in picture sets as unpleasant stimuli due to their intense negative valence. Traditionally, snakes and spiders have been the prototypical unpleasant animals selected in both behavioral and emotional studies (Gerdes et al., 2009; Öhman, 1993; Öhman & Mineka, 2001). However, our results suggest that cockroaches could be perceived as even more unpleasant—featuring the comparison with other arthropods such as spiders—, which offers new possibilities and avenues within this research field. Additionally, the current results might be useful for further investigations specifically interested in the study of cockroach phobia. In this line, a recent fMRI study has described the emotional brain circuits involved in cockroach phobia when phobic people watched cockroach images compared to non-phobic people (Rivero et al., 2017). Those results are comparable to prior findings reported in neuroimaging studies focused on snake phobia (Schaefer et al., 2014), describing a similar strong activation of specific brain areas when phobic people are confronted with the stimulus they fear. Both studies emphasized the implication of the insula, amygdala, anterior cingulate and thalamus, subcortical areas in which phobic people showed higher activation compared to non-phobic. These similarities demonstrate the relevance of continuing to explore underlying mechanisms in cockroach phobia, since people with this disorder suffer real and intense fear of these animals, being comparable to the extreme fear that snakes evoke. In this regard, a recent study focused on evaluating fear and disgust reactions of common animals showed that cockroaches are perceived as more disgusting than grass snakes while evoking similar levels of fear (Polák et al., 2020). The common presence of cockroaches in the cities of tropical and humid areas, as the Mediterranean zones (Bueno-Marí et al., 2013) where the prevalence of this disorder is especially high, aggravate this problem. Hence, we believe that our findings can contribute to this research field, helping and encouraging future studies to compare different specific phobia subtypes, delving and further describing its cognitive and neural processes.

Regarding our second hypothesis, results showed that, in line with previous studies (Lang & Bradley, 2007), women rated the unpleasant images—depicting cockroaches, snakes and spiders—more negatively, and they experienced, in general, higher arousal for cockroach pictures, confirming our hypothesis. Furthermore, there were also significant differences between men and women on pleasant images because women rated images depicting cats as more positive compared to the other categories. These results indicated that women reported more extreme ranges of affective responses for both positive and negative stimuli, compared to men, who were more moderate in their subjective ratings. These gender differences should be taken into account in future research designs.

Potential additional factors that might influence our findings were also included to ensure the reliability of the current results. In this vein, recent studies have explored how emotions such as fear or disgust can influence the emotional valuation of stimuli (Rádlová et al., 2020), showing that participants who are more sensitive to fear lose their ability to differentiate between both discrete emotions, assigning always higher values when rating the feared images. From a dimensional viewpoint, this influence could result in a lack of ability for discriminating different levels of valence and arousal, which would lead high fearful participants to give extreme low levels in valence and high in arousal when evaluating the feared images. In order to control the influence of these factors, fear of cockroaches and propensity and sensitivity to disgust measures were included in this research. Regarding the disgust scores, results in the current sample were very similar (even smaller) to the normative values obtained in the original validation work (Sandín et al., 2008) in both propensity (M = 13.72; SD = 5.06 vs. M = 15.3; SD = 3.5) and sensitivity to disgust subscales (M =11.55; SD = 4.19 vs. M = 12.2; SD = 4.0), which could indicate that our sample showed normal values for these measures. With regard to fear of cockroaches scores, no work offering normative values in Spanish population has been published yet. However, the mean score obtained in our sample (M = 44.93; SD = 32.99) was similar than the post-treatment mean score obtained by a clinical sample (M = 44.47; SD = 21.38) in a randomized control trial conducted by Botella et al. (2016), in which the patients no longer presented a diagnosis of phobia. Considering these reference values, we can conclude that our sample did not present high mean values of fear to cockroaches neither propensity nor sensitivity to disgust, being highly similar to the normative ones. Therefore, their affective ratings seemed not to be influenced by abnormal levels of fear or disgust. Furthermore, gender differences found in these variables, specially related to fear of cockroaches, are in line with the differences found in valence and arousal ratings of cockroach images, which women rated as more negative and arousing than men. This trend has been described in previous literature, as generally women tend to exhibit greater levels of fear of animals compared to men, especially for small animals and invertebrates, which could be derived from a higher disgust sensitivity and a greater aversive disposition in females (Davey, 1994). Finally, according to this evidence, analyses of the effect of gender, besides fear and disgust scores, revealed that both fear and gender were relevant predictor variables for valence ratings, while fear and propensity to disgust would significantly predict arousal ratings.

The three affective categories of images established previously (i.e., snakes, spiders and cockroaches as negative scenes, butterflies as neutral, and cats as positive stimuli) were also confirmed. Furthermore, all the picture categories were significantly different from each other, even the three negative categories (snakes, spiders, and cockroaches), which is an important requirement for standardized image sets. We want to note that for the cats’ images included in this set, the pictures generally depict “cute cats” or “kittens”, which has possibly led to more positive values in valence ratings for this specific category. Similar effects could have been observed in butterfly images, which generally include beautiful or colorful butterflies resulting in more neutral values, as these insects tend to evoke more negative reactions when the picture is too close to the viewer and much more detailed, or if it depicts more simpler butterflies. The clear differentiation between specific categories might facilitate stimulus selection, offering an easy and accurate way to find appropriate images for future studies. In this regard, the scarce differences found between images in the arousal dimension also contribute to stimulus selection because the images only differ in their affective value. This could be especially relevant when designing further attentional bias studies, where it is important for stimuli to vary only on their intrinsic emotional value, but not on other potentially disturbing attributes, such as how arousing they are perceived to be (e.g., Rinck & Becker, 2006).

In fact, neutral pictures (i.e., butterflies) were the only ones that obtained significatively lower levels of arousal compared to the other categories of images. These differences could be explained by the specific “U” shape in the distribution of the valence and arousal ratings. The images are distributed in the valence dimension from the neutral center to the positive and negative extremes. The stimuli perceived as emotionally neutral produced lower arousal, whereas pleasant and unpleasant stimuli led to higher arousal ratings. These relationships reported between arousal and valence ratings are consistent with prior studies with other picture sets (Lang et al., 2008; Moltó et al., 2013). Moreover, these results have also been obtained in previous studies using other kinds of emotional stimuli, such as affective sounds (Bradley & Lang, 1999a) and words (Bradley & Lang, 1999b).

To our knowledge, this is the first validated image set that includes a representative number of pictures of cockroaches. This particular feature of our stimuli set could be useful in specific phobia research because, until now, a validated set of pictures with a sufficient number of images depicting cockroaches was not available. In this regard, we think the number of images included in each category could be sufficient to carry out an attentional bias research task, although a larger number of pictures could certainly improve the scientific possibilities of the current picture set.

As mentioned above, attentional bias studies can benefit greatly from this validated set of animal images. The literature available in this field is especially extensive in social phobia research (e.g., Amir et al., 2003; Mogg et al., 2004), where other specific databases could be very useful (i.e., Lundqvist et al., 1998). However, studies on attentional biases in small-animal phobia are less abundant, and the majority focus on snake or spider phobia (e.g., Mogg & Bradley, 2006; Öhman et al., 2001), producing a lack of research about cockroach phobia. The differences found in our study between cockroaches and both snakes and spiders, in turn, could reflect important psychological differences among all these types of phobias, showing the need to explore the cognitive and behavioral mechanisms involved in cockroach phobia. We hope our specific animal image set can help researchers to carry out future empirical studies to improve the knowledge about cockroach phobia, as well as the psychological treatments available.

Materials availability

The complete ASSAI stimulus set, that contains 240 animal images in high resolution (1280 x 720 pixels), can be freely downloaded and used for investigation purposes from: http://hdl.handle.net/10234/186930. The archive contains, along with the images, a data file where are exposed the mean and standard deviation for valence and arousal ratings of each image. Since the results showed significant gender differences, this data file also includes the averaged affective ratings per image broken down by gender.