Affective Visual Stimuli: Characterization of the Picture Sequences Impacts by Means of Nonlinear Approaches.

INTRODUCTION: The main objective of the present study was to investigate the effect of preceding pictorial stimulus on the emotional autonomic responses of the subsequent one.
METHODS: To this effect, physiological signals, including Electrocardiogram (ECG), Pulse Rate (PR), and Galvanic Skin Response (GSR) were collected. As these signals have random and chaotic nature, nonlinear dynamics of these physiological signals were evaluated with the methods of nonlinear system theory. Considering the hypothesis that emotional responses are usually associated with previous experiences of a subject, the subjective ratings of 4 emotional states were also evaluated. Four nonlinear characteristics (including Detrended Fluctuation Analysis (DFA), based parameters, Lyapunov exponent, and approximate entropy) were implemented. Nine standard features (including mean, standard deviation, minimum, maximum, median, mode, the second, third, and fourth moment) were also extracted.
RESULTS: To evaluate the ability of features in discriminating different types of emotions, some classification approaches were appraised, of them, Probabilistic Neural Network (PNN) led to the best classification rate of 100%. The results show that considering the emotional sequences, GSR is the best candidate for the representation of the physiological changes. DISCUSSION: Lower discrimination was attained when the sequence occurred in the diagonal line of valence-arousal coordinates (for instance, positive valence and positive arousal versus negative valence and negative arousal). By employing self-assessment ranks, no obvious improvement was achieved.

Introduction

Recently, assessing physiological changes to detect human emotion has received significant attention. Therefore, to find a suitable emotion recognition system, researchers examined the potential of the physiological signals, i.e. electrocardiogram (ECG), Galvanic Skin Response (GSR)/Skin Conductivity (SC), electroencephalogram (EEG), Pulse Rate (PR), electromyography (EMG), Respiration Activity (RSP), and temperature, under different emotional states. This goal can be achieved by a wide variety of methodologies. Most of human knowledge about emotional behaviors and responses in physiological systems has been acquired by means of standard approaches; however, many biosignals are naturally random or chaotic, and administrated by nonlinear dynamical rules (Henry, Lovell, & Camacho, 2001).

Several nonlinear approaches have been offered to extract information encoded in chaotic signals; however, little attention has been paid for the application of these techniques in analyzing affective physiological parameters. Previously, the role of emotions on physiological responses was investigated by means of correlation dimension and the largest Lyapunov exponent as measures of chaos and complexity (Jeong, Joung, & Kim, 1998). The findings support the idea that chaos plays an important role in the emotional brain function. Investigations of complexity, the Lyapunov exponent, and entropy of emotional brain activity were performed by Aftanas et al. (1998, 1997). Their results suggest the necessity of complexity enhancement of cortical dynamics, up to a certain level, for emotional processing. In response to positive and negative stimuli, the Hurst exponent of cardiac signals was studied (Costa, Galati, & Rognoni, 2009). Hilbert-Huang Transform (HHT) as a nonlinear tool was applied for emotion recognition from physiological signals, including ECG, EMG, SC, and RSP (Zong and Chetouani, 2009). The classification rates indicate that HHT outperforms customary statistical techniques.

Valenza et al. (2012a) examined the role of nonlinear methods in affective valence and arousal recognition. They reported that by applying nonlinear techniques on ECG, RSP, and SC, the emotion recognition rates increases. Concurrently, in two studies, Autonomic Nervous System (ANS) dynamics were appraised during emotional elicitation (Valenza et al., 2012b; Lanata, Valenza, & Scilingo, 2013). The former evaluated the dominant Lyapunov exponent and approximate entropy of HRV during emotional visual elicitation (Valenza et al., 2012b). Results established the capability of the proposed methodology in distinction between neutral and arousal elicitation. Whereas, the latter (Lanata, Valenza, & Scilingo, 2013) examined the relationship between emotions and information coming from the eyes (eye gaze pattern and pupil size variation, as measures of ANS activity) by means of the Recurrence Quantification Analysis (RQA). Promising achievements were obtained with respect to discrimination of emotional states with different arousal content by nonlinear approaches. By applying Higher Order Statistics (HOS) and nonlinear analysis, the emotional pattern associated with ECG signals was studied (Selvaraj, Murugappan, Wan, & Yaacob, 2013). In another study, the possibility of identification of six emotional states (happiness, sadness, fear, surprise, disgust, and neutral state) was investigated using HHT of ECG (Jerritta, Murugappan, Wan, & Yaacob, 2014), where interesting results were achieved. More recently, it has claimed that immediate, personalized, and automatic assessment of emotion recognition is possible through the analysis of cardiovascular dynamics on short-time emotional stimuli (Valenza et al., 2014). Through applying advanced methodologies of biomedical signal processing, the crucial role of nonlinear dynamics in ANS analysis associated with emotion induction was also highlighted (Valenza, and Scilingo, 2014).

However, the emotional response might be influenced by several factors such as stimulus and its content (Bradley, Codispoti, Cuthbert, & Lang, 2001), age, gender (Labouvie-Vief, Lumley, Jain, & Heinze, 2003), and the like.

The effect of visual affective stimulation on physiological responses has been extensively evaluated. However, the effects of image sequences in the pictorial emotional stimulus on physiological responses have overlooked so far. This matter is examined in the current study. The scope of this investigation was to obtain information about some important issues.

First, it is endeavored to identify what features can be the most representative in emotional states. In other words, which biosignal characteristics, including standard and chaotic behavior, have the best representation of affective responses to pictorial stimuli?

The second issue attempts to find if there are any preferences among autonomic functions (including ECG, GSR and PR) to demonstrate the emotional responses rather than the others. The third one tries to assess the association between physiological responses and emotional states ranked by the subjects.

The last challengeable issue deals with the arrangement of the motivations among successive images. It has been hypothesized that the sequence of the pictorial stimuli has an impact on the emotional responses. Consequently, the effect of the preceding stimulus on the current one is examined. In particular, the aim of the present study was to investigate this matter through recordings of biosignals for the first time.

The organization of this article is as follows. First, the process of data acquisition is briefly described. Then, the concepts of standard and nonlinear features applied in the current study are introduced. Afterwards, the experimental results are reported. Finally, the conclusions are presented.

Methods

Different stages of processing and methodology applied in the study are demonstrated in Figure 1. After collecting the data, signals were preprocessed. AC power line noise (50 Hz) was removed and the signals were segmented according to the blocks of emotional load. The proposal of the current work was to investigate the role of visual stimulation sequences on emotional physiological responses. Therefore, some standard and nonlinear features, which their role was confirmed in the literature of affective computing (Aftanas et al., 1998; Aftanas et al., 1997; Jeong, Joung, & Kim, 1998; Costa, Galati, & Rognoni, 2009) were examined. Finally, a statistical test was performed.

Figure 1.

A block diagram of the methodology applied in the current study.

Data selection

To understand the physiological fluctuations elicited by emotional images, ECG, GSR, and PR of 47 college students attending Sahand University of Technology, including 31 females (age range: 19–25 years; mean age: 21.90±1.7 years) and 16 males (age range: 19–23 years; mean age: 21.1±1.48 years) were collected. All participants are Iranian students.

To elicit emotions in the participants, images from the International Affective Picture System (IAPS) were used (Lang, Bradley, & Cuthbert, 2005). The IAPS comprises hundreds of images, which are emotionally evocative. The data set is assessed by some American participants on three dimensions (on a discrete 9-point scale): valence, arousal, and dominance. In addition, participants rated the felt emotion using an emotion wheel. For all dimensions, the mean and variance of participant assessments were calculated from these evaluation scores (Lang et al., 2005).

Based on dimensional structure of the emotional space (Figure 2), the images of the IAPS corresponding to the following classes of emotions were chosen: relaxed, happy, sad, and afraid (each state is referred to a quarter of emotional spaces, varying on arousal [relaxing/stimulating], and valence [pleasant/unpleasant] dimensions).

Figure 2.

Dimensional structure of the emotional space (Scherer, 2002).

Thirty-five images per each category of emotion were selected (140 images in total). Each subset was chosen via empirical thresholds on valence and arousal scores (Table 1).

Table 1.

Empirical thresholds on valence and arousal scores for each emotion category.

Relaxed	1.5 <Aroused<4.5	5.5<Valence<9
Happy	5<Aroused<7.5	6.5<Valence<9
Sad	2<Aroused<5	1<Valence<3.5
Afraid	6<Aroused<9	1.2<Valence<4.8

Upon arrival to the laboratory, all participants were asked to read and sign a consent form to take part in the experiment. Also, the participants should declare of any particular conditions, which would cause them to be in a very relaxed or very aroused state at that moment. It should be mentioned that all participants reported no history of neurological disease, cardiovascular, epileptic, and hypertension diseases. The subjects were asked not to eat caffeine, salt, or fat foods two hours before data recording. The mean temperature of the room was about 25°C.

The participants were instructed to remain still during data recordings, particularly avoid movements of their fingers, hand, and leg. Seating in front of the laptop screen (15.5-inch, VAIO E Series), physiological responses (i.e. GSR, ECG and PR) of each subject were recorded. It should be noted that due to changes in blood pressure, expansion and contraction of the finger perimeter can be measured by a transducer. By attaching the piezo-electric transducer to the finger, monitoring of finger peripheral pulse was provided. Its acquisition is simple and easy applicable. Pulse wave is neither a pulse-oximeter signal, nor a feature of ECG. Simultaneously, ECG signals were recorded from lead I and bipolar finger electrodes were attached to the left hand to acquire GSR.

The whole procedure took about 15 minutes and images were represented after 2 minutes of rest. In the initial baseline measurement, subjects were instructed to keep their eyes open and watch a blank screen. Then, 28 blocks of pictorial stimuli were brought to the screen by random to avoid habituation in subjects. Furthermore, they were balanced among subjects. Each block consisted of 5 pictures from the same emotional class displayed about 15 seconds with a 10 second of the blank screen period at the end.

This process was done to insure the stability of the emotion over time. The blank screen period was applied to allow the return of physiological fluctuations to the baseline and assure the regularity in demonstration of different emotional images. The blank screen is followed by a white plus (for 3 seconds) to prompt the subjects to concentrate and look at the center of the screen and prepare them for the next block. Figure 3 demonstrates the protocol description.

Figure 3.

Protocol description.

As emotions are dependent on the past experiences of the subjects, they were asked to self-assess their emotional states. All signals were recorded in Computational Neuroscience Laboratory of Sahand University of Technology using 16-channel PowerLab (manufactured by AD Instruments) with a sampling rate of 400 Hz. A digital notch filter was applied to the data at 50 Hz to remove any artifacts caused by alternating current line noise.

Standard features

In the present study, the following time-based analyses were performed:

Mean

Standard deviation

Moments: the central moment of order k is demonstrated as follows: where k is considered to be 2, 3, and 4.

Maximum: the largest value of the data.

Minimum: the smallest value of the data.

Mode: the most frequent values in the series.

Median: the median value of the data.

Nonlinear analysis

Detrended fluctuation analysis

Peng et al. (1994) introduced detrended fluctuation analysis (DFA) as a measure of the statistical self-affinity of a signal. This technique is generally used for determining fractal scaling properties and long range correlation detection in noisy time series or non-stationary one.

At first, the time series with N samples is integrated. A division of the integrated time series into boxes with equal length of n, until a least squares line is fitted to the data. The integrated time series, y(k), is detrend by subtracting the local trend, yn(k), where yn(k) denotes y coordinate of the straight line segments. Finally, the root-mean-square fluctuation is calculated as:

The power-law behavior is defined as:

The scaling exponent α is an indicator of the nature of the fluctuations in the time-series, which can be correlated to the fractal dimension by D=3−α.

Lyapunov exponents

The Lyapunov exponent (LE) is defined by the average growth rate λi of the initial distance between two (typically the nearest) neighboring points in phase, where ‖δxi (0)‖ is the distance of the points at time 0 and ‖δxi (t)‖ is the distance of the points at time t:

Lyapunov exponent (λ) is often calculated to discriminate between chaotic dynamics and periodic behavior of signals:

A negative exponent indicates that the orbits approach a common fixed point.

A zero exponent indicates that the orbits maintain their relative positions; they are on a stable attractor.

A positive exponent implies that the orbits are on a chaotic attractor (Haykin & Li, 1995; Abarbanel, Brown, & Kennel, 1991).

In the current study, the actual estimation technique of Lyapunov exponent was adopted based on the method developed by Rosenstein et al. (Rosenstein, Collins, & DeLuca, 1993).

Approximate entropy

Approximate entropy (ApEn), a measure of complexity (Pincus, 1991), is defined as the logarithm likelihood that the patterns of the closest points of the data will remain near each other for the next comparison. Accordingly, a generalized measure of regularity is provided by ApEn.

After choosing two parameters of m and r, the computation of ApEn can be done. Here, the correlation integral Cm(r) must be computed (with embedding dimension m and time lag 1). This measure was finally obtained as follows: where m specifies the pattern length and r is the effective filter. In this study, m was set to 2 and r was set to 15% of the standard deviation of each time-series.

Results

Standard and nonlinear features of the biosignals, including GSR, ECG, and PR, were calculated in four affective states: afraid, sad, relaxed, and happy. Despite the focus of the majority of studies on the examination of heart rate and heart rate variability, in this study, the features have been extracted from row ECG. The gaussianity of each feature probability distribution was examined. An example is offered in the Figure 4. Then, statistical analysis was performed by means of t test to show significant differences between emotional states and rest, and between each pair of emotive parameters (Tables 2 and 3).

Figure 4.

The feature probability distribution for Lyapunov exponents of GSR during sad stimuli.

Table 2.

Comparison between emotional states and rest of autonomic measures, including GSR, ECG, and PR.

Rest with	GSR				ECG				PR
Feature	Afraid	Sad	Relaxed	Happy	Afraid	Sad	Relaxed	Happy	Afraid	Sad	Relaxed	Happy
Mean	**	**	**	**	-	-	-	-	-	*	*	-
STD	*	*	*	-	*	*	*	-	**	**	**	**
Maximum	**	**	**	**	-	-	*	-	**	**	**	**
Minimum	**	**	**	**	*	*	*	-	**	*	**	*
Median	**	**	**	**	*	*	*	-	**	**	**	**
Mode	**	**	**	**	-	-	-	-	**	**	**	**
2nd Moment	-	-	-	-	-	*	*	-	**	**	**	**
3rd Moment	-	-	-	-	*	*	-	-	**	**	**	**
4th Moment	-	-	-	-	-	-	-	-	*	*	*	*
D	**	**	**	**	*	**	**	*	**	**	**	**
α	**	**	**	**	*	**	**	*	**	**	**	**
LE	**	**	**	**	-	-	-	-	**	**	**	**
ApEn	**	*	**	*	-	-	-	-	-	-	-	-

Note:

P<0.05;

P<10−5

Table 3.

Comparison between each pair of emotional states on autonomic measures.

Rest with	GSR						ECG						PR
Feature	Sad with Afraid	Happy with Sad	Happy with Afraid	Calm with Sad	Calm with Happy	Calm with Afraid	Sad with Afraid	Happy with Sad	Happy with Afraid	Calm with Sad	Calm with Happy	Calm with Afraid	Sad with Afraid	Happy with Sad	Happy with Afraid	Calm with Sad	Calm with Happy	Calm with Afraid
Mean	**	*	**	*	**	*	-	-	-	*	*	-	-	-	-	-	-	-
STD	-	*	*	-	*	-	-	-	-	-	-	-	*	-	**	*	*	-
Maximum	**	*	**	*	**	-	-	-	-	-	-	*	**	*	**	-	*	*
Minimum	**	-	**	*	**	*	-	*	-	-	-	-	**	-	-	-	-	*
Median	**	*	**	*	**	*	-	-	*	-	*	-	*	-	*	*	*	-
Mode	**	*	**	*	**	*	-	*	*	-	*	-	*	-	-	*	-	-
2nd Moment	*	*	-	-	-	-	-	-	-	-	-	-	*	-	*	-	-	-
3rd Moment	-	-	-	-	-	-	-	-	-	-	-	*	-	-	-	-	-	-
4th Moment	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
D	-	-	*	*	*	-	-	*	-	-	*	*	-	*	-	-	*	-
α	-	-	*	*	*	-	-	*	-	-	*	*	-	*	-	-	*	-
LE	*	**	-	-	**	*	-	-	-	-	-	-	-	*	*	*	**	-
ApEn	*	*	-	*	-	-	-	-	-	-	-	-	-	-	-	-	-	-

Note:

P<0.05;

P<10−5

Comparing emotional states and the rest, shows that median is the most valid feature in terms of time domain parameters. Similarly, DFA parameters, as measures of chaotic behaviors of biosignals, represent the best pointer of affective states. In contrast, the moments, as the standard features and approximate entropy, as a nonlinear feature, lacked interesting results. In addition, it was found that PR and GSR measures showed significant differences (P<0.05) between emotional states and rest; whereas, better distinction between each pair of emotional states was achieved by means of GSR signal. Considering the results carefully, one can conclude that almost all features of GSR can discriminate emotional states from the rest, except the moments. In addition, approximate entropy does not seem to be a valid parameter in PR processing (Table 2). The results of ECG analysis show that DFA is the best feature for discriminating affective states from the rest. As there is no significant difference in some ECG features, GSR and PR are more reliable in such studies.

To assess the ability of features in discriminating different groups of emotions, some classification techniques, including feed forward back propagation network with different structures and Probabilistic Neural Network (PNN) were evaluated. In this regard, PNN outperformed the others. To this effect, the extracted features are randomly divided into two sets: a training set and a testing set. Two-third of the samples were used to train the classifiers, while one third was applied to test the performance of each classifier. An optimum smoothing parameter for PNN was derived by trial and error and set to 0.1. Results of the classification for test data set expressed in terms of confusion matrix (as shown in Table 4).

Table 4.

Confusion matrix of PNN classifier.

True Lable	Happy	Relaxed	Sad	Fear	Total
Happy	347	0	0	0	347
Relax	0	318	0	0	318
Sad	0	0	328	0	328
Fear	0	0	0	323	323
Total	347	318	328	323	1316

The results indicate that discrimination between different types of emotions can be successfully realized applying the proposed features and PNN.

In the next stage, the effect of previous pictorial stimuli category on the emotional responses of the current one was examined (Table 5). For instance, was there any difference between emotional responses to scary stimuli displayed after a sad or happy one? Considering the results, ECG parameters were not trustful indicators for affective states and GSR outperformed the others. Furthermore, considering the sequences of affective pictorial stimulus, there were no significant differences between some emotional states and rest, including a sad stimuli presented after a happy one, an afraid motivation offered after a relaxed one, and a relaxed stimulus just shown after a sad one.

Table 5.

Comparison between emotional states and rest, considering the emotional sequences.

Signal	Feature	Rest with Afraid which is after			Rest with Happy which is after			Rest with Relax which is after			Rest with Sad which is after
		Happy	Relax	Sad	Afraid	Relax	Sad	Afraid	Sad	Happy	Afraid	Relax	Happy
GSR	Mean	**	-	**	**	**	*	**	-	*	*	**	-
	STD	-	-	*	*	*	-	*	-	*	*	*	-
	Maximum	**	-	**	**	**	*	**	-	*	*	**	-
	Minimum	**	-	**	**	**	*	**	-	*	*	**	-
	Median	**	-	**	**	**	*	**	-	*	*	**	-
	Mode	**	-	**	**	**	*	**	-	*	*	**	-
	2nd Moment	-	-	-	-	*	-	*	-	-	*	-	-
	3rd Moment	-	*	-	-	*	-	*	*	-	*	-	-
	4th Moment	-	-	-	-	-	-	-	-	-	*	-	-
	D	**	-	**	*	**	-	*	*	*	*	**	-
	α	**	-	**	*	**	-	*	*	*	*	**	-
	LE	*	*	*	*	*	*	**	-	*	*	**	-
	ApEn	*	-	-	-	*	-	-	-	-	-	*	-
ECG	Mean	-	-	-	-	-	-	-	-	-	-	-	-
	STD	-	-	-	-	-	-	-	-	-	-	-	-
	Maximum	-	-	-	-	-	-	-	-	-	-	-	-
	Minimum	-	-	-	-	-	-	-	-	-	-	-	-
	Median	-	-	-	-	-	-	-	-	-	-	-	-
	Mode	-	-	-	-	-	-	-	-	-	-	-	-
	2nd Moment	-	-	-	-	-	-	-	-	-	-	-	-
	3rd Moment	-	-	-	-	-	-	-	-	-	-	-	-
	4th Moment	-	-	-	-	-	-	-	-	-	-	-	-
	D	-	-	-	*	-	-	*	-	*	-	-	-
	α	-	-	-	*	-	-	*	-	*	-	-	-
	LE	-	-	-	-	-	-	-	-	-	-	-	-
	ApEn	-	*	-	-	*	-	-	-	*	*	-	-
PR	Mean	-	*	-	-	-	-	-	-	-	-	-	-
	STD	**	-	**	*	**	*	*	-	*	*	*	-
	Maximum	*	-	*	*	**	*	*	-	*	*	*	-
	Minimum	*	-	*	*	*	-	*	-	-	-	-	-
	Median	**	*	**	*	**	*	*	-	*	*	*	-
	Mode	*	-	**	**	**	*	*	-	*	*	*	-
	2nd Moment	*	-	*	*	*	*	*	-	*	*	*	-
	3rd Moment	-	-	*	-	*	-	-	-	-	-	-	-
	4th Moment	-	-	-	-	-	-	-	-	-	-	-	-
	D	-	-	-	-	-	-	-	-	-	-	-	-
	α	-	-	-	-	-	-	-	-	-	-	-	-
	LE	*	-	*	*	*	*	*	-	*	*	*	-
	ApEn	-	-	-	-	-	*	-	*	-	-	-	-

Note:

P<0.05;

P<10−5

In the next stage, the self-assessments were examined. It was assumed that individuals were able and willing to identify and express their emotions. This method depends on simply asking subjects to define the nature of their experience. If a subject did not correctly recognize 89% of the total blocks (more than 3 mistakes), which labelled as happy, sad, scary, and relaxed, then the data were excluded for further analysis. The features of the remaining subjects were extracted and the statistical test was performed. The concise results were demonstrated in Tables 6–8. Considering the self-assessment, no obvious improvement was achieved.

Table 6.

Comparison between emotional states and rest of autonomic measures, considering the self-assessments ranks.

Rest with	GSR				ECG				PR
Feature	Afraid	Sad	Relaxed	Happy	Afraid	Sad	Relaxed	Happy	Afraid	Sad	Relaxed	Happy
Mean	**	**	**	**	-	-	-	-	-	-	-	-
STD	*	*	*	-	*	*	*	-	**	**	**	**
Maximum	**	**	**	**	-	-	*	-	**	**	**	**
Minimum	**	**	**	**	*	*	-	-	**	*	*	*
Median	**	**	**	**	*	*	*	-	**	**	**	**
Mode	**	**	**	**	-	-	-	-	**	**	**	**
2nd Moment	-	-	-	-	-	-	-	-	**	**	**	**
3rd Moment	-	-	-	*	-	*	-	-	**	**	**	**
4th Moment	-	-	-	-	-	-	-	-	**	**	**	**
D	**	**	**	**	*	*	*	*	**	**	*	**
α	**	**	**	**	*	*	*	*	**	**	*	**
LE	**	**	**	**	-	-	-	-	**	**	**	*
ApEn	**	*	**	*	-	-	-	-	-	-	-	-

Note:

P<0.05;

P<10−5

Table 8.

Comparison between emotional states and rest, considering the emotional sequences and the self-assessments ranks.

Signal	Feature	Rest with Afraid which is after			Rest with Happy which is after			Rest with Relaxed which is after			Rest with Sad which is after
		Happy	Relaxed	Sad	Afraid	Relaxed	Sad	Afraid	Sad	Happy	Afraid	Relaxed	Happy
GSR	Mean	**	*	**	**	**	*	**	-	*	*	**	-
	STD	*	-	-	*	*	-	*	-	-	*	-	-
	Maximum	**	*	**	**	**	*	**	-	*	*	**	-
	Minimum	**	*	**	**	**	*	**	-	*	*	**	-
	Median	**	*	**	**	**	*	**	-	*	*	**	-
	Mode	**	-	**	**	**	*	**	-	*	*	**	-
	2nd Moment	-	-	-	-	*	-	*	-	-	*	-	-
	3rd Moment	-	-	-	-	-	-	-	-	-	-	-	-
	4th Moment	**	-	**	*	**	-	*	*	*	*	**	-
	D	**	-	**	*	**	-	*	*	*	*	**	-
	α	**	-	**	*	**	-	*	*	*	*	**	-
	LE	*	-	*	*	*	*	*	-	*	*	*	-
	ApEn	-	-	*	-	*	-	-	-	-	-	*	-
ECG	Mean	-	-	-	-	-	-	-	-	-	-	-	-
	STD	-	-	-	-	-	-	-	-	-	-	-	-
	Maximum	-	-	-	-	-	-	-	-	-	-	-	-
	Minimum	-	-	-	-	-	-	-	-	-	-	-	-
	Median	-	-	-	-	-	-	-	-	-	-	-	-
	Mode	-	-	-	-	-	-	-	-	-	-	-	-
	2nd Moment	-	-	-	-	-	-	-	-	-	-	-	-
	3rd Moment	-	-	-	-	-	-	-	-	-	-	-	-
	4th Moment	-	-	-	*	-	-	-	-	*	-	-	-
	D	-	-	-	*	-	-	-	-	*	-	-	-
	α	-	-	-	*	-	-	-	-	*	-	-	-
	LE	-	-	-	-	-	-	-	-	-	-	-	-
	ApEn	-	-	-	-	-	-	-	-	*	-	-	-
PR	Mean	-	*	-	-	-	-	-	*	-	-	-	-
	STD	*	-	*	*	**	*	*	-	*	*	*	-
	Maximum	*	-	*	*	*	*	*	-	*	*	*	-
	Minimum	*	-	*	*	*	-	*	-	-	-	-	-
	Median	**	*	**	*	**	*	*	*	*	*	*	-
	Mode	**	-	*	*	**	*	*	*	*	*	*	-
	2nd Moment	*	*	*	*	*	*	*	*	*	*	*	-
	3rd Moment	*	*	*	*	*	*	*	*	*	*	*	-
	4th Moment	-	-	-	-	-	-	-	-	-	-	-	-
	D	-	-	-	-	-	-	-	-	-	-	-	-
	α	-	-	-	-	-	-	-	-	-	-	-	-
	LE	*	-	*	*	*	*	*	-	*	*	*	-
	ApEn	-	-	-	-	-	-	-	-	-	-	-	-

Note:

P<0.05;

P<10−5

Discussion

The scope of the present study was to examine the following assumptions: 1) Which of the chaotic and standard biosignal characteristics is associated the most with affective responses to pictorial stimuli? 2) Are there any preferences between autonomic functions to demonstrate better the emotional responses? 3) Do the sequences of the pictorial stimuli have an impact on the emotional responses? In other words, are the emotional responses of the subjects, characterized by biosignal parameters, just dependent on the preceding stimulus? 4) Do the subjective ratings have an influence on the emotional responses?

Earlier, the role of chaos in affective brain function was demonstrated (Jeong et al., 1998). Particularly, it has shown that indexes of the chaotic attractor of the EEG decrease as the pleasure of the music increases. By applying nonlinear approaches, significantly elevated values of EEG parameter were obtained in response to emotional film stimuli (Aftanas et al., 1997). The authors established that the entropy and Lyapunov exponent, as nonlinear measures, were sensitive to EEG fluctuations during emotional stimuli. It was also found that both negative and positive emotions occurred with higher complexity values of EEG signals (Aftanas et al., 1998). The effectiveness of the nonlinear techniques in analyzing affective physiological parameters, including ANS signals, was also confirmed (Valenza, and Scilingo, 2014). To recognize four different emotions, Zong and Chetouani (2009) proposed a methodology based on HHT features of ANS signals and support vector machine. The emotional states were correctly classified with the maximum accuracy of 76%. Lanata et al. (2013) attempted to discriminate between two emotional states using nonlinear features of an ANS parameter. They reported that neutral and high arousal images were recognized with a classification accuracy of 90% and 80%, respectively. Recently, the recognition rate of 52% for classifying 6 emotional states was achieved using empirical mode decomposition and discrete fourier transform of ECG signals (Jerritta et al., 2014). The results of the current study are satisfactory and outperform the previous works (Table 4).

In line with the previous findings, the results of the current study show that a better distinction is achieved applying nonlinear methods. This confirms the dynamic characteristics of biosystems. Owning to the chaotic nature of the physiological signals, by applying analysis methods based on standard features, it is impossible to obtain precise and subtle information about signal characteristics. The median, as a standard measure, and DFA parameters, as measures of chaotic behaviors of biosignals, represent the best pointer of affective states from the rest (Table 2).

Previously, the effectiveness of GSR in capturing the affective states was demonstrated (Kim, & Andre, 2008). It has also been revealed that an affective state could be recognized by means of nonlinear analysis of GSR, including ApEn, the largest Lyapunov exponent, the embedded dimensional, and correlation dimension (Wang, Liu, & Yang, 2014). These findings are in conformity with the results obtained in this study. As it was shown, a better distinction between each pair of emotional states was achieved by means of GSR signal. In addition, analyzing GSR and PR measures, significant differences (P<0.05) between emotional states and the rest were observed.

Considering the emotional sequences, GSR is the best candidate for the representation of the physiological changes. This finding again approves that GSR comprises rich information about emotion-relevant responses and can capture the emotional states of the subjects. However, the effect of the preceding stimulus on the current one is not significant in most features of the ECG. Regarding the following sequences, significant differences from the rest in GSR parameters were achieved (Table 5): sad presented after afraid; afraid presented after sad; happy presented after relaxed; relaxed presented after happy; afraid presented after happy; happy presented after afraid; relaxed presented after afraid; sad presented after relaxed.

If ‘state X presented after state Y’ is significant, then ‘state Y presented after state X’ will be expressive, except for ‘relaxed’ presented after ‘afraid’ and ‘sad’ presented after ‘relaxed’. However, there are no significant differences in the following sequences: a sad stimulus presented after happiness, an afraid motivation offered after relaxed, and a relaxed stimulus just shown after sad one. This finding may be in line with previous findings, which claimed that emotional pictures naturally draw selective attention (Bradley & Lang, 2000). Considering the dimensional structure of emotional states (assume the valence and arousal dimensions), lower discrimination is obtained when the sequence is occurring in the diagonal line (for instance, positive valence and arousal versus negative valence and arousal). Therefore, the arrangement of images in a visual induction has a significant effect on the emotional responses of individuals.

The same results were attained applying self-assessment (Table 8). By employing self-assessment ranks (with the recognition rate of 89%), no obvious improvement on the results was accomplished. These results are in contrast with previous findings, where the effect of self-assessment in emotional experimental design was explored (Chanel, Kronegg, Grandjean, & Pun, 2006).

In conclusion, the results of this study indicate that there is a relation between emotional responses to visual stimulation and sequences of images. Particularly, the best nominee for the representation of the physiological fluctuations is GSR. For some sequences of images, discrimination between affective states has been decreased. The advantages of image sequences on emotional responses can be used in many different applications such as commercial advertising, social networking, entertainment and gaming, as well as health applications. For future works, more data and different processing techniques can be taken into account to strongly confirm the results of the present study.

Table 7.

Comparison between each pair of emotional states on autonomic measures, considering the self-assessments ranks.

Rest with	GSR						ECG						PR
Feature	Sad with Afraid	Happy with Sad	Happy with Afraid	Calm with Sad	Calm with Happy	Calm with Afraid	Sad with Afraid	Happy with Sad	Happy with Afraid	Calm with Sad	Calm with Happy	Calm with Afraid	Sad with Afraid	Happy with Sad	Happy with Afraid	Calm with Sad	Calm with Happy	Calm with Afraid
Mean	*	*	**	*	*	-	-	-	-	-	-	-	-	-	-	-	-	-
STD	-	*	*	-	-	-	-	-	-	-	-	-	*	-	**	-	*	-
Maximum	*	*	**	-	*	-	-	-	-	-	*	-	*	-	*	-	*	*
Minimum	*	-	**	*	*	-	-	-	-	-	-	-	*	-	-	-	-	*
Median	*	*	**	*	*	-	-	-	-	-	-	-	*	-	*	-	-	-
Mode	*	-	**	*	*	*	-	-	-	-	-	-	-	-	*	-	*	-
2nd Moment	-	*	-	-	-	-	-	-	-	-	-	-	*	-	*	-	-	*
3rd Moment	-	-	-	-	-	-	-	-	-	-	-	-	*	-	*	-	-	*
4th Moment	-	-	-	-	-	-	-	-	-	-	-	-	*	-	*	-	-	*
D	-	-	*	-	*	-	-	*	-	-	*	-	-	*	-	-	*	-
α	-	-	*	*	*	-	-	*	-	-	*	-	-	*	-	-	*	-
LE	-	*	*	-	*	-	-	-	-	-	-	-	-	-	*	-	*	-
ApEn	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-

Note:

P<0.05;

P<10−5