Thermal infrared imaging in psychophysiology: potentialities and limits.

Functional infrared thermal imaging (fITI) is considered an upcoming, promising methodology in the emotional arena. Driven by sympathetic nerves, observations of affective nature derive from muscular activity subcutaneous blood flow as well as perspiration patterns in specific body parts. A review of 23 experimental procedures that employed fITI for investigations of affective nature is provided, along with the adopted experimental protocol and the thermal changes that took place on selected regions of interest in human and nonhuman subjects. Discussion is provided regarding the selection of an appropriate baseline, the autonomic nature of the thermal print, the experimental setup, methodological issues, limitations, and considerations, as well as future directions.

From Hippocrates' early understanding to Galileo's famous thermoscope and today's technological advancements for remote temperature measurements, scientists have always been fascinated by diagnostic temperature phenomena (Ring, 2004). Sir William Herschel in 1800 became the first scientist that measured heat beyond the visible spectrum (Ring, 2000), and, following his father's discovery, John Herschel produced the first “thermogram” in 1840 using sunlight and the evaporograph technique. Current technological advancements allowed measurement of emitted infrared heat by electronic thermal imaging (Ring, 2004; Ring & Ammer, 2012).

Thermal infrared imaging, by harnessing the body's naturally emitted thermal irradiation, enables cutaneous temperature recordings to be measured noninvasively, ecologically, and contact free. The autonomic nervous system (ANS) is at the forefront of biological heat displays, controlling unconscious heart rate, breathing, tissue metabolism, perspiration, respiration, and cutaneous blood perfusion, providing grounds for observations of emotional inference to be made. Thus, thermal infrared imaging (also referred to as functional infrared imaging, fITI), enables the characterization of the competing subdivisions of the ANS (as has already been demonstrated by previous research in the field; Ioannou et al., 2013). Bioheat-based computations of thermal infrared signs have in the majority been based on individuals' faces. This preference is attributed to the fact that the face is not obscured and is open to social communication and interaction.

Unlike conventional methods of autonomic monitoring, fITI provides versatility. It enables recording of perspiration (Ebisch et al., 2012; Pavlidis et al., 2012), cutaneous and subcutaneous temperature variations (Merla, Di Donato, Rossini, & Romani, 2004; Hahn, Whitehead, Albrecht, Lefevre, & Perrett, 2012), blood flow (Puri, Olson, Pavlidis, Levine, & Starren, 2005), cardiac pulse (Garbey, Sun, Merla, & Pavlidis, 2007), as well as metabolic breathing patterns (Pavlidis et al., 2007). The reliability of this tool has been repeatedly proven with the use of simultaneous recordings grounding fITI on gold standard methods, such as electrocardiography (ECG), piezoelectric thorax stripe for breathing monitoring, nasal thermistors, skin conductance, or galvanic skin response (GSR). As for the latter, studies have demonstrated that fITI and GSR have similar detection power (Coli, Fontanella, Ippoliti, & Merla, 2007; Kuraoka & Nakamura, 2011; Pavlidis et al., 2012; Shastri, Merla, Tsiamyrtzis, & Pavlidis, 2009; see also Figure 4 in Pavlidis et al., 2012, p. 6). Also, it has been suggested that fITI not only provides a reliable tool that enables one to infer psychophysiological excitement but also differentiates between baseline and affective states (Nhan & Chau, 2010).

Autonomic Nature of the Thermal Print for Psychophysiological Responses

Human temperature is of particular significance to medicine. Homeostatic control of cutaneous temperature is functional for both biological and psychological reasons, such as to face an environmental change, to fight a virus (e.g., fever, Skitzki, Chen, Wang, & Evans, 2007), or to support physiological demands in case of an external threat (Porges, 2001). However, temperature control associated with emotional reactions is far more complex as it serves a different purpose, has distinct neuroregulatory systems, and carries its own thermal imprints.

Triggered by cognitive appraisal of affectively charged events, the human body deploys physiological strategies that cover any environmental demand, ranging from social interaction to “fight or flight” responses. These responses are controlled by cortical segments that are phylogenetically organized in three evolutionary hierarchical structures (Porges, 2001). The newest division, the frontal lobe, inhibits lower, more primitive medullary structures, enabling vagal engagement and fostering among others social interaction. In such occasions, the environment is perceived as safe. Thus, the parasympathetic nervous system (PNS) permits normal physiological function that entails nonemergency, vegetative states, (Porges, 2001, 2009). Conversely, in bodily or emotionally threatening circumstances, the 10th cranial nerve, or vagus nerve, is disengaged (Porges, 1992), enabling the sympathetic nervous system (SNS) to take action preparing the body for the fight or flight response. Neural evaluation of threat does not require conscious awareness and involves subcortical limbic structures (Morris, deBonis, & Dolan, 2002) that receive primary inputs from the auditory and visual cortex (Uwano, Nishijo, Ono, & Tamura, 1995).

The amygdala is at the forefront of the defense response, acting as the gatekeeper for the initiation of the SNS. Located in the centre of the cortex, the amygdala affects two main cortical structures. Forebrain areas receive information for the initiation of threat engagement strategies (Lacroix, Spinelli, Heidbreder, & Feldon 2000), where specialized cells in the prefrontal cortex appraise pleasant or unpleasant events in less than 8 ms (Kawasaki et al., 2001). In addition, inputs are also received by the hypothalamus, preparing the body for periods of vigorous physical activity (Owen et al., 2006). To achieve such a goal, the hypothalamus sends nerve impulses to the adrenal medulla for the release of steroid hormones, epinephrine, and norepinephrine, enabling alertness and metabolic output (Kallat, 2007). Two biological mechanisms enable thermal observation of affective nature: subcutaneous vasoconstriction and emotional sweating. Activated by epinephrine released in the blood stream, subcutaneous vasoconstriction is a threat response that minimizes the blood volume within veins under the skin (Kistler, Mariazouls, & Berlepsch, 1998; Pavlidis, Levine, & Baukol, 2001). This mechanism protects against excessive blood loss and possible hemorrhage in case of injury and is concentrated in the most exposed parts of the body (Chien, 1967; Haddy, Overbeck, & Daugherty, 1968; Pearce & D'Alecy, 1980; Vatner, 1974; Vianna & Carrive, 2005). On the contrary, once threat has been faced, vascular relaxation is observed accompanied by a gradual temperature rise resulting from parasympathetic restoration (Nhan & Chau, 2010). Laser Doppler flowmetry and photoplethysmography suggest that changes in microcirculation caused by subcutaneous vascular constrictions or dilation need to last for at least 5 s for decreases in temperature to take place (Kistler et al., 1998). Emotional sweating is activated by norepinephrine binding on sympathetic preganglionic neurons that are situated on the spinal cord. This enables acetylcholine release in the synaptic cleft, which stimulates secretion at the sweat glands. This physiological phenomenon occurs mainly in specific body parts such as the palms, axillae, and soles of the feet. This increases elasticity and reduces friction of the skin in regions that have increased contact with the environment and external objects (Kamei et al., 1998; Porges, 1992; Vetrugno, Liguori, Cortelli, & Montagna, 2003).

The ANS also regulates facial muscles. Controlled by the brainstem through the myelinated vagus, these muscles cause expressions according to psychological and environmental factors (Nhan & Chau, 2010). Constituting an integral part of interpersonal socioemotional signaling, facial expressions have been argued to be a behavioral gateway for healthy psychological function (e.g., autism, aggressive disorders, schizophrenia; Porges, 2001). Facial muscles, like all organs of the body, require nutrients supplied through the blood stream. Thus, adjustments in blood flow occur to cover muscular activity changing the emitted thermal print. Periorbital (Levine, Pavlidis, & Cooper, 2001) and supraorbital vessels (Puri et al., 2005) of the face have been observed to show heat escalations according to stressors that are believed to facilitate preparedness for rapid eye movement in fight or flight (Pavlidis et al., 2001). During occasions such as these, and particularly startles, a temperature dip on the cheeks was observed, suggested to be the result of redirected blood to the eye musculature (Pavlidis et al., 2001) as well as of emotional sweating (Merla & Romani, 2007). The supraorbital and periorbital vessels feed the main muscles surrounding the eyes—the corrugator, procerus, and orbicularis oculi. The periorbital region has been suggested to carry information about short-lived stressors such as startles (Levine et al., 2001; Nakayama, Goto, Kuraoka, & Nakamura, 2005), which are controlled by the midbrain central gray matter, and the nucleus of the tractus solitaris in the pons (Zhao & Davis, 2004). Furthermore, the supraorbital regions have been postulated to represent prolonged periods of stress due to mental engagement (Zhu, Tsiamyrtzis, & Pavlidis, 2007; Puri et al., 2005). Although the above findings are interesting, literature on the current topic is rather inconsistent since the researchers, although using similar if not identical experimental findings, found two different sets of results regarding the effect of startles (Calvin, & Duffy, 2007; Gane, Power, Kushki, & Chau, 2012).

Validity of Infrared Imaging

To examine the reliability of the maxillary area for inferring levels of peripheral arousal (Pavlidis et al., 2012), collected temperature values of two infrared cameras were validated on GSR. Temperature measures were taken from the index finger and the upper lip, during which a total of 18 participants underwent a startle task. The synchronicity of the physiological measures was assessed on three parameters: (1) onset of activation, (2) peak, and (3) offset (relaxation). In total, the experiment lasted 4 min, in which three startles were presented. In all three critical times, the GSR showed a strong positive correlation with the temperature of the maxillary area and the index finger (3 × 3 × 2). Intensity-wise evaluation of the two slopes, ascending (peak onset) and descending (peak offset), showed that the signal trend of the GSR in each event is indistinguishable from the thermal print of the two regions of interest (ROI) (2 × 3 × 2). Despite differences in the onset of the physiological change, Merla and Romani (2007) observed a negative relationship between GSR and the thermal print of the palm and face in 10 participants that received a mild electric shock. In a seminal study, Kistler et al. (1998), using laser Doppler flowmetry, investigated whether during photoplethysmography and thermal imaging changes in fingertip temperature are the derivative of sympathetic subcutaneous vasoconstriction. By employing a variety of stressors, such as horror movies (Kistler et al., 1998), cotton swab chewing (Kistler et al., 1998), acupuncture, and deep inspiration (Kistler et al., 1996), it was observed that immediate blood flow changes as a result of vasoconstriction cause the temperature of the dorsal and palmar fingertips to decrease 15 s after stimulation. To establish whether changes of skin temperature were the result of subcutaneous constrictions, the percentage of coincidence was calculated between blood flowmetry and the thermal print while taking into account the 15-s delay. In total, 30 individuals and 222 vasoconstrictions were analyzed, indicating that on 92.3% of occasions decreases in blood flow led to a fingertip temperature dip. Moving from human to nonhuman primates, Kuraoka and Nakamura (2011) postulated that GSR and thermal imaging measures yielded the same results during analyses of variance (ANOVAs). The magnitude of change, across conditions, for both fITI and GSR was similar, as illustrated in a video of monkeys vocalizing aggressive calls and screams that significantly differed from cooing sounds. However, during the last phase of the experiment, thermal signals and GSR measures were not in agreement. Whereas GSR did not show any significant differences between visual, auditory, and audiovisual presentation of aggressive threats, fITI showed that audiovisual presentations cause a significant decrease in temperature compared with unimodal cognitive cues. Finally, studies on fear-conditioned rats show similar tendencies across physiological measures with fITI. Specifically, exposure of the subject to the fear chamber introduced increases in mean arterial pressure (MAP) by +26 mmHgm, heart rate (HR) +80 bpm, body temperature +1.41°C, back temperature +1.2°C, tail temperature -5.3°C, paws −7.5°C, eyes +1.20°C, and head +1.5°C. Changes in MAP were observed within 10 min whereas for HR it was 15 min after exposure. Upon return to the home box, levels of arousal returned to baseline levels within 50–60 min. On the other hand, thermal imaging of different parts of the body showed a further delay for temperature to reach baseline values. The body took 60–75 min, the back 30, the tail 10, the paws 15, the eyes 14, and the head 30 min. Approaching the reliability of fITI from a behavioral rather than physiological perspective, Ioannou et al. (2013) observed that, while temperature of the nose decreased, behavioral signs of distress increased, whereas when those signs decreased, the nose returned to baseline values. In addition, Pavlidis et al. (2012) reported that, during the laparoscopic drilling task, novices compared to experts surgeons exhibited substantially more distressful facial signs that were in agreement with higher temperatures as well as activation of perspiration pores on the upper lip. Although the above evidence in animals and humans provides a valuable insight into the reliability of fITI, a direct comparison between physiological measures might not be the most ideal as they have different response latencies and receive inputs for arousal by different control systems.

Creating an Experimental Setting

Cutaneous thermal responses to external stimuli of psychophysiological valence could result in small temperature variations of the ROIs. Thus, it is extremely important to ensure that the observed temperature variations are not artifacts due to either environmental physiological causes or simply subject motion. As for the environment, a crucial role is played by the experimental room or setup in which measurements take place. The environmental temperature has to be steady throughout the experimental session. No direct ventilation should hit the subject (Levine et al., 2001). In addition, if possible, room temperature and relative humidity should be set at comfortable values (i.e., thermoneutrality) for the subject. For example, in Western countries these values are usually set at approximately 22–24°C and 50–60%, respectively (Merla & Romani, 2006). Several technical solutions are available in this perspective, all of them capitalizing on the continuous monitoring of the environmental conditions (Gane et al., 2012; Nakanishi & Imai-Matsumura, 2008). Other issues that should be taken into account during the experimental setup are the prevalence or absence of systemic sources of thermal noise. These include thermal reflective walls, furniture material, direct sunlight through windows, and heat-emitting monitors in close proximity to the participant's face. These will result in overestimation or underestimation of the physiological temperature changes in the given ROIs. Prior to the initiation of the study, the participant or subject (along with the experimenter) should be left to acclimatize in the room for 10–20 min. This enables the restoration of cardiovascular and respiratory activity, as well as allows skin temperature to reach a thermal equilibrium with the experimental room (Ring & Ammer, 2006). Some authors suggest, removal of corrective eyewear prior to the experimental session because glass is opaque to infrared light. Once removed, enough time should be provided to allow pressure-related temperature restoration of the surrounding tissue of the nose (Gane et al., 2012). During recordings, the distance between the camera and the subject depends on the size of the ROIs to be imaged and the camera's optics. Usually, the camera was placed 1–3 m away from the participant (Nakanishi & Imai-Matsumura, 2008; Shastri, Merla, Tsiamyrtzis, & Pavlidis, 2009) or 30–70 cm away from the subject (Kuraoka & Nakamura, 2011; Vianna & Carrive 2005).

Exclusion criteria for participation in a fITI study include aspects related to normal cutaneous thermoregulation, such as peripheral neuropathy, micro- and macroangiopathies, connective tissue diseases, and psychophysiological disorders. Requirements for participation include the abstinence from intake or consumption of vasoactive substances (nicotine, caffeine, alcohol) for at least 2–3 h prior to participation in order to improve reliability of the assessments (Merla & Romani, 2006).

Finally, it is important to take into account the circadian rhythm of the human body when conducting an experiment. Recordings should take place for the majority of participants at the same time and season in order to have consistent group comparisons and to be able to observe temperature variations on the same scale. It has been illustrated that skin temperature varies throughout the day. Whereas in the evening the core body temperature and proximal skin temperature rise in contrast to distal skin temperature, the opposite effect seems to take place in the morning (Kräuchi & Wirz-Justice, 1994). Furthermore, since heat exchange with the environment occurs by “means of conduction, convection, radiation and evaporation” (Kräuchi & Wirz-Justice, 1994, p. 148), different homeostatic mechanisms take place during different seasons. Sweating and increased blood flow followed by peripheral blood vessel constriction is observed in warm seasons, whereas the opposite happens in cold climatic conditions. This phenomenon occurs mainly through smooth muscles in arterioles and arteriovenous anastomoses in distal skin regions such as the fingers, nose, toes, and ears (Hales, 1985).

Selection of an Appropriate Baseline in fITI

Facial thermal prints provide a great channel for inferring emotional arousal. In order to interpret observations of an affective nature, the selection of an appropriate baseline represents a major methodological challenge (Levenson, 1988). Establishing an autonomic starting point will be the foundation for the definition of the directionality of the physiological change during emotional arousal. So, for example, if a particular emotion, such as fear, is assumed to cause an increase in supraorbital and periorbital temperature, then the question would be, “What would make an appropriate baseline?” Two baseline conditions can be used for observations of an affective nature: one that serves as an actual comparison with the experimental condition and the other that is totally absent, basing observation only on the thermal signature of the affective stimulus.

The use of “rest” as a baseline condition provides grounds for experimental comparisons. In such occasions, participants are required to “rest and empty their minds of all thoughts, feelings, and memories” (Levenson, 1988, p. 24). Using rest as a baseline does give adequate contrast power since during this emotional state the parasympathetic nervous system is active and the vagal nerve is engaged. However, it rarely represents a natural emotional setting. Emotions occur while the ANS and the organism are engaged. Thus, it would be better to employ a method that has moderate levels of ANS activation. Opposing or near opposing emotions to the experimental condition could also serve as appropriate baselines. So, if happiness is the emotion of interest, then appropriate baselines would be sadness, anger, and even fear. This comparison is based on the assumption that the ANS runs on two interlinked opposing subdivisions. Happiness, although a positive emotion, is characterized by an increase in heart rate as a result of vagal withdrawal; however, it differs from negative emotions in terms of peripheral vasodilation (Kreibig, 2010, p. 23). One primary component of temperature change is the change of blood flow to the surface of the skin. Thus, if the above emotions are matched consistently in the scientific literature, if not in all at least in most aspects of physiological activity, peripheral constriction or vasodilation is going to provide the basis on which the thermal observation will be observed. Thus, the above emotions provide a good contrast pair since the ANS is always active and not under the conscious control of the individual (Winkielman & Berridge, 2003, 2004).

The alternative baseline is to eliminate it entirely, without examining changes from a reference point (Levenson, 1988). This approach could be used either for between-subjects comparisons and within-subject comparison. Although this method appears to provide an easy solution for defining an appropriate baseline, it has two main drawbacks. First, no baseline means no directionality description for the target emotion. Second, if an individual is subjected to two different emotional conditions and does not provide measurable autonomic signs in one of the two, then no matter how good the results of the other condition are, no contrast between the two emotions can be performed.

The average temperatures that have been documented in humans during rest or prestimulation were 32.3°C for the palm, 34.75 °C for the face (Merla & Romani, 2007), 30–34°C for the fingertips (Kistler et al., 1998), 34°C for the forehead, and 31°C for the nose (Calvin & Duffy, 2007). Moreover, for experiments for which stimuli were used to establish a baseline, the nose was on average 33.08 (Ioannou et al., 2013), and 35.6 for the face (Hahn et al., 2013). In addition, Nakayama et al. (2005) documented temperatures of 28.2 ± 0.5, 27.0 ± 0.7, 34.6 ± 0.3, and 33.4 ± 0.3°C for four different subjects during prestimulation. Kuraoka & Nakamura (2011) observed an average temperature of the nose during baseline of approximately 34–36°C. Finally, in the case of rats during rest, the body temperature was 37.45°C, for the back 31.7°C, for the tail 31.6°C, paws 34.8°C, eyes 35.2°C, and head 32.8°C.

No gold standard exists for choosing an appropriate baseline. However, different types of baselines have been used across thermal imaging studies, and in certain cases they could be easily applied in studies with similar design. Kreibig (2010) provides a review of autonomic reactions during positive and negative emotional states, which should be used as a guide for choosing an appropriate reference point.

Differentiating Between Emotions

There are two main theories of the autonomic function in emotions: the differentiated (Alexander, 1950) and the undifferentiated ANS (Mandler, 1975; Schachter & Singer, 1962). Whereas the differentiated theory talks about different emotions in multiple patterns of ANS activation, the undifferentiated theory states that all emotions are governed by two ANS subdivisions (Cannon, 1929). In contemporary research, the undifferentiated theory has been the most dominant one. Very few emotions in the arena of the ANS can be specified by their physiological mark since the ANS is divided into PNS and SNS. However, if multiple physiological measures (e.g., breathing, galvanic skin response, heart rate, blood volume, cutaneous thermal variation) are taken into account (Kreibig, 2010) along with the speed of onset, intensity, and duration, then it is possible that specific patterns could be defined for each specific emotion (Levenson, 1988). In support of this argument, Nummenmaa, Glerean, Hari, and Hietanen (2013), using a unique topographical self-report method, have observed in a large population sample (n = 701) that different emotions have statistically separable bodily sensation maps. The collected responses showed that different emotions not only are consciously felt but also represent topographically distinct somatosensory experiences. Finally, although it is given to assume that parasympathetic arousal is associated with pleasurable emotions and sympathetic with negative ones, this is not always true. For example, lacrimal glands responsible for tear secretion are reached only by parasympathetic efferent nerves (Lutz, 1999). Even positive emotions, such as laughter, exhibit a sympathetic element rather than a parasympathetic one (Nakanishi & Imai-Matsumura, 2008).

Regions of Interest for Studying Psychophysiology with fITI

fITI has been adopted in a variety of studies involving human emotions as well as reflexes. In particular, it has been used to study startle response, empathy, guilt, embarrassment, sexual arousal, stress, fear, anxiety, pain, and joy. To extract information of affective nature, ROIs are used. Regions on which most observations were based were the nose or nose tip, the periorbital and supraorbital vessels of the face usually associated with the corrugator muscle, forehead, and the orbicularis oculi (surrounding the eyes), as well as the maxillary area or the upper lip (perinasal). Regions on which fewer observations were gathered were the cheeks, carotid, eye, fingers, as well as the lips (Figure 1). According to the subjects' response to the emotional stimulus as well as the ROI, temperature elevates or decreases. Table 1 provides a summary of the emotions as well as the regions in which observations were based. The direction of the average temperature change in those regions is reported as well (see also Appendix Table 1001).

Figure 1

Thermal representation for extraction of ROIs along with a vascular representation of the major vessels affecting the subcutaneous temperature of the face (Berkovitz, Kirsch, Moxham, Alusi, & Cheesman, 2013).

Table 1

Overview of the Direction of Temperature Variation in the Considered Regions of Interest Across Emotions

Emotions
	Stress	Fear	Startle	Sexual arousal	Anxiety	Joy	Pain	Guilt
Regions
Nose	↓	↓		↑		↓		↓
Cheeks			↓
Periorbital			↑	↑	↑
Supraorbital			↑		↑
Forehead	↓↑	↓		↑	↑		↓
Maxillary	↓	↓	↓				↓	↓
Neck–carotid			↑
Nose	↓
Tail		↓					↓
Fingers/palm		↓					↓
Lips/mouth				↑

Table A1

Overview of fITI-Based Studies in Psychophysiology

No.	Author	Year	N	Emotion	Experimental paradigm	Baseline	ROIs
1	Calvin & Duffy	2007	33	Stress	Driving/mental loading task	Rest	Forehead, nose
2	Pavlidis et al.	2012	17	Stress	Laparoscopic drill training	Natural landscapes	Perinasal
3	Puri et al.	2005	12	Stress	Stroop test	Rest	Supraorbital vessels
4	Kang et al.	2006	9	Stress	Alphabet arithmetic task	Rest	Forehead, nose
5	Mizukami et al.	1990	34 (pairs)	Stress	Mother–infant separation stress/stranger exposure	Held by mother	Forehead
6	Kistler, et al.	1998	20	Fear	Horror movie	Prestimulation	Fingers
7	Merla & Romani	2007	10	Fear	Electric stimulation & trigger	Prestimulation	Face, palm
8	Kuraoka & Nakamura	2011	3 (monkeys)	Fear	Raging monkey, aggressive expressions & calls	Rest, prestimulation acclimatization period	Nose
9	Nakayama et al.	2005	4 (monkeys)	Fear	Threatening person	Rest, prestimulation acclimatization period	Nose
10	Vianna & Carrive	2005	12 (rats)	Fear	Foot shock chamber	Rest, prestimulation	All the body
11	Shastri et al.	2012	10	Startle	glass breaking, phone ringing	Mental task-counting circles	Periorbital, supraorbital, maxillary
12	Naemura et al.	1993	52	Startle	White noise (45–100 db)	Comparison between groups	Nasal region
13	Pavlidis et al.	2001	6	Startle	Loud noise (60 dB)	Rest, sit quietly in a dark room.	Periorbital area, cheeks, neck area
14	Gane et al.	2011	11	Startle	Loud noise (102 dB)	Image-matching task	Periorbital
15	Ebisch et al.	2012	12 (dyads)	Empathy	Toy mishap	Playing with toys	Face: nose, maxillary
16	Manini et al.	2013	18 (dyads)	Empathy	Toy mishap	Playing with toys	Face: nose, maxillary
17	Ioannou et al.	2013	15	Guilt	Toy mishap	Playing with toys	Nose
18	Hahn et al.	2012	16	Sexual arousal	Touch on high intimate regions	Neutral face presentation	Nose, lip, periorbital
19	Merla & Romani	2007	10	Embarrassment	Presence of strangers while performing a mental task	Prestimulation	Maxillary, face, palm
20	Nakanishi & Matsumura	2008	12	Laughter	Playing	Prestimulation/acclimatization	Nose, forehead, cheek
21	Pavlidis et al.	2002	12	Anxiety	Mock interrogation	Prestimulation	Face
22	Tsiamyrtzis et al.	2006	39	Anxiety	Mock interrogation	Prestimulation	Periorbital vessels
23	Zhu et al.	2007	38	Anxiety	Mock interrogation	Prestimulation	Supraorbital vessels

Note. Emotion labels were given according to the author's focus of study. The table indicates the baseline used as well as the experimental approach followed to induce the emotion of interest. The last column represents the region of interest (ROI) in which affective observations were made.

Infrared Data Acquisition

To perform thermal imaging recordings, researchers have used camera models of different resolution. The majority of studies used a camera with a resolution of 256 × 256 (Merla & Romani, 2007) to 320 × 240 (Calvin & Duffy, 2007; Ebisch et al., 2012; Hahn et al., 2012; Ioannou et al., 2013; Kang, McGinley, McFadyen, & Babski-Reeves, 2006; Kuraoka & Nakamura, 2011; Pavlidis, Eberhardt, & Levine, 2002; Pavlidis et al., 2001, 2012; Puri et al., 2005; Shastri, Papadakis, Tsiamyrtzis, Bass, & Pavlidis, 2012; Vianna & Carrive, 2005; Zhu et al., 2007). Recordings, however, of higher image quality such as 640 × 480 (Gane et al., 2012; Manini et al., 2013) to 640 × 512 (Pavlidis et al., 2012; Shastri et al., 2012) were relatively more rare. One reason for this is because thermal cameras are rather expensive and their price depends on the quality of the produced image as well as the frame rate. The use of higher or lower image quality is relevant to the experimental question as activation of perspiration pores is difficult to identify at low resolution (Pavlidis at al., 2012). In addition, higher resolution means maximum storage size and, when the temporal latency of the thermal print is a variable of interest, then the researcher needs to sacrifice image quality for higher frame rate (Merla & Romani, 2007; Puri et al., 2005).

Temperature measurements were extracted in most experiments continuously. Frame acquisition rate varied from 60 frames per second (fps) (Tsiamyrtzis et al., 2006), 50 fps (Merla & Romani, 2007), 31 fps (Pavlidis et al., 2002, 2012; Puri et al., 2005; Shastri et al., 2012; Zhu et al., 2007), 15 fps (Gane et al., 2012), 1 fps (Ebisch et al., 2012; Kang et al., 2006; Ioannou et al., 2013; Kuraoka & Nakamura, 2011; Manini et al., 2013; Nakanishi & Imai-Matsumura, 2008), 1 frame every 10 s (Kistler et al., 1998; Nakayama et al., 2005), and 1 frame every 75 s (Hahn et al., 2012). Other experiments have used isolated images equidistant in time to make their analyses such as Vianna and Carrive (2005), who acquired frames every 2 min. In the case of Calvin and Duffy (2007), temperature was documented in isolated time intervals, prior (baseline) and just immediately after the completion of the task, leaving a “physiological washout” period between the experimental conditions. Nevertheless, more frames throughout an experimental task enable the possibility of more efficient movement tracking (Manini et al., 2013; Pavlidis et al., 2002, 2012; Shastri et al., 2012; Tsiamyrtzis et al., 2006; Zhu et al., 2007), yield more temperature data, allow investigation of the temporal occurrence of the physiological phenomenon in more detail, and provide the flexibility of discarding frames that are judged unusable because of the participants' movements (Ebisch et al., 2012; Ioannou et al., 2013; Nakanishi & Imai-Matsumura, 2008).

Temperature Extraction

Moving from problems of physiological definition to methodological considerations, ROI size across thermal frames poses a challenge. When it comes to extracting data from single frames, manual fluctuations in the size of ROIs can cause thermal observations of a nonphysiologic nature, which can lead to false conclusions as a result of methodological inconsistencies. This can be easily tackled; however, it needs to be part of the knowledge of any thermographer. Analyses of ecological experiments where participants are free to move without any restriction are rather time consuming as frame-by-frame preprocessing is required. All frames need be extracted of approximately the same angle so that temperature artifacts are avoided (Ebisch et al., 2012; Ioannou et al., 2013). Even in this case, though, frames might appear to have some small temperature fluctuations from ± 0.1 to ± 0.2. This is insignificant and will not affect the overall value of the condition. Once all data is extracted, however, it is advised that a stem-and-leaf plot is created to identify any outliers that might exist in the data set so that they can be eliminated. Most fITI studies have not controlled for the reliability of data extraction. This is a crucial part and should not be overlooked in future studies. To control for reliability issues, a rater naïve to the experimental protocol can perform the analyses, or the data set can be split in two with two independent raters performing temperature extraction, since manual extraction of temperature is quite laborious. Nevertheless, probably the best possible solution in which complete confidence can be provided to data extraction is if the leading scientist performs the initial preprocessing of frames for all participants and then provides the selected frames for interrater evaluation. The second rater will then select and evaluate a sample of the participants without the knowledge of the primary rater for at least one third of the sample. Then, the mean for each of the participants will be calculated for each condition and compared between the two raters using ANOVAs to examine if significant differences between the two data sets exist. Interrater reliability can also be assessed using a kappa measure of agreement; however, the tendency of the temperature from baseline to experimental condition will first need to be assessed for each individual using a Mann-Whitney U and then coded categorically for each condition as (a) ascending, (b) descending, or (c) stable. In addition, likely the best solution would be to calculate the mean degree of change from one condition to the other for each individual and then run a kappa measure of agreement or an ANOVA to examine if interrater coding is in agreement. Studies with automatic tracking, on the other hand, do not entail dangers as such (Manini et al., 2013; Tsiamyrtzis et al., 2006; Zhu et al., 2007). Particularly, Dowdal, Pavlidis, and Tsiamyrtzis (2006) have illustrated a tracking method using coalition game theory, which accurately monitors the motion of the target surface even if the surface is partially occluded or uneven. The main principle of this technique lies in the fact that ROIs or trackers “communicate” with each other as they are interlinked in a shape divided by smaller surface areas. On occasions where a particular ROI is occluded because of the participant's movement, the rest of the trackers that are still on target continue to reliably track tissue by “tipping off” others that have gone off view. This computational method can automatically extract infrared data from ROIs (e.g., nose) continuously, without losing consistency across measures or frames and without introducing noise due to the participant's motion or due to human error. Finally, manual data extraction compared with tracking gives the researcher complete control over the data set as even in the occasion of tracking, if unnoticed, movement can still induce some noise artifacts (Gane et al., 2012). Although the tracking algorithm by Tsiamyrtzis et al. (2006) probably provides the best possible solution in terms of software tracking in either voluntary or involuntary movement, Gane et al. (2012) suggest that “in an access context, multiple thermal imaging devices positioned at different angles may be required to ensure that the user's face remains in view” (p. 7).

Statistical Testing of Heat Maps

The analyses of the collected heat prints across conditions has so far been based on the research question as well as on the way in which physiological arousal was collected and coded. The majority of experiments have extracted temperature based on the average heat signature of pixels in a ROI (Ioannou et al., 2013). However, Kuraoka and Nakamura (2011) as well as Nakayama et al. (2005) measured temperature on a ROI pixel by pixel, and reported the absolute temperature of single pixels. This was possible only because the subjects were relatively static and corrections were still applied for movement artifacts. In general, studies have used ANOVAs or their nonparametric alternative to draw conclusions about differences between conditions and groups. Prior to any type of analyses, the mean temperature for each ROI and individual was calculated and the mean of each phase was compared using one-way repeated measures ANOVAs (Ebisch et al., 2012; Gane et al., 2012; Ioannou et al., 2013; Kang et al., 2006; Kuraoka & Nakamura, 2011; Manini et al., 2013; Merla & Romani, 2007; Vianna & Carrive, 2005). The nonparametric alternative of repeated measures ANOVA, a Friedman test was used by Nakayama et al. (2005) to evaluate changes on nasal skin temperature during different respiration rates. On occasions, however, where measurements took place for separate groups, one-way between-groups ANOVA (Kuraoka & Nakamura, 2011) with a Tukey HSD, a two-way between-groups ANOVA (Pavlidis et al., 2012) with a Tukey HSD or a Kruskal-Wallis test (Nakanishi & Imai-Matsumura, 2008) followed by a Steel-Dwass test have been used. Moreover, when temperature was assessed for fewer than three conditions, the majority of experiments used a paired sample t test with Bonferonni corrections (Hahn et al., 2012; Kistler et al., 1998; Kuraoka & Nakamura, 2011; Nakayama et al., 2005; Pavlidis et al., 2012) or a nonparametric Wilcoxon matched pair signed rank test (Calvin & Duffy, 2007). Others have used algorithmic computational application to assess intraindividual changes by first segmenting periorbital and supraorbital blood vessels and then calculating changes in heat patterns between conditions (Gane et al., 2012; Pavlidis et al., 2002; Shastri et al., 2012; Tsiamyrtzis et al., 2006; Zhu et al., 2007). At the individual level, only two studies have examined temperature changes across conditions and they used a paired sample t test (Nakayama et al., 2005) or a Mann-Whitney U (Ioannou et al., 2013).

Temporal Latency of Cutaneous Temperature Change

Thermal signal development as a result of vascular change, perspiration, or muscular activity is rather sluggish compared with other measures of physiological arousal (Kang et al., 2006). Kuraoka and Nakamura (2011) observed that GSR has a latency of 3 s whereas the fastest observable change that can be recorded with thermal imaging is 10 s. In addition, Merla and Romani (2007) reported the same GSR latency compared with a thermal response of 3.8 s after stimulus onset. To investigate the timing of the temperature change, repeated measures ANOVAs with Dunnett's t test have been used (Ebisch et al., 2012; Kuraoka & Nakamura, 2011) and paired sample t tests with Bonferonni correction (Kistler et al., 1998; Kuraoka & Nakamura, 2011; Nakayama et al., 2005). To examine changes in temperature using t tests, researchers selected frames from the prestimulation and poststimulation equidistant in time. All studies prior to any analyses have calculated the average value of each frame and reported their results at the group level. Only Nakayama et al. (2005) provided statistical changes in temperature that occurred also at the individual level. In the case of Kuraoka and Nakamura (2011), a total of 10 frames were extracted from each individual for the t test. Three of those (30 s, 20 s, 10 s) were prior to the stimulation (0 s) and six after (10 s, 20 s, 30 s, 40 s, 50 s, 60 s) the stimulation. For Dunnett's t test, eight frames in total were selected taken 10 s prior to the stimulation, during stimulation 0 s, and after stimulation with a 10-s time difference (10 s, 20 s, 30 s, 40 s, 50 s, 60 s). During these tests, results have shown that fear induction significantly decreases the temperature of the nose of the monkey on average within 50 s (Nakayama et al., 2005). At the individual level, these changes have taken place from 10–110 s, lasted 220–280 s during and even after stimulation. Pixels of the nose, which had shorter response latencies, in particular 10 s, were situated on the bridge of the nose, whereas pixels with delayed responses were located on the nasal tip. Another study in the fear domain observed on average changes within 20 s of stimulus presentation that lasted in excess of 60 s (Kuraoka & Nakamura, 2011). Kistler et al. (1998), by using thermal infrared imaging and laser Doppler flux, provide valuable insight into the delayed response of the thermal print. The researchers, while controlling for perspiration patterns with GSR, concluded that cutaneous temperature change occurs only if decreases in subcutaneous blood flows are present for at least 5 s, leading to a heat evasion of 15 s after stimulation, averaging a delay of approximately 20 s. Although the above observation was made on dorsal fingertips, the same phenomenon is true on facial skin temperature as the layer of flesh is very thin (Pavlidis et al., 2001). From humans to mice, the latency of the thermal print looks different. Fear-conditioned mice showed that different parts of the body have different response latencies. Re-exposure of the subject to the aversive shock box caused an increase in body temperature. When the rat was returned to the home box, body temperature took 60–75 min to return to baseline values. The same temperature tendency was observed also for the back and the head of the animal, with temperature values descending to baseline 30 min after exposure. The eyes of the animal also rose in temperature but, unlike the above regions, baseline temperatures were reached in 14 min. On the contrary, the tail and paws showed a decrease in temperature after re-exposure to the shock box. Once the animal was in a safe environment again, temperature started to rise reaching baseline levels within 10 min for the tail and 15 min for the paws. In a similar experimental context, Merla and Romani (2007) observed that subpainful stimulation caused a decrease in temperature of the face at 3.8 s that lasted for approximately 15–20 s on average after a mild electric shock was delivered. On the other hand, reports on startles by Pavlidis et al. (2001) document a temperature change in less than 30 s upon stimulation. Moving away from fear and into the context of guilt and empathy, Ebisch et al. (2012) observed temperature changes at the maxillary area and nasal tip 10 s after each condition, lasting approximately 20–30 s regardless of distress or soothing. Finally, Nakanishi and Imai-Matsumura (2008) postulated that infants of 2–3 months old had a decrease in nasal temperature 2 min after the onset of laughter, whereas in infants 4–6 months old and 8–10 months old temperature had significantly decreased within the first 15 s. All three infant groups showed the same temporal dip on the forehead and cheeks 2 min after the onset of laughter.

Empirical Findings of Emotions Studied with fITI

Stress

Working conditions are governed by mental tasks. Nonintrusive physiological measures for assessing mental workload are required, and fITI may provide a possible solution. Focused on professional drivers, a study of occupational ergonomics assessed mental workload using fITI. Participants were exposed to simulator driving tasks both in the city as well as on the highway while cognitively challenged with a mental loading task (MLT). Compared with temperatures of the predriving session (baseline), significant differences were observed in the nose temperature across all conditions. The MLT seemed to have a defining effect on the temperature decrease of the nose, which dropped 0.55°C below baseline during the simulated city drive. No significant changes were observed on the forehead (Calvin & Duffy, 2007). Unlike other surfaces of the body, the forehead has the most stable temperature (Stoll, 1964). Staying in the occupational arena, in a seminal study, levels of stress in expert and novice surgeons were measured during training on three different drilling tasks, designed for laparoscopic surgery. The authors monitored the perinasal facial regions of participants and observed higher levels of distress in novice compared with expert surgeons. Distress signs were assessed by lower temperatures of the perinasal region along with the activation of perspiration pores (Pavlidis et al., 2012; Shastri et al., 2012; see also Figure 2 in Pavlidis et al., 2012, p. 4). In another case, and particularly in the topic of human–computer interaction, authors used a Stroop task to wirelessly exploit signs of frustration. Based on frontal forehead regions, they observed that, compared with rest, stress increased blood volume into supraorbital vessels, which in turn dissipated convective heat (Puri et al., 2005; Zhu et al., 2007). Thermal IR imaging has also been used to assess affective training times. Learning proficiency patterns were based on an alphabet arithmetic task (Kang et al., 2006). During the first trials, nose temperatures were lower as a sign of task difficulty. However, with task proficiency after training trials, not only nose temperatures rose systematically across the seven blocks but also individuals became more accurate and quicker in their responses. Forehead temperature remained the same throughout the experimental phases. Moving from affective learning to child development, Mizukami, Kobayashi, Ishii, and Iwata (1990) studied early infant attachment with the help of thermography. Infants were exposed to three different experimental phases including separation from the mother, a short-lived replacement of the mother by a stranger, as well as the infant being in the presence of the mother and a stranger. By observing negative temperature changes on the infants' forehead, the researchers illustrated that infants are aware of strangers and that infants form an attachment earlier than previously thought, specifically from 2–4 months after birth.

Fear

In an interesting experimental paradigm, Kistler et al. (1998) induced fear in participants through scenes from a thriller movie. The shower scene of Alfred Hitchcock's movie “Psycho” was used for this purpose. Baseline comparisons prior to the killing section showed that temperature changes of the fingertips reached decreases of up to 2°C as a result of vasoconstriction. In another study, Merla and Romani (2007) studied thermal signals of the face in fear-conditioned individuals. Unexpected subpainful mild electric stimuli were randomly delivered to the subject's median nerve. Results showed a reduction of temperature and sweating on the perioral region, forehead, and palms. Despite the above human studies, the majority of fear-related studies involve animals. Monkeys exposed to a video of a raging monkey along with aggressive threat vocalizations had the most marked temperature decrease on the nose than any of the above stimuli presented alone (Kuraoka & Nakamura, 2011). On the other hand, sounds produced in response to food and as a result of separation from the mother or the social group had no significant changes on the nasal temperature in comparison to baseline (Kuraoka & Nakamura, 2011). The authors concluded that the temperature of the nose should be used as an indicator of affective states in animals. In another experiment involving monkeys, fear was elicited with the approach of a threatening person. Temperature decreases were documented on the nasal regions, whereas temperature increases on the eyelid's adjacent regions as well as the region under the nostrils were considered inconsistent among subjects (Nakayama et al., 2005). Rats have so far been the subjects of preference for the majority of neuroscience laboratories. In an interesting study, fear-conditioned rats showed temperature decreases in their paws and tails as a response to the shock box (designed to deliver small electric shocks). On the contrary, the subject's return to the home box restored the body heat to baseline temperatures (Vianna & Carrive, 2005).

Startle

The startle response is a natural reflex. In real-life situations, it occurs when subjects are cognitively engaged with a task and a sudden event requires immediate shift of attention followed by rapid somatic (i.e., motor and behavioral) response such as blinking or avoidance. Shastri et al. (2009) managed to grasp this element of surprise by using natural sounds (such as glass breaking and phone ringing) as the experimental condition. Baseline recordings were made while individuals were performing a mental task (counting randomly appearing circles on the monitor). Results showed that, during startles, perspiration pores on the maxillary area became active, decreasing the temperature of the upper lip and surrounding regions (see also Figure 1 in Shastri et al., 2012, p. 367). In addition, temperature increases were observed on supraorbital and periorbital regions of the face. Naemura, Tsuda, and Suzuki (1993) observed that, compared with a startle of 45 dB, white noise of 100 db causes nasal skin temperature to decrease. Pavlidis et al. (2001) reported similar results in an experiment in which participants were exposed to a loud startle of 60 dB after sitting quietly in a dark room. Temperature increases were observed on the periorbital and neck areas (over the carotid), whereas the participant's cheeks cooled. To understand this physiological phenomenon of heat distribution, the researchers injected epinephrine (adrenaline) shots to the participant's hand. Evacuation of heat from the point of the injection to the surrounding vasculature and tissue was observed. Taking into account the warming of the carotid, investigators attributed the selective heat change of the face to the adrenergic system, further suggesting the redirection of blood from the cheeks to the periorbital region. On the other hand, similar experimental protocols did not yield the same results, and investigators reported no temperature changes on the periorbital regions as a response to startle (Gane et al., 2012). In this experiment, participants were required to complete an image-matching task, and they had to press a button when image pairs matched. At unexpected time intervals, the 102 dB auditory startle stimulus was presented.

Empathy

Studies involving children have used a more controlled and considered baseline than ordinary studies. When empathy was studied between children and their mothers in distressful situations, the investigators (Ebisch et al., 2012) made sure that the child would first feel safe and comfortable with the experimenter. This was achieved initially in the presence of the child's caretaker, where the experimenter started neutrally interacting with the child, allowing enough time for the child to psychoemotionally familiarize with the stranger. Then, baseline recordings were taken in the absence of the mother, while the experimenter continued playing with the child. The mother was able to observe the child behind a mirror glass. In order to induce distress and measure the thermal facial synchrony of the mother and the child, a toy, which was preplanned to break in the child's hands, was used as a stressor. Unaware of the situation, both participants showed simultaneous temperature decreases on the maxillary and nasal regions of the face. Once the mishap was restored during the soothing phase of the experimental session, both maxillary and nasal regions increased in heat (see also Figure 1 in Ebisch et al., 2012, p. 4). An extension of the above study including an additional group of female participants showed that mother–child dyads in contrast to other women–child dyads have faster empathic reactions to the child's emotional state (Manini et al., 2013).

Guilt

The above research paradigms used an experiment inducing guilt, further explained in Ioannou et al. (2013), with a particular focus on the sympathetic relevance of the nose tip for each individual child. Although the group analyses in the above study yielded a relatively clean effect for the mishap and soothing phase, at the individual level physiological responses throughout conditions were not always in the same direction. Significantly, different responses were observed in 8 out of 15 children during playing, with 2 showing an increase and 6 a decrease. Mishap, on the other hand, was much more homogeneous, with 12 out of 15 children showing a dip in temperature. The entrance of the experimenter was not marked with the same group tendency, as 4 out of 15 children reached a significant temperature decrease and 7 an increase. Finally, during the soothing phase, 12 children showed a rise in temperature compared with previous conditions (see also Figure 1 in Ioannou et al., 2013, p. 6).

Interpersonal Contact and Sexual Arousal

Sexual arousal has been characterized by sympathetic influences. To establish a direction of the thermal print while participants viewed an erotic movie for 5 min, researchers used a sport movie as a reference point for temperature comparison (Merla & Romani, 2007). During stimulation, the temperature of the forehead, lips, and periorbital regions increased. In another study, Hahn et al. (2012) using interpersonal physical contact through a handheld light-flashing device to examine social contact and sexual arousal. As a baseline, an acclimatization period was used in which participants viewed a series of emotionally neutral faces. Physical contact through the handheld device was performed on different parts of the body such as the face, chest (high intimate), arm and palm (low intimate) from both male and female experimenters. It was observed that the temperature of the face increased. Specifically, it was observed that when high-intimate regions were touched temperature increase was higher, reaching even higher levels when that act was performed by the opposite sex. This temperature increase was mainly localized on the mouth, nose, and the periorbital regions of the face.

Embarrassment

Providing the only study for embarrassment, Merla and Romani (2007) had participants perform a Stroop task in front of unknown people. The study was designed in order to elicit feelings of embarrassment and mild stress when the participants performed the task incorrectly in the presence of others. The prestimulation period was used as the baseline. Temperature decreases were observed on the palm, the face, and especially around the mouth due to emotional sweating. Thermal data were grounded with GSR recordings.

Joyful Expressions

Laughter is a crucial part of social interaction, and it carries its own autonomic print. Researchers from Japan studied joyful expressions in infants across three age groups (2–3, 4–6, 8–10 months old) in which laughter was elicited during playing with a stranger. For baseline measures, the prestimulation period was used, prior to the onset of laughter (Nakanishi & Imai-Matsumura, 2008). All age groups showed a temperature decrease on the nose compared with the forehead and cheeks, which remained rather stable. This change was more evident in 4–10-month-old infants. The nose temperatures of the two older aged groups (4–6, 8–10-month-old) significantly differed from 2–3-month-old infants. The nose temperature of a 4–6-month-old infant started increasing 15–30 s after laughter onset but then decreased after 1 min. In addition, five out of ten 4–6-month-olds had a decrease greater than 0.5°C after 1 min, and from those five, three were more than 1°C. Five out of six 8–10-month-old infants had decreases in nose temperature within 15 s. On the forehead, temperature change across individuals was more homogeneous as all age groups showed decreases of less than 0.5°C 2 min after laughter onset. On average, the forehead temperature dropped for 2–3-month-old infants 0.5°C, whereas for the 4-6 and 8–10 infant group it was 0.1 ± 0.33°C for the former and 0.3 ± 0.24°C for the latter. Temperature on the cheeks of 2–3-month-olds was less than 0.5°C 2 min after laughter onset, whereas on the older age groups there was a large individual variability. While an infant of 4–6 months old showed an increase of more than 0.5°C, another showed a decrease of 1°C. Overall, in the 4–6 age group, there was a 0.2 ± 0.40°C decrease after 2 min. Finally, the cheek temperature of three out of six 8–10-month-old infants showed a change of less than 0.5°C during the 2-min period. All 8–10-month-old infants had an average increase of 0.1 ± 0.45°C after 2 min.

Lie Detection and Deception-Anxiety

Due to the strong nature of fITI in reading emotional states, a variety of studies have examined this technique in the context of lie detection for home security purposes. Pavlidis et al. (2002), in order to assess the reliability of fITI in identifying thermal patterns of deceit, devised an experiment in which participants had to stab a dummy, steal $20, and then assert their innocence about the crime. The prestimulation period was used as the baseline, and temperature was measured prior to the participants claiming their innocence to the critical question, “Did you steal the $20?” Thermal imaging managed to accurately identify 11 out of 12 subjects as guilty through the increase of cutaneous perfusion in the periorbital region, while temperature increases were observed on the forehead and in regions surrounding the eyes. Following the same experimental approach, with an additional number of participants, other studies on the topic focused on particular regions of the face. Tsiamyrtzis et al. (2006) suggested that temperature monitoring of the periorbital vessel during interrogation provides 87.2% accuracy in detecting deceptive individuals. In addition, Zhu et al. (2007), by focusing on the forehead and particularly on the corrugator muscle supplied by supraorbital vessels, gave 76.3% accuracy for lie detection. Temperature increases for the above studies were accounted for by increased blood perfusion to facial muscles as a result of mental stress and fight or flight response.

Discussion

Benefits

Kistler, Mariauzouls, Link, and von Berlepsch (1997) postulates that “acquiring temperature measurements in psychophysiological studies offers the advantage of simplicity of performance and analyses, compared with more complex analyses of flux and pulse volume” (p. 35). Over the years, the sensitivity and resolution of the thermal camera has dramatically improved (Vianna & Carrive, 2005). Physiological events that would otherwise be invisible to the naked eye, such as activation of perspiration pores, can now be observed and documented (Pavlidis et al., 2012). Infrared cameras allow reliable, wireless recordings to take place, from a distance, without interfering with the subject's behavior (Anbar, 2002; Kastberger & Stachl, 2003; Head & Elliott, 2002; Vianna & Carrive, 2005), or the experimental procedure (Pavlidis et al., 2012). In addition, fITI allows recordings in an ecological experimental setting despite the participants' movements (Ebisch et al., 2012; Ioannou et al., 2013; Manini et al., 2013; Nakanishi & Matsmura, 2008). Furthermore, whereas GSR electrolytes can change their conductivity over time (Levenson, 1988), occur spontaneously during baseline recordings (Laine, Spitler, Mosher, & Gothard, 2009) or during arm movements towards the rewarded stimuli (Amiez, Procyk, Honore, Sequeira, & Joseph 2003), fITI provides the same physiological recording efficiency reflecting accurately the autonomic nature of the psychological phenomenon throughout the experimental procedure. Moreover, GSRs are so sensitive to stimuli of emotional value reaching maximum levels of activity in conditions of variable degrees of intensity, making them indistinguishable from each other, an obstacle that fITI overcomes (Kuraoka & Nakamura, 2011). Kreibig (2010) stated that “investigations of ANS responding in emotion have long been impeded by the exclusive use of ‘convenience measures,’ such as HR and electrodermal activity, as sole indicators of the activation state of the organism” (as cited in Kreibig, 2010, p. 29). Thermal imaging provides a new tool for psychophysiological monitoring, especially valuable in individuals that cannot express their emotional state verbally such as infants (Mizukami et al., 1990; Nakanishi & Imai-Matsumura, 2008) and for individual that cannot give subjective measures due to their physical state such as paralyzed patients, patients with locked-in syndrome, or patients in a coma. Given the right experimental paradigm, fITI can provide novel avenues in the arena of biofeedback, clinical diagnostics, psychopharmacological assessment, as well as the efficacy of psychotherapeutic treatments in several types of developmental, social, personality, and psychotic disorders.

Limitations

Despite the advantages offered by fITI, thermal signal development as a result of vascular change, perspiration, or muscular activity is rather slow. Kuraoka and Nakamura (2011) stress the fact that fITI has a longer latency than SCR. Specifically, the earliest that temperature changes can be observed is within 10 s compared with SCR, where changes in signal are evident in 3 s. In addition, most observations with fITI have been based on the nose; however, controversy exists as to what might be causing the observed temperature change. Studies that addressed this phenomenon of nasal temperature decrease observed that, even when respiration rate changed to two fold as a result of heavy breathing, no temperature change was observed on the nose of the subjects (Nakayama et al., 2005). On the contrary, Pavlidis et al. (2001) observed that noses of participants during leisure walking got colder and ascribed the effect to more active breathing patterns. While the temperature dip of the nose has so far been ascribed to the presence of arteriovenous anastomoses controlled by sympathetic nerves that regulate blood flow to the surface of the skin, the above scientific literature causes certain misconceptions. So far, it is difficult to exclude the fact that nasal temperature observations are not based on airflow patterns or blood flow associated to muscular activity (Nakanishi & Imai-Matsumura, 2008). Nevertheless, despite the origin of the phenomenon, nose temperature changes have so far been shown to respond to stimuli of affective nature. Finally fITI is a quite expensive physiological tool for monitoring the peripheral nervous system, which is the primary reason that makes its prevalence among studies scarce.

Future Directions

For the future of thermal imaging, it is important to develop a consistent methodology as well as a user-friendly software that will be used for temperature extraction and be freely available to all users. This will overcome problems across studies related to temperature extraction, such as the size of ROI, where the ROI should be placed, whether the maximum, average, or minimum temperature should be taken into account for data analysis, and whether observations should be based on a number of active voxels. Some progress has been made on the above matter in terms of software development (Dowdall et al., 2006; Merla, Di Donato, & Romani, 2002). However, literature is lacking on a methodological protocol that would be followed by the majority of fITI studies. Finally, it would be important to couple thermal observations to brain regions, fulfilling a bidirectional union between the brain and the visceral organs (Kuraoka & Nakamura, 2011). Stimulation of the central nucleus (Bagshaw & Coppock, 1968; Laine et al., 2009) as well as the corticomedial nucleus (Potegal, Hebert, DeCoster, & Meyerhoff, 1996) of the amygdala provide good cortical candidates for observing thermal changes associated with negative emotional states. Electroencephalography can also be integrated with thermal imaging since it does not occlude the participant's face, thus allowing the associations of subcortical emotion-related regions with the thermal print.

Conclusions

Functional ITI seems to provide new avenues for the study of emotions. The majority of studies have observed that, on occasions of sympathetic activation, the temperature of the nose decreased, attributing this effect to vasoconstrictive mechanisms restricting the blood flow to the surface of the skin. Similar decreases in temperature have been observed on the upper lip or maxillary area of the face as a result of sweat gland activation. On the contrary, temperature increases have been observed on the forehead as well as in the region between the eyes and the nose. Muscular activity and increased blood flow by supraorbital and periorbital vessels, respectively, are responsible for the observed temperature changes. Cooling of the cheeks has also been observed during the above phenomenon, suggested by researchers to be caused by the adrenergic system and partially by the redirection of blood to facial regions of increased importance. Moving from human to animal studies, profound decreases of tail temperature have also been observed in rats under conditioned fear. Overall, the above scientific evidence shows a new wireless, ecological methodology for quantifying emotional arousal and differentiating between rest and affective states. Although still in infancy, fITI provides alternative ways for addressing questions regarding ANS function in response to emotions that have so far been unexplored.