The Pupil Dilation Response to Auditory Stimuli: Current State of Knowledge.

The measurement of cognitive resource allocation during listening, or listening effort, provides valuable insight in the factors influencing auditory processing. In recent years, many studies inside and outside the field of hearing science have measured the pupil response evoked by auditory stimuli. The aim of the current review was to provide an exhaustive overview of these studies. The 146 studies included in this review originated from multiple domains, including hearing science and linguistics, but the review also covers research into motivation, memory, and emotion. The present review provides a unique overview of these studies and is organized according to the components of the Framework for Understanding Effortful Listening. A summary table presents the sample characteristics, an outline of the study design, stimuli, the pupil parameters analyzed, and the main findings of each study. The results indicate that the pupil response is sensitive to various task manipulations as well as interindividual differences. Many of the findings have been replicated. Frequent interactions between the independent factors affecting the pupil response have been reported, which indicates complex processes underlying cognitive resource allocation. This complexity should be taken into account in future studies that should focus more on interindividual differences, also including older participants. This review facilitates the careful design of new studies by indicating the factors that should be controlled for. In conclusion, measuring the pupil dilation response to auditory stimuli has been demonstrated to be sensitive method applicable to numerous research questions. The sensitivity of the measure calls for carefully designed stimuli.

Introduction

Research in the field of audiology has been increasingly focusing on the assessment of listening effort next to the measurement of perception performance (Ohlenforst et al., 2017a; Peelle, 2018; Pichora-Fuller et al., 2016; Strauss & Francis, 2017). The assessment of listening effort provides an additional dimension to evaluate auditory stimulus perception. Like speech perception performance, listening effort is sensitive to task demands and motivation (Peelle et al., 2017), and both outcomes can complement each other. In the recently developed Framework for Understanding Effortful Listening (FUEL; Pichora-Fuller et al., 2016), listening effort is defined as the deliberate allocation of resources (means available) to overcome obstacles (factors that make task completion more difficult) in goal pursuit when carrying out a listening task. Effort allocation depends on motivation: the energization of behavior directed toward positive stimuli (or behavior directed away from negative stimuli, ).

FUEL

The FUEL is an adaptation of Kahneman’s Capacity Model for Attention (Kahneman, 1973). The core of the Capacity Model adopted by the FUEL consists of available cognitive capacity that varies with arousal, the allocation policy, and the possible activities to which capacity is allocated (Pichora-Fuller et al., 2016). The allocation policy can be influenced by automatic (or involuntary) attention (e.g., by sudden sounds), intentional (voluntary) attention (e.g., by the instruction to attend to a certain stimulus), and the general arousal level. This arousal level is affected by input-related demands. The evaluation of demands on capacity component reflects an individual’s evaluation of the expected benefits of successful performance relative to the effort required to achieve that performance (Pichora-Fuller et al., 2016). The framework describes that fatigue can affect the evaluation of demands, and motivation can influence intentional or voluntary attention. This is also outlined by the Motivational Intensity Theory (Richter, Gendolla, & Wright, 2016), which identifies motivation as a factor moderating the link between task demand (the cognitive and perceptual resources needed to complete the listening task) and effort (the allocation of cognitive resources during the task). The result of this evaluation process feeds into the allocation policy such that capacity allocation depends on the current balance between success importance and the effort required to achieve success. Displeasure, fatigue, or low motivation may result in task disengagement even when the available capacity is sufficient for performing the current task (Peelle et al., 2017; Pichora-Fuller et al., 2016; Richter et al., 2016).

Measures Sensitive to Cognitive Resource Allocation: The Pupil Dilation Response

There are various ways to measure listening effort or cognitive resource allocation in hearing science (Ohlenforst et al., 2017a). In his seminal work, Kahneman (1973) proposed three criteria for any physiological indicator of processing load (i.e., allocation of resources or listening effort): (a) The measure has to be sensitive to within-task variations in processing load, (b) the measure should be sensitive to between-task variations in processing load, and (c) it should be sensitive to individual differences in processing load. The current review focuses on the task-evoked pupil dilation response. This response fulfills these three criteria when measured in tasks manipulating memory load (Kahneman & Beatty, 1966), but also when assessed in the field of hearing science (Kramer et al., 2012; Kramer, Kapteyn, Festen, & Kuik, 1997). The pupil size has been shown to be sensitive to changes in cognitive resource allocation in a wide variety of tasks presenting auditory stimuli (for overviews, see Beatty & Lucero-Wagoner, 2000; Hepach & Westermann, 2016; Kahneman & Beatty, 1966; Laeng, Sirois, & Gredebäck, 2012; Schmidtke, 2017; Sirois & Brisson, 2014). We refer to the task-evoked pupil response as the phasic change in pupil diameter (Aston-Jones & Cohen, 2005) in response to a momentary event such as an auditory stimulus presented in a listening task. The sensitivity of the measure is one of the reasons why it recently has gained popularity (Kramer et al., 1997; Ohlenforst et al., 2017a; Piquado, Isaacowitz, & Wingfield, 2010; Zekveld, Kramer, & Festen, 2010).

Physiologically, the size of the pupil is controlled by two muscles: the sphincter (constrictor) muscles reducing the size of the pupil and the dilator muscles increasing the pupil size. The balance between the sympathetic and parasympathetic nervous systems determines the pupil size (Loewenfeld & Lowenstein, 1993; Wang et al., 2018b). The mechanisms underlying the parasympathetic and sympathetic contribution to the pupil size are complex—for example, the relative contribution of the two systems depends on the illumination level, cognitive activity, and fatigue (Steinhauer, Siegle, Condray, & Pless, 2004; Wang et al., 2018a, 2018b). The pupil dilation response to arousal and mental resource allocation has been associated with the projections from the noradrenergic locus coeruleus to the parasympathetic Edinger–Westphal nucleus. Locus coeruleus activity results in inhibition of the Edinger–Westphal nucleus, and this leads to inhibition of the constricting muscle of the pupil, thereby resulting in pupil dilation (Eckstein, Guerra-Carrillo, Singley, & Bunge, 2017; Wang et al., 2016). In general, pupil size increases with increasing task demands as long as the individual stays engaged in the task (Granholm, Asarnow, Sarkin, & Dykes, 1996), and the pupil constricts in response to increasing illumination level (pupil light reflex). The resting state or baseline pupil size has been used as index of the baseline arousal level or task engagement (Aston-Jones & Cohen, 2005; Hopstaken, van der Linden, Bakker, & Kompier, 2015) and is associated with tonic levels of norepinephrine activity (Aston-Jones & Cohen, 2005; Jepma & Nieuwenhuis, 2011; Murphy, O'connell, O'sullivan, Robertson, & Balsters, 2014).

Resource allocation depends on input-related demands (factors that make task completion more difficult), attentional processes, motivation, and fatigue (Pichora-Fuller et al., 2016). Factors affecting the input-related demands include, for example, intelligibility and linguistic complexity. In addition, internal factors such as hearing loss will affect the task demands. These various factors may each affect different types of processes (e.g., linguistic processing, auditory processing, memory processing, response preparation processes) and thereby influence cognitive resource allocation. We assume that the pupil dilation response to auditory stimuli can be influenced by multiple processes that all influence the allocation of cognitive resources. Hence, when measuring the pupil dilation response, it may be important to design the study such that the task manipulation or the between-subject factor affects the process of interest, without affecting processes that are not within the primary focus of the study, but may confound the results when also affecting cognitive resource allocation. For example, Liao, Kidani, Yoneya, Kashino, and Furukawa (2016) recently observed that the pupil diameter was larger for sounds that were rated as more salient, louder, more vigorous, more annoying, less preferred, and less beautiful. However, when controlling for differences in loudness between the stimuli, these relationships disappeared. A follow-up experiment indicated that the pupil diameter was associated with the subjective saliency of sounds, which predominantly appeared to be defined by the loudness feature of the auditory stimuli.

Considering the increasing popularity of the method of pupillometry, it is time to carefully review the existing literature addressing the pupil dilation to auditory stimuli. We applied a broad inclusive approach, as many studies in other fields of research can be relevant to hearing science—for example, manipulating sentence length may affect both linguistic and memory processes. Knowledge of the processes that possibly affect the pupil dilation response is highly needed as we need to consider what confounding factors should be controlled for or factored into an individual differences approach to the study of (listening) effort (see Pichora-Fuller et al., 2016). The knowledge acquired through this review will facilitate the cautious design of studies that assess the specific contribution of the factors under study. The current literature review therefore aimed at providing an exhaustive overview of the current knowledge regarding the pupil dilation response to auditory stimuli.

Methods

The results summarized in the following were adopted from English peer-reviewed articles from inception to December 2017 that applied auditory stimuli and simultaneously recorded the pupil size or pupil dilation response in humans. The articles were identified by using relevant search terms (see Appendix) in PubMed and by checking references to relevant articles in the included articles. Excluded articles were those reporting case studies and studies that did not focus on measuring the task-evoked pupil dilation response to auditory stimuli. For example, drug studies were excluded, as well as studies primarily focusing on the pupil dilation response to visual stimuli. The evidence presented in this article is not meant to be conclusive. The information extraction was performed by the first author (AAZ). Authors TK and SEK checked the information added in the tables for a randomly identified 10% of the articles (N = 14).

Results

The PubMed search identified 342 articles. Of these, 162 articles were unique. Another 69 articles were identified by checking the references in the included articles. Figure 1 shows the number of records that were checked, included, and excluded.

Figure 1.

Overview of records that were identified, screened, excluded, and included in the review. The reasons for excluding the full-text articles are outlined in the text.

Based on the title, abstract, and full text of the articles, 146 articles were included in the current review. Of the 85 excluded articles, 26 studies did not focus on (i.e., measure, analyze, formally test) the measurement of the pupil response to an auditory stimulus, 14 articles described animal studies, 11 articles described case studies, 10 articles described drug studies, 7 studies did not measure the pupil response, 6 articles were reviews, 4 articles were not written in English, 6 studies did include formal statistical tests, and 1 article was not peer-reviewed. Please note that some of the excluded articles included early studies that were rejected because of the lack of formal statistical tests (e.g., Berrien & Huntington, 1943; Bradshaw, 1969, 1970; Hess & Polt, 1964; Schaefer, Ferguson, Klein, & Rawson, 1968).

Figure 2 shows the number of publications as function of publication year. A clear increase in the number of publications over the years can be observed.

Figure 2.

Number of publications included in the present review as function of publication year. In total, 146 publications were included.

Tables 1 and 2 and Supplementary Table 1 provide an overview of the main results of the studies included. We included results only if these were tested with formal statistical tests. Please note that many studies reported interactions between the independent factors influencing the pupil dilation response. These interactions are indicated (but not described) in summary Tables 1 and 2. A summary of each study can be found in Supplementary Table 1 that also briefly includes details about the sample size, stimuli and tasks presented, the main findings, and the pupil measure analyzed. Depending on the design of the study (i.e., between-subjects or within-subjects factor manipulations), some of the factors that are categorized in Table 1 as external factor are included in Table 2 as individual factor if a between-subjects design was employed. This was, for example, the case for the effect of guilt or deception on the pupil size (e.g., Dionisio, Granholm, Hillix, & Perrine, 2001; Webb, Honts, Kircher, Bernhardt, & Cook, 2009).

Table 1.

Overview of Results of the External Factors Influencing the Pupil Dilation Response (PDR) to Auditory Stimuli.

	Confirmed	No effect	Opposite effect
Auditory stimulus presentation is associated with larger pupil size/PDR
Cajal, 2011; Wang et al., 2017	2
Increasing sound level is associated with larger pupil size/PDR
Antikainen & Niemi, 1983a; Jones et al., 1977; Nunnally et al., 1967	3
Unpredictable/infrequent events are associated with larger pupil size/PDR
Damsma & Van Rijn, 2017; Friedman et al., 1973; Gilzenrat et al., 2010; Hoffing & Seitz, 2015; Knapen et al., 2016; Korn & Bach, 2016; Marois et al., 2018; Preuschoff et al., 2011; Qiyuan et al., 1985b; Steiner & Barry, 2011; Steinhauer & Zubin, 1982; Wetzel et al., 2016a	12
Stanners & Headley, 1972; Stelmack & Siddle, 1982		2
Poorer performance is associated with larger pupil size/PDR
Miles et al., 2017; Zekveld et al., 2010	2
Tsai, Viirre, Strychacz, Chase, & Jung, 2007		1
Larger baseline pupil size is associated with poorer performance
Gilzenrat et al., 2010; Murphy et al., 2011	2
More complex auditory processing is associated with larger pupil size/PDR
Bianchi et al., 2016a; Dlugosch et al., 2013; Fletcher et al., 2016a; Kahneman & Beatty, 1967; Kramer et al., 2012; Kun et al., 2013; Liao et al., 2016; Palinko et al., 2010; Recarte & Nunes, 2003; Schlemmer et al., 2005	10
Ambler et al., 1976; Einhäuser et al., 2008; Hoeks & Levelt, 1993; Lisi et al., 2015		4
Increased degradation level is associated with larger pupil size/PDR (up till resource overload)
Koelewijn et al., 2012a, 2012b, 2014a, 2014b, 2015; Kramer et al., 1997a, 2016[a]; Kuchinsky et al., 2013; McGarrigle et al., 2017b; McMahon et al., 2016b; Miles et al., 2017; Tamási et al., 2017; Wagner et al., 2016b; Wendt et al., 2016, 2017; Winn, 2016; Winn et al., 2015; Zekveld et al., 2010, 2011[a], 2013, 2014a; Zekveld & Kramer, 2014	22
Koch & Janse, 2016; Miles et al., 2017		2
McGarrigle et al., 2017a			1
Degradation type: Informational maskers are associated with larger PDR
Koelewijn et al., 2012a, 2012b, 2014a, 2014b; Zekveld et al., 2014a, 2014b	6
Ohlenforst et al., 2017b		1
Increasing linguistic complexity/linguistic processing is associated with larger pupil size/PDR
Ahern & Beatty, 1981; Ambler et al., 1976; Ben-Nun, 1986; Byers-Heinlein et al., 2017a,b; Chapman & Hallowell, 2015; Demberg, 2013b; Demberg & Sayeed, 2016b; Elshtain & Schaeffer, 1968; Engelhardt et al., 2010; Hochmann & Papeo, 2014; Hyönä et al., 1995b; Koch & Janse, 2016; Kramer et al., 2012; Kruger et al., 2013; Kuchinsky et al., 2013; Kuipers & Thierry, 2011, 2013[a]; Ledoux et al., 2016; Piquado et al., 2010a; Scheepers et al., 2013b; Schluroff, 1982; Schmidtke, 2014a; Stanners & Headley, 1972b; Tromp et al., 2016b; Vogelzang et al., 2016; Wagner et al., 2016b; Wendt et al., 2016b; Winn, 2016; Wright & Kahneman, 1971b; Zellin et al., 2011b	30
Carver, 1971		1
Higher memory load is associated with larger pupil size/PDR (up till resource overload)
Ahern & Beatty, 1981a; Bijleveld et al., 2009; Cabestrero et al., 2009; Elshtain & Schaeffer, 1968b; Gardner et al., 1975; Granholm et al., 1996b, 1997[a]; Johnson et al., 2014a; Kahneman et al., 1968, 1969; Kahneman & Beatty, 1966; Kahneman & Wright, 1971b; Karatekin, 2004; Klingner et al., 2011; Peavler, 1974; Piquado et al., 2010	16
Ambler et al., 1976; Kahneman et al., 1967		2
Larger memory processing demands are associated with larger pupil size/PDR
Johnson, 1971; Kahneman & Beatty, 1966; Koelewijn et al., 2017; Otero et al., 2011; Papesh et al., 2012; Peavler, 1974; Stanners et al., 1979; Wong & Epps, 2016	8
Kahneman & Wright, 1971b			1
More complex mental problems are associated with larger pupil size/PDR
Ahern & Beatty, 1979, 1981; Klingner et al., 2011; Marshall, 2007	4
Increased attentional demands/uncertainty are associated with larger pupil size/PDR
Ambler et al., 1976; Fish & Granholm, 2008; Kang & Wheatley, 2015; Klingner et al., 2011; Koelewijn et al., 2015b, 2017; Laeng et al., 2016b; Lempert et al., 2015b; McCloy et al., 2016, 2017	10
Kun et al., 2013; Zekveld et al., 2014a		2
Increased motivation/reward/threat is associated with larger pupil size/PDR
Bijleveld et al., 2009b; Kluge et al., 2011; Knapen et al., 2016; Korn et al., 2017	4
Stanners et al., 1979		1
Emotional valence and engagement is associated with larger pupil size/PDR
Babiker et al., 2015; Burley et al., 2017; Chaney et al., 1989; Dabbs, 1997a,b; Fletcher et al., 2015a; Gilzenrat et al., 2010; Gingras et al., 2015a; Jin et al., 2015; Kang & Wheatley, 2017; Laeng et al., 2016b; Partala & Surakka, 2003; Stanners et al., 1979; Weiss et al., 2016; White & Maltzman, 1978	14
Horrey et al., 2017; Rosa et al., 2017		2
Guilt/deception is associated with larger pupil size/PDR
Dionisio et al., 2001b; Bradley & Janisse, 1979	2
Word meaning influences pupil size/PDR
Mathôt et al., 2017	1
Response preparation is associated with larger pupil size/PDR
Beatty, 1982; D’Ascenzo et al., 2018b; Kahneman & Beatty, 1967; McCloy et al., 2016; Simpson, 1969	5
At the beginning of a block/test session, larger pupil sizes are observed (habituation/fatigue effects)
Ambler et al., 1976b; Antikainen & Niemi, 1983; Beatty, 1982; Dahlman et al., 2009; Damsma & Van Rijn, 2017; Demberg, 2013b; Demberg & Sayeed, 2016b; Fletcher et al., 2015a,b; Hyönä et al., 1995; Kahneman & Beatty, 1967b; Koch & Janse, 2016; Marois et al., 2018; McGarrigle et al., 2017a; Murphy et al., 2011; Nunnally et al., 1967; Stanners and Headley, 1972b; Steiner & Barry, 2011; Stelmack & Siddle, 1982; Zekveld et al., 2010	19
Fletcher et al., 2015; Kluge et al., 2011; McGarrigle et al., 2017b; Schlemmer et al., 2005		4

Note. Please note that some of the listed effects interacted with other factors; these interactions are indicated by asterisks. We cited and counted the number of studies confirming the stated hypothesis, the number of studies that did not observe any effect of the factors studied, and the number of studies finding an opposite effect. Empty cells reflect 0 findings.

Interaction with listener-related factor. bInteraction with external factor.

Table 2.

Overview of Results of the Individual Factors Influencing the Pupil Dilation Response (PDR) to Auditory Stimuli.

	Confirmed	No effect	Opposite effect
Increasing ages relates to smaller pupil size or smaller pupil size/PDR
Karatekin, 2004; Koch & Janse, 2016**; Morris et al., 1997; Steel et al., 2015**; Wetzel et al., 2016*; Zekveld et al., 2011	6
Ayasse et al., 2017; Chaney et al., 1989; Koelewijn et al., 2012a; Kuchinsky et al., 2016		4
Johnson et al., 2014*; Piquado et al., 2010			2
Nonnative listeners have larger pupil size/PDR
Schmidtke, 2014*	1
More severe hearing loss is associated with reduced pupil size/PDR
Kramer et al., 2016*; Kuchinsky et al., 2014; Ohlenforst et al., 2017b*; Wang et al., 2018a	4
Koelewijn et al., 2017; Kuchinsky et al., 2016		2
Ayasse et al., 2017; Kitajima et al., 2010; Steel et al., 2015; Winn, 2016*			4
Better cognitive abilities are associated with larger pupil size/PDR
Koelewijn et al., 2012b; Kuchinsky et al., 2016; Wendt et al., 2016; Zekveld et al., 2011; Zekveld & Kramer, 2014	5
Koelewijn et al., 2014b; Zekveld et al., 2013, 2014b		3
Ahern & Beatty, 1979, 1981*; Koch & Janse, 2016; Wendt et al., 2017			4
Higher musical expertise is associated with larger pupil size/PDR
Damsma & Van Rijn, 2017		1
Bianchi et al., 2016*; Schlemmer et al., 2005			1
Females have larger pupil size/PDR than males
Dabbs, 1997*,**; Partala & Surakka, 2003	2
Burley et al., 2017		1
Gingras et al., 2015			1
Higher level of fatigue is associated with smaller pupil size/PDR
Wang et al., 2018a	1
Auditory training is associated with larger PDR
Kuchinsky et al., 2014*	1
Vestibular neuritis is associated with altered pupil size/PDR
Kitajima et al., 2013	1
Schizophrenia is related to smaller PDR
Fish & Granholm, 2008; Granholm et al., 1997*; Morris et al., 1997	3
Dementia is associated with altered PDR
Fletcher et al., 2015, 2016	2
Depression is associated with altered pupil size/PDR
Jin et al., 2015		1
Psychopathy is associated with pupil size/PDR
Burley et al., 2017		1
Fear is associated with altered pupil size/PDR
Stanners et al., 1979		1
Guilt/deception is associated with altered pupil size/PDR
Bradley & Janisse, 1981; Webb et al., 2009*	2
Neuroticism is associated with larger pupil size/PDR
Antikainen & Niemi, 1983*	1
Aphasia is associated with larger pupil size/PDR
Chapman & Hallowell, 2015		1
Higher level of motion sickness is related to increased pupil size/PDR
Dahlman et al., 2009		1

Note. Please note that some of the listed effects interacted with other factors; these interactions are indicated by asterisks. We cited and counted the number of studies confirming the stated hypothesis, the number of studies that did not observe any effect of the factors studied, and the number of studies finding an opposite effect. Empty cells reflect 0 findings.

Interaction with external factor. **Interaction with listener-related factor.

Many different approaches to the preprocessing of the pupil signal have been adopted (see Winn et al., current Special Issue). Also, a range of parameters extracted from the pupil signal have been used as outcome, and different analysis techniques have been applied across studies. For example, in several studies, a scaling method has been applied correcting the pupil dilation response for the dynamic range of the pupil size (e.g., Ayasse, Lash, & Wingfield, 2017; Mathôt, Grainger, & Strijkers, 2017; Preuschoff, Marius’t Hart, & Einhäuser, 2011). Apart from the preprocessing approach adopted, the parameters extracted from the pupil signal also differed between studies. Examples are the absolute pupil size and the peak response relative to baseline. However, the results of the current review indicate a general consensus (in line with Beatty, 1982) that, independent from the analysis method, the larger the task-evoked pupil dilation response, the higher the cognitive resource allocation. As the main aim of the current review was to describe the factors influencing the general pupil dilation response to auditory stimuli, we do not compare the specific parameters analyzed in the reviewed studies. In Supplementary Table 1, we briefly state the preprocessing approach (e.g., baseline correction, normalization) adopted for each of the studies included in the review.

In the following, we summarize the findings of the identified articles. We grouped the results according to whether these were based on external (see Table 1) versus internal (individual) factors (Table 2) and furthermore used the FUEL categories to structure the results (Pichora-Fuller et al., 2016).

Automatic Attention and the Task-Evoked Pupil Dilation Response

Automatic (or involuntary) attention includes the responses to novel or sudden stimuli. Several studies assessed whether stimuli evoking such automatic attentional effects influence the pupil size by manipulating the presence of auditory stimuli or the loudness of these stimuli. In an early study, Nunnally, Knott, Duchnowski, and Parker (1967) showed that increasingly louder tones were associated with larger pupil sizes (see also Antikainen & Niemi, 1983; Jones, Loeb, & Cohen, 1977), especially for sound levels above 90 dB (Nunnally et al., 1967). Furthermore, Nunnally et al. already indicated the strong order effect on the pupil response (see the following for more evidence)—tones presented in the second half of the sequence resulted in smaller pupil sizes than tones with the same loudness presented in the first half of the sequence. Cajal (2011) was furthermore able to assess pupillary responses to vibro-acoustic stimulation in human fetuses.

The pupil dilation response is sensitive to factors evoking an orienting reflex, with larger pupil sizes for deviant tones when presented in a sequence of standard (more frequent) tones (Friedman, Hakerem, Sutton, & Fleiss, 1973; Gilzenrat, Nieuwenhuis, Jepma, & Cohen, 2010; Hong, Walz, & Sajda, 2014; Korn & Bach, 2016; Marois, Labonté, Parent, & Vachon, 2018; Steiner & Barry, 2011; Wetzel, Buttelmann, Schieler, & Widmann, 2016). This effect was associated with electroencephalographic components (Hong et al., 2014) and even observed for deviant “stimuli” consisting of omitted tones (Qiyuan, Richer, Wagoner, & Beatty, 1985) or beats (Damsma & Van Rijn, 2017) when these are expected by the listener. Interestingly, smaller baseline pupil sizes have been associated with larger pupil dilation relative to baseline and better detection performances during such auditory oddball tasks (Gilzenrat et al., 2010). Furthermore, the pupil dilation response also signals surprise effects in more complex tasks, such as auditory gambling games (Preuschoff et al., 2011). Event probability and uncertainty influences pupil size, with largest changes in pupil size for rare events, smallest dilation for frequent and predictable tones, and intermediate pupil dilation to unpredictable but frequent tones (see Steinhauer & Zubin, 1982, but see Stanners & Headley, 1972; Stelmack & Siddle, 1982 for absent effects of deviant stimuli).

In a study on the basic response to auditory stimuli such as sounds, tones, and noise-bursts, Liao et al. (2016) observed that stimuli that are experienced as more loudly are associated with larger pupil diameters. However, presenting two tones versus a single tone did not affect the pupil dilation response (Hoeks & Levelt, 1993). In a study by Einhäuser, Stout, Koch, and Carter (2008), the perception of ambiguous auditory stimuli did not influence the pupil dilation response, but this may have been related to the relatively small sample size in that study. In summary, the presentation of auditory stimuli (compared with silence), deviant stimuli, or absent stimuli when a stimulus is expected evokes a pupil dilation response.

Intentional Attention and the Task-Evoked Pupil Dilation Response

Various studies assessed the effect of intentional or voluntary attention on the pupil response. For example, in an auditory vigilance test, participants monitored a sequence of ascending numbers. At predefined numbers, errors could occur. The pupil dilated relative to baseline in response to the occurrence of such errors (Klingner, Tversky, & Hanrahan, 2011). Also, actively listening evokes larger pupil sizes than passively listening to music samples (Laeng, Eidet, Sulutvedt, & Panksepp, 2016). Furthermore, Lempert, Chen, and Fleming (2015) showed that the moment a decision is made (in a choice reaction time task), the pupil size increases. Intentional attention can also be manipulated by presenting distractive stimuli. Fish and Granholm (2008) presented a digit span task in which relevant digits were spoken by a female voice. In the distraction condition, a male voice pronounced irrelevant digits in between the relevant digits. Both the performance level and the pupil dilation relative to baseline increased in the distraction condition in healthy subjects but not in patients with schizophrenia.

In several studies, attentional processing was manipulated by presenting dichotic auditory stimuli and by instructing the listener to attend to one or both ears, or to switch attention between the ears. An example of a study is Ambler, Fisicaro, and Proctor (1976). Participants were asked to listen to a string of letters presented in one ear. Distracting, irrelevant stimuli (letters, words, proses, or digits) were presented in the other ear. The nature of the distracting stimulus influenced the decrease in pupil response over the course of the test session: The decrease was smaller when distractors were from the same category as the target stimuli when compared with distractors from different categories. The nature of the unattended message can influence the ability of the attentional system to focus on the attended message (Ambler et al., 1976). McCloy, Larson, Lau, and Lee (2016) and McCloy, Lau, Larson, Pratt, and Lee (2017) observed that switching attention between two streams of speakers increases the pupil size relative to maintaining attention at one of the two streams. Furthermore, the pupil dilates more when two sentences are presented instead of one and when participants have to report two sentences instead of one (Koelewijn, Shinn-Cunningham, Zekveld, & Kramer, 2014a). Again using a dichotic sentence perception task, Koelewijn, de Kluiver, Shinn-Cunningham, Zekveld, and Kramer (2015) assessed the effect of prior knowledge about the location, voice identity, and speech onset on the pupil dilation response. Only talker location uncertainty was associated with a larger pupil dilation response relative to baseline. Finally, in an innovative study by Kang and Wheatley (2015), it was demonstrated that the pupil signal can reveal which one of two music clips listeners attend to when both are simultaneously presented dichotically.

Several researchers have applied dual-task paradigms to manipulate attentional load. Kahneman, Beatty, and Pollack (1967) presented a memory task in which a sequence of four digits was presented and participants had to add 1 to each digit when repeating them. The task was performed alone or together with a visual detection task. The pupil size was larger for the listening than for the visual detection task, but no additional effect of the dual-task condition was observed. More complex designs have been used to assess the effort associated with having a conversation while driving. In general, having a conversation while driving increases the index of cognitive activity, which is based on a wavelet analysis of the pupil signal that quantifies the high-frequency detail in the pupil signal (Demberg, 2013; Dlugosch, Conti, & Bengler, 2013). Kun, Palinko, Medenica, and Heeman (2013) measured the pupil dilation response during an actual live conversation between two participants who played a game. One of the participants operated a simulated vehicle. The pupil response of the driver increased between the start of the conversation and before the verbal response of the driver but was not influenced by the difficulty level of the driving task (curvy vs. straight roads). Palinko, Kun, Shyrokov, and Heeman (2010) also combined a driving task with conversation tasks. These task included games such as guessing a word that the conversational partner had in mind. They observed a larger pupil diameter change during the driver’s turn when compared with the turn of the conversation partner (see also Recarte & Nunes, 2003). The studies reviewed earlier suggest that in most studies, influencing intentional attention by presenting distracting stimuli or by adding a secondary task affected the pupil size during listening.

Input-Related Demands and the Task-Evoked Pupil Dilation Response

Factors influencing the input-related demands include source factors (e.g., linguistic factors), transmission factors (e.g., degradation level), message factors, context, and listener factors. Here, we first summarize the input-related or external factors. Listener factors are separately discussed afterward.

Auditory task demands

Kahneman and Beatty (1967) presented a tone-discrimination task and observed that a more difficult tone-discrimination condition resulted in increased pupil size. Gilzenrat et al. (2010) and Murphy, Robertson, Balsters, and O'connell (2011) suggested that the baseline pupil size and the pupil dilation response relative to baseline in a tone-discrimination task might reflect task engagement. Gilzenrat et al. (2010) reported a large baseline pupil size and small dilation relative to baseline when the discrimination task was impossibly difficult, suggesting that task disengagement is related to a small pupil dilation relative to baseline (see the following for related findings of studies to speech intelligibility and memory load).

By presenting auditory stimuli more diverse in their complexity, Kramer et al. (2012) observed an effect of the type of task on the pupil dilation response, with larger dilation to word identification when compared with noise detection and no-task conditions. Furthermore, auditory and audiovisual stimuli evoke a faster and larger pupil response than visual stimuli when presented in a spatial attention task (Wang, Blohm, Huang, Boehnke, & Munoz, 2017, but see Taylor, 1981). However, changing the spatial configuration of sentences masked by interfering speech did not influence the pupil dilation response in a spatial monitoring task (Lisi, Bonato, & Zorzi, 2015), or in a speech intelligibility task in which intelligibility was kept constant (Zekveld, Rudner, Kramer, Lyzenga, & Rönnberg, 2014a). In summary, manipulating the demands imposed by an auditory task can influence the pupil dilation although the evidence regarding the sensitivity of the pupil dilation response to the spatial characteristics of the task is currently mixed.

Intelligibility or stimulus degradation level

The evidence currently reviewed indicates a robust effect of intelligibility level or stimulus degradation on the pupil response to auditory stimuli. The effect of speech intelligibility has been assessed by applying speech reception threshold tests using an adaptive procedure to vary the speech-to-noise ratio (SNR) while targeting different intelligibility levels. Some studies (Koelewijn, Zekveld, Festen, & Kramer, 2012a; Koelewijn, Zekveld, Festen, Rönnberg, & Kramer, 2012b; Zekveld et al., 2010; Zekveld, Festen, & Kramer, 2013; Zekveld, Heslenfeld, Johnsrude, Versfeld, & Kramer, 2014b) have observed lager pupil dilations relative to baseline in listeners with normal hearing for lower intelligibility levels when targeting intelligibility levels between 50%, 71%, or 84% correct. Also, larger pupil dilation has been observed for incorrectly perceived sentences when compared with correctly perceived sentences with fixed SNRs (Zekveld et al., 2010). Relatively similar effects of intelligibility level or SNR have been shown in listeners with hearing loss (Koelewijn, Zekveld, Festen, & Kramer, 2014b; Kramer et al., 1997; Zekveld, Kramer, & Festen, 2011), although the pupil dilation response as function of intelligibility or SNR seems to be influenced by hearing status (for details, see later). Reliable effects of speech intelligibility have also been shown when fixing the SNR of the stimuli at various levels, in listeners with normal hearing (Koelewijn et al., 2014a, 2015; McGarrigle, Dawes, Stewart, Kuchinsky, & Munro, 2017b; Wendt, Dau, & Hjortkjær, 2016; Zekveld & Kramer, 2014) and also in older listeners (Kuchinsky et al., 2013). The application of hearing aid processing schemes that may affect intelligibility can also influence the pupil dilation response (Wendt, Hietkamp, & Lunner, 2017).

Intelligibility can be reduced by applying a masker signal (e.g., by applying an interfering-speech or noise masker). In addition, intelligibility can also be manipulated by degrading the quality of the auditory stimuli. Several studies have examined the effect of applying spectral degradation to sentences (McCloy et al., 2017; McMahon et al., 2016; Miles et al., 2017; Wagner, Toffanin, & Başkent, 2016; Winn, Edwards, & Litovsky, 2015; Zekveld et al., 2014b). Each of these studies, except McCloy et al. (2017), indicated that more severely degraded sentences resulted in lower intelligibility levels and larger pupil size changes. The intelligibility effect on pupil dilation was also observed by Winn (2016) in a group of cochlear implant users. Even in young toddlers, the pupil was found to be sensitive to differences in intelligibility caused by mispronunciations (Tamási, McKean, Gafos, Fritzsche, & Höhle, 2017). Words that included mispronunciations resulted in larger pupil dilation relative to baseline in 30-month-old children when compared with correctly pronounced words. In contrast, increasing the speech rate did not influence the pupil dilation response in young or older listeners (Koch & Janse, 2016).

Pupil dilation responses to a wide range of intelligibility conditions

In general, the studies reviewed earlier reported larger pupil dilation responses to stimuli with lower intelligibility levels. However, evidence is accumulating that at very low intelligibility levels, the pupil response can be relatively small when compared with intermediate intelligibility levels. This may indicate that listeners may give up trying to perceive the speech when it is very difficult to achieve successful performance, thereby reflecting the possible impact of the evaluation of demands on capacity mechanism as included in the FUEL (Pichora-Fuller et al., 2016). This effect may furthermore reflect the influence of low motivation on intentional attention. Zekveld and Kramer (2014) were the first to find support for the decrease in pupil dilation relative to baseline in relatively difficult speech perception conditions. In their study, the relatively small pupil dilation at a very low performance level (around 0% correct) was associated with a higher reported frequency of giving up during the listening task. This association was confirmed by Kramer, Teunissen, and Zekveld (2016) in listeners with normal hearing and listeners with hearing loss. McMahon et al. (2016) also observed that for masked speech that was noise-vocoded, the relation between SNR and pupil dilation relative to baseline was nonlinear, with plateauing pupil dilation for moderate SNRs and increasing pupil dilation for increasing SNRs. The inverse-U-shaped function of the pupil response across a wide range of intelligibility levels was recently confirmed by Ohlenforst et al. (2017b) who included both listeners with normal hearing and with hearing loss. Note that the shape of this function differed between the two groups and also depended on the masker type applied. Potentially contrasting findings have been reported by McGarrigle, Dawes, Stewart, Kuchinsky, and Munro (2017a) who assessed the negative slope of the pupil response. They showed that this slope was steeper for sentences presented at lower SNRs, even when the intelligibility level of these items was relatively high. This effect interacted with test block as it was observed only in the second half of the test session. The studies described earlier generally indicate reliable effects of intelligibility or degradation level on the pupil dilation response. Effective intelligibility manipulations included masking the speech with noise or interfering speech and degrading the target speech by applying noise-vocoding or by adding mispronunciations. The pupil response across a wide range of degradation levels likely has an inverse-U-shaped function, at least for the masker types for which such a large range of intelligibility conditions (i.e., 0%–100%) have been presented.

Masker type

Besides the degradation level applied, the type of degradation affects the pupil dilation response even when the intelligibility level is comparable between conditions. Speech masked with a single-talker masker evokes larger pupil dilation responses relative to baseline than speech masked with fluctuating noise in listeners with normal hearing and in listeners with hearing loss (Koelewijn et al., 2012a, 2012b, 2014b). Similarly, a single-talker masker evokes larger pupil dilation responses than a fluctuating noise masker or degrading the target speech by applying noise-vocoding (Zekveld et al., 2014b). Also, masking a female target speaker with female interfering speech increases the pupil dilation response relative to male interfering speech (Zekveld et al., 2014a). In contrast, Ohlenforst et al. (2017b) did not observe a main effect of masker type when comparing a stationary noise masker with a single-talker masker.

Linguistic processing

The current review identified a large number of studies that manipulated the linguistic complexity of spoken stimuli and examined the pupil dilation response.

Word processing

Larger pupil responses have been observed during the perception of semantically difficult when compared with easy words (Chapman & Hallowell, 2015). Also, lower word frequency was associated with increased pupil size when compared with high word frequency, for high storage-load conditions, as shown in a study of Elshtain and Schaeffer (1968). Increasing word frequency, removing lexical competitors, facilitating semantic processing, or more sparse neighborhood density have more recently been shown to reduce or fasten the pupil dilation response (Koch & Janse, 2016; Kuchinsky et al., 2013; Kuipers & Thierry, 2011; Wagner et al., 2016), with larger facilitating effects in monolinguals when compared with bilinguals (Kuipers & Thierry, 2013; Schmidtke, 2014). A more extreme manipulation was performed in the study of Ledoux et al. (2016) in which completely unknown words were compared with known words in a visual-world paradigm and in a congruency task. Unknown words evoked a larger pupil dilation response. Similarly, encoding nonwords resulted in larger pupil diameters than low- and high-frequency words, but only if these nonwords were recognized afterward (Papesh, Goldinger, & Hout, 2012). Training can affect linguistic processing: The pupil response to degraded words was larger and peaked faster when participants were trained when compared with a control group (Kuchinsky et al., 2014). Word meaning can influence the pupil size: The processing of words conveying brightness is associated with smaller pupil sizes than the processing of those that convey darkness (Mathôt et al., 2017). Besides the previously mentioned evidence of the sensitivity of the pupil dilation response to linguistic processing of words in adult listeners, an effect of a linguistic manipulation (presenting a series of syllables or words with or without a deviant stimulus) has been observed in babies as young as 3 and 6 months old (Hochmann & Papeo, 2014).

Sentence processing

In an early study by Stanners and Headley (1972), increasing sentence complexity was found to be related to larger pupil responses during listening. Sentences containing ambiguous phrases when compared with sentences that were unambiguous were furthermore associated with larger pupil responses (Ben-Nun, 1986). Similar findings were reported by Ahern and Beatty (1981); Kramer et al. (2012); Piquado et al. (2010); Schluroff (1982); and Wendt et al. (2016) who presented sentences differing in semantic complexity or stimulus length. Vogelzang, Hendriks, and van Rijn (2016) showed that ambiguity resolution in pronoun processing influenced the pupil response, with larger responses for pronominal versus full noun phrases. Conflicting versus cooperative prosody also influenced the pupil dilation response to sentence processing (Engelhardt, Ferreira, & Patsenko, 2010). Furthermore, pragmatic manipulations affect the pupil response (Tromp, Hagoort, & Meyer, 2016), with larger responses for indirect requests for action (e.g., a picture of a window presented together with the sentence “it is very hot here”).

Zellin, Pannekamp, Toepel, and van der Meer (2011) assessed the effect of information focus (new information vs. corrective information) and the adequacy of prosodic accents on the pupil dilation response. Corrective information with inadequate focus yielded the largest pupil dilation response. Demberg (2013) and Demberg and Sayeed (2016) showed that increasing linguistic complexity (manipulated by grammatical gender, semantic violations, or connective type) increased the index of cognitive activity, which is derived from the pupil signal using wavelet analysis. Finally, high-context sentences evoke smaller pupil dilation responses than sentences with lower context (Winn, 2016), while conflicting effects of word probability were reported by Koch and Janse (2016). Together, this body of research indicates that the pupil response is sensitive to the integration of various types of linguistic information in sentence processing.

Besides the presentation of words and sentences, longer or more complex stimuli have been presented. Semantic and syntactic violations in the last sentence of Limerick rhymes did not influence the pupil size relative to a control condition with a correct last line (Scheepers, Mohr, Fischer, & Roberts, 2013). However, a rhyme violation resulted in increased pupil dilation relative to the control condition. Hyönä, Tommola, and Alaja (1995) assessed the effect of a native versus nonnative language in several conditions in which skilled interpreters had to listen to, shadow, or translate passages or words. The pupil response was larger for nonnative speech and was also sensitive to the lexical task requirements, with largest pupil dilation for (simultaneous) translation when compared with shadowing and listening to the stimuli (see also Ambler et al., 1976). Hearing language switches is also associated with increased pupil dilation, in adults as well as in 20-month-old listeners (Byers-Heinlein, Morin-Lessard, & Lew-Williams, 2017). Finally, Kruger, Hefer, and Matthew (2013) asked students to watch a video of an English-spoken lecture with or without English subtitles and found larger changes in pupil size when no subtitles were presented. In contrast to the previously mentioned studies, Carver (1971) did not find an effect of difficulty level of auditory passages on the pupil dilation response.

Verbal (Working) Memory Load and the Pupil Dilation Response

The current review identified numerous studies that applied an auditory memory task, which may be partly based on the seminal work of Kahneman and Beatty (1966). The studies showed that the pupil size increases with increasing memory load in a digit span task. This effect of memory load on pupil size has been replicated in many studies (Cabestrero, Crespo, & Quirós, 2009; Gardner, Beltramo, & Krinsky, 1975; Granholm et al., 1996; Granholm, Morris, Sarkin, Asarnow, & Jeste, 1997; Kahneman, Tursky, Shapiro, & Crider, 1969; Karatekin, 2004; Klingner et al., 2011; Peavler, 1974; Piquado et al., 2010; Stanners, Coulter, Sweet, & Murphy, 1979; Wong & Epps, 2016). Memory load was differentially manipulated by Wright and Kahneman (1971) by asking participants to either repeat relatively complex sentences or to answer a question about the sentence content. The pupil diameter was larger for sentence repetition toward the end of sentence presentation and during recall. Furthermore, the memory load effect on the pupil response observed in a digit span task can be influenced by processing instructions. For example, presenting a tone indicating that the preceding words can be forgotten reduces pupil size (Johnson, 1971; Wong & Epps, 2016). Kahneman, Onuska, and Wolman (1968) furthermore showed that presenting a nine-digit string divided into subgroups improved performance and decreased the pupil diameter relative to presenting the digits in a single stream. Recalling digits resulted in smaller pupil responses than recalling words, and even larger responses were evoked by a transformation (or working memory) task in which participants had to manipulate the digits perceived (Kahneman & Beatty, 1966). Similarly, Stanners et al. (1979) presented an auditory working memory task in which participants had to add numbers (0, 1, or 3) to each of four digits. The pupil dilation response was larger for the add-1 and add-3 conditions when compared with the add-0 condition.

Besides short-term memory (span) tasks, recognition memory has been assessed in studies applying pupillometry. In the recognition phase of a memory task, the pupil dilated more to deeply encoded audiovisual words than to words that were encoded in a shallow way or to new words (Otero, Weekes, & Hutton, 2011). Moreover, the pupil diameter during encoding was associated with subsequent recognition performance, and the pupil diameter during recognition is sensitive to voice familiarity, recognition performance, and recognition confidence (Papesh et al., 2012). In summary, the pupil size sensitively reflects the memory demands imposed by auditory stimuli, with larger responses when more items are encoded, larger responses for words when compared with digits, and larger responses are evoked by working memory tasks when compared with short-term memory tasks.

Pupil dilation responses to long digit strings: Overloading memory capacity

Similar to the results obtained when assessing the influence of intelligibility level on the pupil dilation response, studies that manipulated memory demands have found evidence for reduced pupil dilation in conditions imposing a high memory load when compared with conditions imposing a smaller memory load. Peavler (1974) was the first to demonstrate this effect when showing that the pupil size was smaller for a recall condition with 13-digit when compared with 9-digit strings. Karatekin (2004) applied a digit span task and observed the common finding of increasing pupil dilation with each digit to be remembered. However, in children, this relation was less steep than in adults. Also, children had a smaller dilation in the most difficult (eight-digit) condition when compared with an easier (six-digit) condition. The authors suggested that the memory load may have exceeded the capacity of the children in the most difficult condition. Similar results were found by Johnson, Miller Singley, Peckham, Johnson, and Bunge (2014). Granholm et al. (1996) also observed such signs of overload (reducing pupil dilation responses) when long digit sequences were presented, whereas Cabestrero et al. (2009) showed that the pupil diameter plateaued when resources were exceeded.

Mental Problem-Solving

In general, performing (solving) more difficult mental tasks evoke larger pupil dilation responses than performing easier mental tasks (see Ahern & Beatty, 1979, 1981; Klingner et al., 2011). Also, a larger index of cognitive activity has been observed when solving arithmetic problems when compared with rest (Marshall, 2007). Illumination differences did not affect this measure.

Emotional Valence

As described in the Introduction section, it is evident that the pupil response is sensitive to the emotional valence of auditory stimuli. Emotional stimuli evoke larger pupil responses than neutral stimuli in healthy controls (e.g., Fletcher et al., 2015; Jin, Steding, & Webb, 2015). Usually, the nature of the emotions evoked (e.g., positive vs. negative) do differentially affect the pupil size (Partala & Surakka, 2003; White & Maltzman, 1978). However, there is some evidence that largest pupil sizes are observed for aversive stimuli (Babiker, Faye, Prehn, & Malik, 2015; Burley, Gray, & Snowden, 2017; Stanners et al., 1979), but Chaney, Givens, Aoki, and Gombiner (1989) observed pupil constriction in response to harsh commands. Kang and Wheatley (2017) assessed the synchrony in the pupil dilation of speakers and listeners watching or listening to a recording of these speakers. The synchrony in the pupil dilation patterns of speakers and listeners was larger for expressive speakers and for listeners showing larger empathy and was also larger for highly engaging portions of the narrative when compared with less engaging phrases. In contrast, presenting boring or interesting auditory fragments during a driving simulator task did not influence the pupil size (Horrey, Lesch, Garabet, Simmons, & Maikala, 2017), and the emotional valence of auditory, visual, or audiovisual stimuli did not influence the pupil diameter in the study of Rosa, Oliveira, Alghazzawi, Fardoun, and Gamito (2017) either.

Music is a well-established method of inducing emotions—the pupil dilates in response to music-evoked chills (Laeng et al., 2016). Vocal music resulted in larger pupil dilation than piano music (Weiss, Trehub, Schellenberg, & Habashi, 2016). Gingras, Marin, Puig-Waldmüller, and Fitch (2015) showed that pupil response to music reflected the perceived arousal or tension levels of the music excerpts, but the effect interacted with the attitudes of the listeners toward the music sample or to music in general.

Arousal, Fatigue, Displeasure, and the Task-Evoked Pupil Response

Effects of task-induced fatigue or time-on-task on the baseline and pupil dilation response have been reported in several studies (Beatty, 1982; Kahneman & Beatty, 1967; Murphy et al., 2011). The pupil size decreases as function of increasing trial number when similar stimuli are repeatedly presented (i.e., the habituation response; Damsma & Van Rijn, 2017; Marois et al., 2018; Steiner & Barry, 2011; Stelmack & Siddle, 1982, but see McGarrigle et al. 2017b; Schlemmer, Kulke, Kuchinke, & Van Der Meer, 2005). In contrast, Murphy et al. (2011) showed increasing pupil baseline diameter during the course of a test session. In general, the pupil dilation response is larger in the beginning of a test session and in the beginning of each block of stimuli when compared with later blocks/sentences (Dahlman, Sjörs, Lindström, Ledin, & Falkmer, 2009; Koch & Janse, 2016; Murphy et al., 2011; Zekveld et al., 2010). Stanners and Headley (1972) showed that the pupil response to sentences was smaller in the second half of the test session when compared with the first half of the test session. Moreover, this effect interacted with some of the main stimulus conditions, which was also the case in the studies of Ambler et al. (1976); Demberg (2013); Demberg and Sayeed (2016); Fletcher et al. (2016); and McGarrigle et al. (2017a).

Besides time-on-task or fatigue, motivation can be directly manipulated by imposing rewards or threats. Bijleveld, Custers, and Aarts (2009) and Knapen et al. (2016) showed larger pupil sizes when providing a high versus a low monetary reward during a digit span task. In contrast, Stanners et al. (1979) did not observe an effect of incentive (no incentive, monetary reward, or threat of shock) on the pupil dilation response during the performance of an auditory working memory task. The pupil is however sensitive to stimuli that have been paired with an aversive stimulus (Kluge et al., 2011; Korn, Staib, Tzovara, Castegnetti, & Bach, 2017), and this effect does not habituate over the course of the test session (Kluge et al., 2011). Gilzenrat et al. (2010) presented a tone-discrimination task including impossible trials and showed that such trials, when combined with negative feedback, are associated with a relatively small pupil dilation response relative to baseline. This possibly reflects disengagement of the individuals performing the task. Indeed, when given the option to “escape” the impossible trials, participants often did so.

Listener Factors Influencing the Pupil Dilation Response to Auditory Stimuli

Age

Young (M age = 14 months; Wetzel et al., 2016) and older (M age = 10 years; Karatekin, 2004) children seem to have a larger initial (or absolute) pupil diameter than young adult participants (between 18 and 27 years of age). Wetzel et al. (2016) applied a principal component analysis to separate the main factors underlying the pupil response. In line with Steinhauer and Hakerem (1992), a parasympathetic and sympathetic component was obtained that together explained more than 95% of the variance. This supports the idea that the pupil response reflects the summation of parasympathetic and sympathetic activity (Wetzel et al., 2016). Interestingly, Wetzel et al. observed an interaction between age-group and the effect of the stimulus on the two components such that they concluded that the 14-month-old children have a larger sympathetic response to noise and the cry of a peer (baby) when compared with young adults.

Steel, Papsin, and Gordon (2015) showed that older normal-hearing children had a smaller pupil diameter change in response to an auditory fusion task than younger normal-hearing children (M age = 12 years; age range not provided). In adults, the baseline pupil size decreased with increasing age (age range: 25–75 years; Morris, Granholm, Sarkin, & Jeste, 1997). Johnson et al. (2014) found that the pupil response to strings of 9 or 11 digits was similar for children (M age = 11 years) when compared with young adults (M age = 18 years). For children, the increases in pupil size with each additional digit encoded in memory plateaued around six digits. This plateau was around eight digits for the adults, which may suggest that the children had a smaller memory capacity than the adults.

Increasing age is associated with reduced dynamic range of the pupil in older listeners when compared with younger listeners (Piquado et al., 2010). However, when analyzing the normalized pupil dilation responses (relative to the dynamic range of the pupil), Piquado et al. found larger pupil dilation in the older group. Compared with young (M age = 21 years) listeners, middle-aged and older (M ages = 50 or 67 years, respectively) listeners had smaller and faster pupil dilation in response to the perception of conversational fragments (Koch & Janse, 2016). Zekveld et al. (2011) observed that in middle-aged listeners (aged around 57 years), the pupil dilation response has a longer duration than the response of younger listeners (M age = 23 years). Also, the baseline pupil size was smaller in middle-aged listeners when compared with younger listeners.

However, no relationship between age and the pupil response has been shown by Ayasse et al. (2017); Chaney et al., 1989; Koelewijn et al. (2012a); Kuchinsky et al. (2016); and Morris et al. (1997) even though the participants included in some of these studies covered a relatively wide age range (e.g., Ayasse et al., 2017). Hochmann and Papeo (2014) showed that the effect of a linguistic manipulation that was present in 6-month-old children was absent in 3-month-old children, indicating that pupillometry can be applied to assess linguistic development in young babies.

Hearing status

Kramer et al. (1997) were the first to demonstrate an interaction between hearing status and listening condition on the pupil response. With increasing intelligibility, the decrease in pupil dilation relative to baseline was smaller for listeners with hearing loss. Note that the groups of listeners with normal hearing were not matched in age to those with hearing loss. This interaction effect was replicated by Kramer et al. (2016) and Zekveld et al. (2011). Kuchinsky et al. (2014) observed that more severe hearing loss was associated with a flatter pupil dilation response. Absent effects of hearing status on the pupil response were reported by Kuchinsky et al. (2016) and by Koelewijn, Versfeld, and Kramer (2017), although the latency of the peak dilation was smaller for listeners with hearing impairment when compared with listeners with normal hearing in Koelewijn et al. (2017).

Ohlenforst et al. (2017b) recently measured the pupil response of listeners with normal hearing and listeners with hearing impairment and demonstrated that the pattern of pupil dilation across a wide range of SNRs differed between the two groups. Part of this effect was associated with hearing-related intelligibility differences between the two groups, but the data suggested that the pupil dilation response of listeners with hearing loss was relatively small in difficult conditions, around 50% intelligibility (see also Wang et al., 2018a). In contrast, participants with poorer hearing had a larger increase in the pupil size during a visual-world paradigm than participants with relatively slight hearing loss (Ayasse et al., 2017). Steel et al. (2015) showed that children (M age = 11 years) with a cochlear implant showed larger pupil dilation in a binaural fusion task when compared with age-matched children with normal hearing. The pupil response was associated with the fusion performance that was not matched between groups: The poorer the performance level, the larger the pupil response. Finally, the decline in the pupil response to clear speech has been shown to be slower in cochlear implant users when compared with listeners with normal hearing when processing degraded speech. However, age differences between the groups may have confounded this effect (Winn, 2016). Finally, Kitajima et al. (2010) assessed the pupil response to loud clicks in deaf participants and did not observe a difference in their response when compared with that of listeners with normal hearing. In summary, the relatively few studies directly assessing the effect of hearing status on the pupil dilation response to auditory stimuli have shown mixed results. The studies that observed an effect seem to point to an interaction between stimulus condition and hearing status.

Cognitive ability

Regarding the association between cognitive ability and effort, several hypotheses have been formulated (see Van der Meer et al., 2010; Zekveld et al., 2011 for an overview). In short, the effort hypothesis suggests that larger cognitive capacity is associated with larger investment of resources in the task, regardless of task difficulty. In contrast, persons with large cognitive capacity may allocate their capacity more efficiently, which would imply smaller resource allocation (efficiency hypothesis). Finally, drawing on the resource hypothesis one could argue that capacity affects resource allocation particularly in challenging conditions. In these conditions, larger capacity would increase performance and processing load (Zekveld et al., 2011).

Ahern and Beatty (1981) provided support for the resource hypothesis. In their study, higher scholastic aptitude test scores were associated with smaller pupil responses in several cognitive tasks (see also Ahern & Beatty, 1979). However, in the most difficult digit span condition, pupil response was larger for the participants with higher scholastic scores. The same association between larger cognitive capacity and smaller or faster pupil dilation was observed by Koch and Janse (2016) and Wendt et al. (2017). In contrast, in young and middle-aged listeners with normal hearing, better linguistic ability was associated with a relatively large and later peak pupil dilation relative to listeners with poorer linguistic ability (Zekveld et al., 2011), and similar results were reported by Zekveld and Kramer (2014) in very difficult listening conditions. Also, larger working memory capacity and better linguistic ability were related to larger pupil dilation responses to sentences masked by interfering speech in middle-aged listeners with normal hearing (Koelewijn et al., 2012b). Kuchinsky et al. (2014) observed that lower vocabulary knowledge was associated with a flatter pupil dilation response. Larger digit span scores in young listeners were related to larger pupil dilation in a sentence perception task (Wendt et al., 2016). In contrast, no relationships between cognitive ability and pupil responses were observed by Zekveld et al. (2013, 2014a) and Koelewijn et al. (2014b). Furthermore, Damsma and Van Rijn (2017) showed that musical expertise did not influence the pupil response evoked by standard or deviant rhythms. However, musicians (Bianchi, Santurette, Wendt, & Dau, 2016) and absolute pitch possessors (Schlemmer et al., 2005) have been shown to have smaller pupil dilation responses during a tone identification and discrimination tasks when compared with nonpossessors and nonmusicians, respectively. Finally, patients with aphasia had the same pupil response to words paired with visual images of the words as those without aphasia. Also, in both groups, the same effect of word difficulty was observed (Chapman & Hallowell, 2015). In summary, the evidence currently available points to an absent relationship between capacity and the pupil dilation response or to a relationship between larger cognitive ability and larger pupil responses. In some studies, this association was most pronounced for more difficult conditions (e.g., Ahern & Beatty, 1981; Zekveld & Kramer, 2014).

Gender effects

The relatively few (N = 4) studies that explicitly tested the difference in pupil response between male and female participants found conflicting evidence. Females were shown to have a larger pupil dilation to neutral sounds (Partala & Surakka, 2003), whereas males had a larger pupil response to music excerpts (Gingras et al., 2015). There was no difference observed by Burley et al. (2017). Dabbs (1997) observed an interaction between testosterone level and sex, with low-testosterone males showing a relatively quick decrease in the pupil response to sexual auditory stimuli.

Personality factors, psychiatric conditions, and dementia

Antikainen and Niemi (1983) reported that the relationship between increasing stimulus loudness and larger pupil size was stronger for neurotic persons when compared with “stable” persons. Schizophrenia is associated with reduced pupil dilation relative to baseline (Fish & Granholm, 2008; Granholm et al., 1997; Steinhauer & Zubin, 1982). Fletcher et al. (2016) showed that patients with semantic dementia and Alzheimer’s dementia had a relatively large pupil dilation response to meaningful when sounds compared with meaningless sounds than healthy controls. In a previous study, they also observed altered pupil responses in patients with several types of dementia syndromes evoked by sounds differing in saliency (Fletcher et al., 2015). Major depressive disorder and psychopathy were not associated with altered pupil dilation to emotional or neutral sounds in the studies of Burley et al. (2017) and Jin et al. (2015).

Guilty knowledge

Bradley and Janisse (1981) assessed the effectiveness of pupillometry to correctly identify participants guilty of a mock crime. They observed that the pupil dilation response significantly discriminated between participants who were guilty or innocent. Similar results were obtained by Bradley and Janisse (1979), Dionisio et al. (2001), and Webb et al. (2009), all supporting the finding that deception is associated with a larger pupil size when compared with telling the truth.

Discussion

The current review aimed to provide an overview of the current knowledge regarding the pupil dilation response to auditory stimuli. The main finding of this review is that a wealth of studies have assessed the pupil dilation response to auditory stimuli (see Supplementary Table 1 for a summary of each study). A large part of this research has been performed in fields other than the field of Audiology and Hearing Sciences. The evidence described by these studies has been reviewed in the context of the FUEL components. An overview is provided in Figure 3.

Figure 3.

Results of the current review. The figure shows the factors influencing the pupil size during auditory processing and the number of publications showing evidence for any effect of the factor on the pupil size. The findings are organized according to the Framework for Understanding Effortful Listening (FUEL). Several articles provided support for the effect of more than one factor. Please refer to Tables 1 and 2 for the relevant references associated with the factors described and for references in which no support was observed for the effect of these factors.

Note that some factors have been assessed much more extensively than others. Although the importance of assessing listener-related factors such as age, gender, and hearing status has been underlined in recent discussions (Peelle et al., 2017; Pichora-Fuller et al., 2016), the vast majority of studies did not formally test the influence of these individual characteristics on the pupil dilation response. This could be partly based on the fact that some of the measurement devices do not provide access to absolute pupil sizes (e.g., Otero et al., 2011; Scheepers et al., 2013).

Regarding the studies to the effect of external factors on the pupil dilation response (see Table 1), the evidence regarding the effect of linguistic complexity, memory load, and intelligibility/degradation level is most convincing. The current review indicates that increasing linguistic processing demands, increasing the memory load, and increasing the degradation level will generally increase the pupil dilation response as long as the demands on capacity do not exceed the available capacity. Furthermore, several studies clearly indicate an effect of the time-on-task (fatigue), motivation, and emotional factors on the pupil size during auditory processing. These factors have been studied extensively with only a few contradictory findings.

The studies that tested the effect of age showed that in general, the (baseline) pupil size as well as the dynamic range of the pupil size tends to be larger in children and young adults when compared with older groups. About one third of the studies has taken into account (age-related) interindividual difference in the pupil size by calculating the relative increase in the pupil response relative to a baseline pupil size (e.g., percentage increase or z-normalized pupil dilation). Note that many studies (i.e., an estimated 65 out of the 108 that provided some information about the age of the participants) included young (age at most 35 years) and often highly educated (student) participant samples, thereby limiting the generalizability of the results to these groups. Only 23 studies explicitly reported to have included participants over 60 years of age (see Supplementary Table 1).

Please note that some of the studies described in this review attempted to control for age-related differences in absolute pupil size (Morris et al., 1997). This was done by scaling the pupil size data to the dynamic range of the individual participants (Ayasse et al., 2017; Piquado et al., 2010), by mean normalizing the data (Kuchinsky et al., 2016), or by analyzing the proportional increase in pupil size relative to the baseline pupil size (Johnson et al., 2014; Karatekin, 2004; Steel et al., 2015). However, even within the studies that used the same scaling, normalization or baseline subtraction procedure, the effect of age was not consistent (although the analysis method may have influenced the results). Furthermore, age may differentially affect different components of the pupil dilation response reflecting parasympathetic versus sympathetic effects, as was shown by Wetzel et al. (2016). Therefore, it is currently unclear whether and how age affects the pupil dilation response to auditory stimuli. More generally, it is relevant to further evaluate how different preprocessing and analyzing techniques impact the results observed (see also Winn et al., current Special Issue).

As described earlier, one important result of the present review is the identification of a gap in knowledge about the feasibility and validity of pupillometry applied in adults older than 60 years of age. As the effect of age is not yet clear, care must be taken to control for age when assessing other listener factors, such as hearing loss. Only 10 studies assessed the effect of hearing status (see Table 2), and three out of the eight studies that observed an effect of hearing status observed an interaction between hearing and intelligibility level. Relatively small pupil dilation responses relative to baseline have been observed in listeners with hearing loss in conditions with intelligibility levels around or below 50% correct (Ohlenforst et al., 2017b; Zekveld et al., 2011). We conclude that the pupil response, and the resource allocation processes reflected by the response, indexes a complex mechanism underlying cognitive resource allocation that is affected by numerous factors. The present results support the value of using the pupil dilation response in the assessment of the influence of the FUEL components on listening effort.

As indicated by Pichora-Fuller et al. (2016), future development of measures of listening effort such as pupillometry should involve clarifying which measure is most appropriate to use for a specific purpose. Also, they state that it is useful to identify the measures that are most responsive to variations in the demand dimension versus the motivation dimension. The present review shows that the pupil size is sensitive not only to task demands and motivation but also to emotional valence, fatigue, and automatic attention (see Table 1). So what does the pupil dilation response reflect? The sensitivity of the method to a wide variety of processes implies that the interpretation of differences in the pupil dilation response should follow the study characteristics: the factors manipulated and those controlled for. This also explains the wide variety in terminology applied in the reviewed studies. Although listening effort has been used in studies to the effect of hearing status and intelligibility, studies in other fields of research did use other terms such as autonomic (or involuntary) activation, cognitive resource allocation, mental activity, and information processing effort. As already concluded by Ben-Nun (1986), any effect of a task manipulation can relate to arousal rather than specific steps or levels in the task itself. Arousal is closely linked to cognitive resource allocation (Pichora-Fuller et al., 2016), and the pupil dilation response sensitively reflects differences in arousal.

In addition, one should note that although some of the reviewed studies specifically focused on auditory processing, others assessed the pupil dilation response to auditory, visual, or audiovisual stimuli using similar test paradigms (e.g., D’Ascenzo et al., 2018; Einhäuser et al., 2008; Klingner et al., 2011; Lisi et al., 2015; Otero et al., 2011; Rosa et al., 2017; Taylor, 1981; Wang et al., 2017). By examining the pupil response to auditory and visual stimuli, insight can be obtained into the degree to which the processes underlying the pupil dilation response are modality general (e.g., Klingner et al., 2011; Lisi et al., 2015; Taylor, 1981) or may reflect multisensory integration (e.g., Wang et al., 2017).

Future research should focus on the factors with limited or conflicting evidence (see Tables 1 and 2). Moreover, it is valuable to provide more evidence allowing testing the FUEL components as well as the presumed relationships and interactions between these components. For example, the current review points to an inverse-U-shaped function of the pupil dilation response across a wide range of intelligibility levels or memory demands (Granholm et al., 1996; Karatekin, 2004; Kramer et al., 2016; Ohlenforst et al., 2017b; Peavler, 1974; Zekveld & Kramer, 2014). The reduction in the pupil response when task conditions are very difficult points to the interaction between motivation and task demands. In general, cognitive resource allocation will likely be larger for tasks imposing greater task (or input-related) demands. It is low when task demands are low and increases with increasing demands. This is the case as long as the participant is motivated to invest effort. The motivation to perform the task will likely be reduced if the task demands are too high (Richter et al., 2016), when there is fatigue (Hornsby, Naylor, & Bess, 2016), or a combination of these factors. Crucially, the change in cognitive resource allocation imposed by a change in the task demands depends on the current (baseline) position on the effort curve. Lowering task demands could reduce the allocation of resources when the baseline condition is at the peak of the curve. However, they could also increase if the listener becomes more motivated to perform the task. To be able to correctly interpret effects of factors affecting the current task demands, it is thus imperative to take these underlying mechanisms into account. One way to do so is by measuring performance next to listening effort and by including a relatively wide range of performance levels when assessing listening effort in a given task (McMahon et al., 2016; Ohlenforst et al., 2017b; Peavler, 1974). Another way is by developing smart designs actually providing the opportunity to (and thereby assessing the tendency of participants) give up performing a task (Gilzenrat et al., 2010). The present literature shows that effort is the result of a dynamic process; future studies should acknowledge the complexity of the relevant mechanisms in play.

Finally, as the pupil size is sensitive to interindividual characteristics (hearing status, psychiatric factors), the clinical relevance and clinical applicability of the method should be considered. The present review points to a lack of or conflicting evidence with respect to the effect of individual differences, especially in older age groups. Also, test–retest reliability should be further assessed (Marandi, Madeleine, Omland, Vuillerme, & Samani, 2018; Stelmack & Siddle, 1982). Therefore, more research is needed before individual data can be validly interpreted. On the other hand, measuring pupil size in additional to existing clinical tasks may be feasible in many cases as the method is not invasive and as the use of auditory stimuli can often be combined relatively easily with the assessment of the size of the pupils.

In conclusion, pupillometry is increasingly being applied (Figure 2). The method was relatively popular during the period when it was established by the seminal publications by Beatty, Kahneman, and coworkers (see also Laeng et al., 2012). Two general observations can be made based on the review of these early studies. First, they assessed factors that are still highly relevant today (e.g., the effect of memory load, linguistic complexity, auditory processing demands). Second, most of their findings have been replicated in later studies even though these early studies applied less advanced methods and technology (i.e., pupillometry using analogue cameras), often combined with a relatively small sample size. This supports the reliability and robustness of the findings reported in the literature. The application areas of pupillometry are seemingly endless, justifying the current enthusiasm for this method. Nevertheless, more work is needed as there are some remaining questions, in particular regarding the effect of listener factors and regarding the impact of the analysis method applied on the results. Recommendations of this review include focusing on other participant samples other than the “default” student groups and on the examination of the interactions between various relevant determinants of resource allocation, such as the task demands, motivation, and attention. We hope that the present overview and summary of the reviewed studies provided in the tables will be of great value in the design, description, and interpretation of future work that apply pupillometry in a wide variety of research areas.

Click here for additional data file.

Supplemental material for The Pupil Dilation Response to Auditory Stimuli: Current State of Knowledge by Adriana A. Zekveld, Thomas Koelewijn and Sophia E. Kramer in Trends in Hearing