Emotion in Action: A Predictive Processing Perspective and Theoretical Synthesis.

Starting from a decidedly Frijdian perspective on emotion in action, we adopt neurocognitive theories of action control to analyze the mechanisms through which emotional action arises. Appraisal of events vis-à-vis concerns gives rise to a determinate motive to establish a specific state of the world; the pragmatic idea of the action's effects incurs the valuation of action options and a change in action readiness in the form of incipient ideomotor capture of the selected action. Forward modeling of the sensory consequences of the selected action option allows for the evaluation and fine-tuning of anticipated action effects, which renders the emotional action impulsive yet purposive. This novel theoretical synthesis depicts the cornerstone principles for a mechanistic view on emotion in action.

“Emotions exist for the sake of signaling states of the world that have to be responded to.”

“. . .the principle of passion: to manifest states of action readiness . . . that bear on the aim of achieving and maintaining, or terminating or decreasing one’s relationship to a particular object or event; . . . and to seek precedence over ongoing behavior.”

Emotion in Action: An Integrative View

Imagine having an itch that triggers the irresistible impulse to scratch. Although the motive (to do away with the itch) is not necessarily a consciously set goal, the action (scratching) is purposive: it implements the aim of the motive that called it forth, namely getting rid of the itch. One may or may not become aware of the itch, or of the motive to get rid of it, or of the act of scratching. The action may be preceded by a consciousness of the purpose to be attained, or may ensue without any premeditation. Hence, our view on emotion in action necessitates neither a distinct mandate of the will, nor the presupposition of awareness of the motive for action. An action may be emotional even if we are unable to render (or even understand) the words to describe the motive.

An emotional act, in contrast to a predetermined act, is determined by its ultimate or proximate end. The motive of the action is its desired end state (Grafton & Hamilton, 2007; Sebanz, Bekkering, & Knoblich, 2006). Emotional actions serve the purpose of rendering one’s relation to the state of the world more pleasant or less unpleasant. Emotional actions are produced by motives to alter the current state of the world so as to approximate a more optimal state. These motives give rise to states of action readiness. As elaborated in Frijda’s work, changes in action readiness form the key to what we call “emotions,” since action readiness covers the variations in self–object relationships (e.g., Frijda, 1986, 2007). These states of action readiness are propelled by how a perceived (or recalled or imagined) object or event is appraised by the individual. Appraisal of events in the world imparts purpose by giving rise to a state of action readiness to change one’s relation to its object. Emotional action deals with the present situation by correlating it with relevant concerns, and valuating the courses of action that will be most likely to lead to its desired end. Predetermined actions are, one might argue, ultimately also geared towards desirable states or away from undesirable states. The difference is that such actions result from innate, hard-wired, or overlearned stimulus–response associations, without the intervening appraisal, motive, and readiness for action that characterizes emotional acts.

Beyond being purposive, emotional actions comprise an impetus for action—an inclination to do something—with a certain strength or urgency. These dynamic and energetic aspects of emotional action have been caught under the designation of control precedence of ongoing actions and strivings (Frijda, 1986, 2007). Control precedence includes persistence of action over time until a particular end state has been reached, and resumption of actions in spite of interruptions and obstacles.

Importantly, emotional action entails some anticipatory image of the effects of the action (Eder & Hommel, 2013; Eder & Rothermund, 2013; Frijda, Ridderinkhof, & Rietveld, 2014). In order to predict action effects, interactive behavior is guided by pragmatic processes that embed sensorimotor coordination in prior knowledge and experiences (Gibson, 1979). Experience endows stimuli with a power of awakening previously appropriate reactions. Purposive action entails prediction or anticipation of intended outcome (Clark, 2013; Engel, 2010). Such pragmatic prediction rests on the neural mechanisms of forward modeling of action effects. Forward models deliver predictions that help interpret the sensory effects of one’s actions, or those of others (Blakemore, Wolpert, & Frith, 2002; Desmurget & Grafton, 2000; O’Regan & Noë, 2001; Wolpert & Flanagan, 2001).

Present Aims

In what follows, we will study the mechanisms through which emotional action arises. We will see how principles of predictive processing map on the notions of incipient ideomotor capture and readiness for action, how motives are selected through appraisal, how action options are selected through valuation, and how the discrepancy between actual and desired action effects is mitigated through forward modeling. The notions of pragmatic idea and incipient ideomotor capture will be found to constitute cornerstones for emotion in action. These proposals are consistent with, on the one hand, Frijda’s recent views on emotion vis-à-vis impulsive action (Frijda et al., 2014), and, on the other hand, the principles and mechanisms of perception–action coordination laid out in a recent integrative theoretical framework (impetus, motivation, and prediction in perception–action coordination theory, or IMPPACT; Ridderinkhof, 2014). The core mechanism for emotional action, as derived from these positions, is summarized in Figure 1, and will be detailed in what follows.

Figure 1.

Schematic architecture for ideomotor action according to the IMPPACT model. Stimuli and ideas (denoted by letters A, B, etc.) activate the corresponding motor controllers (denoted by numbers 1, 2, etc.) via a series of ideomotor processes. Appraisal of the stimuli and ideas yields motives in the form of desired action effects (denoted by letters A’’, B’’, etc.). Action options (denoted by numbers 1’’, 2’’, etc.) are valued in terms of optimal opportunity for bringing about the desired action ends. The elected course of action captures the motor system incipiently before being executed in full. Action effects (as perceived through exteroceptive senses) are fed into a comparator (symbolized by the purple-colored hexagon) to be compared against the desired action effects, giving rise (in case of discrepancy) to a prediction error (PE) which is used to revaluate and adjust the chose action option. (Adapted from Ridderinkhof, 2014.)

While the IMPPACT framework may be relatively new, many of the ideas in the model and the views on the mechanisms through which emotional action arises borrow heavily from views that have been voiced in the (often very old) literature. While some of these views have been central to theoretical views on emotion, others are newly introduced into this context. A particularly novel contribution of this article will be to evaluate the implications of introducing the forward model component, central to IMPPACT, for understanding emotional action. Thus, this article aims at a synthesis that integrates previous work and extends it with the notion of forward modeling. For somewhat similar theoretical work and empirical support, we refer the reader to recent work by Eder and colleagues (Eder & Hommel, 2013; Eder & Rothermund, 2013; Eder, Rothermund, De Houwer, & Hommel, 2015).

Ingredients of Emotion in Action

Appraisal

Emotions are aroused by events: events as directly perceived, or as recollected, or even as imagined. Motives for emotional action are elicited by events as appraised. Appraisal refers to the information that confers personal meaning to brute perceived events, which is drawn from environmental stimuli plus their temporal and spatial context (Frijda, 2007; Scherer, 2005).[2] Appraisal results from interactions between event properties on the one hand and the individual’s sensitivities to these properties on the other. Appraisal identifies what relationship should be established or modified, and thus gives rise to a determinate motive for action: whether or not something needs to be done about it, and how urgently. Appraisal processes proceed largely automatically, associatively, nonconsciously, and in a fraction of a second (Frijda, 2013; Lambie & Marcel, 2002).

To illustrate: Incoming input (say, very loud heavy metal music) activates not only the features that represent the stimulus itself, but also (through rapidly spreading activation) all kinds of features that have, mostly through previous experience, become associated to the stimulus (such as concerns: a dislike for loud sound, and a dislike for heavy metal, hypothetically speaking). If a stimulus represents an “actual state of the world” (say, being exposed to loud heavy metal) then a course comparison to associated concerns (dislike for loud music, dislike for heavy metal) would, through simple and rapid computations, lead to the activation of the features of a “desired state of the world” (no loud music, no heavy metal).

The central appraisal is whether an event is assessed as promoting or obstructing one’s concerns (Ellsworth & Scherer, 2003; Frijda, 2007; Scherer, 2005). Concerns relate to norms, preferences, interests; to whatever affects the presence, availability, or intactness of everything the individual cares about (cf. Frankfurt, 1988). Concerns can pertain to what the individual may derive from the event (e.g., likely satisfaction or dissatisfaction), or to opportunities for action (the action affordances provided by the environment, given our expertise; Gibson, 1979). Crucially, in the absence of concern pertinence, there is no motive, and no readiness to act; in short, no emotion (Frijda et al., 2014).

Pragmatic Idea

The possibility of emotional acts, in which the response is determined by the effect it will exert, arises with the power of anticipation, so that actions are directed by the relation between act and consequence (Ridderinkhof, 2014). The emotional act is determined by reference to a desired end; hence, we must be able to form some pragmatic idea which is capable of prompting purposive action (consciously or not).

The pivot upon which emotional action turns is the formation of a pragmatic idea of the desired action effect. A pragmatic idea consists of images of kinesthetic sensation associated with the action and its anticipated effects on the world and on our own body (Hobhouse, 1915). An environmental object or event may be perceived as presenting a specific action affordance, soliciting activation of the corresponding motor program, without an agent’s deliberate intention to act (Grèzes & Decety, 2002). The perceived affordance thus instigates an intention-in-action (Searle, 1983) or motor intention (Dreyfus, 2005; Pacherie, 2006). Indeed, pragmatic ideas of movement may activate the descending motor pathways at subthreshold level (e.g., Tanaka, Balleine, & O’Doherty, 2008), and are in fact routinely followed by discharges of their target muscles (Jacobsen, 1927). Harleß (1861/2012) used the term “Effektbild” to denote the consequences of actions, not only in terms of sensory effects but also in terms of outcomes that one can learn to pursue or avoid. The results experienced after yielding to an impulse will associate the impulse, in memory, with a foresight of those results. “An impulse acted out may be said to be acted out, in part at least, for the sake of its results” (James, 1890, p. 390).

Incipient Ideomotor Capture

William James famously wrote “We think the act, and it is done” (1890, p. 522). Unzer (1771/1851) and Herbart (1816/1887) were among the first to enunciate the notion that actions are initiated by the anticipation of the desired sensory effects of the projected action (for an historical analysis see Ridderinkhof, 2014). Herbart proposed that when an action is executed, associations are formed between the action and its sensory effects; these associations can be used subsequently to initiate those actions that produce the desired action effects. Müller (1838) observed that “The idea of a particular motion determines a current of nervous action towards the necessary muscles, and gives rise to the motion independently of the will” (p. 944). Some individuals, when walking closely to the rim of a precipice, experience the compulsion to jump off, as phrased by Bacon (1826): “[F]or, imagining a fall, it putteth his spirit into the very action of a fall” (p. 957). This conception was articulated more generally by Lotze (1852): “As soon as an idea of an accessible goal surfaces into memory, the unfolding action appears as directed to that goal, seeking to approach it” (p. 298). Carpenter (1853) labeled the notion that thinking of the effects of an action tends to set the action in motion as ideomotor (to distinguish it from sensorimotor action).

The pragmatic idea of a desired action effect can serve to retrieve the action that gives rise to the sensory effects associated with the appropriate action effect (Herwig, Prinz, & Waszak, 2007). The ideomotor principle thus holds that any activation of the pragmatic idea of an action’s effect may awaken the corresponding action. As James (1890) observed, “The idea of the movement’s sensory effects will have become an immediate antecedent condition to the production of the movement itself” (p. 586). However, such ideomotor action is only incipient: “The drive does not attain its goal just by itself, like some physical power; it hardly exceeds beyond an aspiration, a striving for an aspired effect” (Lotze, 1852, pp. 296–297). The nascent ideomotor tendency may be left quiescent unless its motive power is sufficiently strong to solicit and capture the action in a maximum degree; the motive becomes motor only when the nascent excitation becomes an unchecked impulse (Lewes, 1877).

Changes in Action Readiness

Incipient ideomotor capture may thus remain limited to mere potentiations of the neural dispositions underlying the actions concerned, without motor engagement (Frijda et al., 2014; van Loon, van den Wildenberg, van Stegeren, Hajcak, & Ridderinkhof, 2010). Such states of action readiness are best described as states of readiness to establish, maintain, or modify some self–object relationship; in other words, to prepare for effecting some action that reduces the discrepancy between our current state and the desired state (Deonna & Teroni, 2012). Changes in action readiness are central to emotions (Frijda, 2007; McDougall, 1928).

The term action readiness is borrowed from Dewey’s theory of emotions (1895): “When we say that John Smith is very resentful at the treatment he has received . . . we mean he is in a certain practical attitude, has assumed a readiness to act in certain ways” (p. 17). Bull (1945) luminously extended the James–Lange theory of the emotions (in which we do not cry because we feel sorry, but feel sorry because we cry) by claiming that the sorry feeling results from readiness to cry, rather than from actually crying.

We feel angry as a result of readiness to strike, and feel afraid as a result of readiness to run away, and not because of actually hitting out or running . . . In other words, feeling, or affect, arises from preparatory attitudes, maintained as readiness or a wish, and held in leash pending the lifting of whatever form of interfering mechanism is holding up the action. (p. 211)

A state of action readiness may be felt by the agent as a nonovert inclination, or experienced as a mental image of action (Ridderinkhof & Brass, 2015).

Valuation of Action Options

Given a specific motive for action, a person can avail herself of a number of different alternative courses of action to accomplish a modification in a particular self–world relationship, such as the riddance of an itch (Frijda, 1986). To offer a typical Frijdian example: When charmed by someone, one may utter endeared words, offer a rose, or try and steal a kiss in an attempt to draw or keep the charming person nearby; when running away in fear, one seeks to increase the distance to the danger source, but one may also hide behind a protective barrier, or throw an obstacle in front of the approaching threat (Frijda, 2007). Action options are then evaluated in terms of their aptness for bringing about these desired action effects. The various action alternatives differ in their value with respect to both their cost (in terms of energy expenditure or financial endowment) and their benefit (in bringing the actual state closer to the desired state).

Adequacy and efficacy of action alternatives scale with the availability of relevant skills or expertise (Rietveld, 2012). To the skilled badminton opponent, appraisal of even a well-concealed crossed drop-shot may generate a relevant affordance for adequate nonreflective reaction which—without premeditation or further deliberation—renders a specific motive state, a readiness to act, an adequate action impulse. By contrast, a lack of relevant affordances may leave the novice stand “frozen”—that is, without an intent, without a concrete motive for action, without any action impulse (Frijda et al., 2014).

The ideomotor principle does not in itself entail a mechanism through which the motivational value of the outcome influences the selection of the best-suited action. Benefits need to be weighed against expenditure in a cost–benefit analysis (for review see Schouppe, Demanet, Boehler, Ridderinkhof, & Notebaert, 2014). Computational approaches accomplish this integration by modifying weights of alternative choices in proportion to reward prediction errors—that is, discrepancies between expected and actual outcomes. At the neurobiological level, such prediction errors are encoded by midbrain dopamine signals (for review see Frank, Cohen, & Sanfey, 2009).

Valuation of action options is driven further by a comparison between desired and anticipated action effects. If this comparison results in discrepancy, then the ensuing prediction error (PE) will serve to update the incentive value of the action option, resulting in value learning as an integrated part of repeated action emulation (for review see Ridderinkhof, 2014).

Forward Modeling of Action Effects

When there is the idea of a desired end, the motive provides the impetus to act. We direct effort to what we desire. Desire impregnates the emotional action with the sense of tension, and of effort to relief the tension by reducing the discrepancy between the current state and the desired end (Lewin, 1926). When there is foresight of the ultimate end, the tension is relieved through purposive effort. The pragmatic idea of the required effort and of the muscular exertion requisite to perform a certain movement is derived from the anticipatory image of the action’s effects. But where does the anticipation, the foresight of the action effect come from?

Emotional action entails a further critical component: predictive processing through forward modeling. The brain continually generates and tests predictions to reduce uncertainty about the effects of our actions (Friston, 2012; Kempf, 1921). Within this conjecture, predictive processing provides a neurobiological and computational framework for understanding emotional action, with forward modeling as a rapid means for verifying whether the projected action meets the desired end (see Figure 2). Forward models predict the sensory effects of the selected action program. The comparison of this prediction to the desired state yields a PE that can be used to optimize the selection of those actions that are adequate for effecting the desired state (Frith, Blakemore, & Wolpert, 2000), allowing for emotional action: that motor program is selected that is most likely to attain the desired action effect (Ridderinkhof, 2014). Without a forward model the organism can only act and learn on the basis of actual behavior; the forward model provides an opportunity for the rapid simulation of action, which provides a platform for action decision and learning without the risks and perils of the effects and consequences of overt action in the real world.

Figure 2.

Schematic architecture for ideomotor action, supplemented with a forward model (turning action selection into an action–effect prediction-and-valuation cycle), according to the IMPPACT model. The forward model calculates the predicted action effects (denoted by letters A’’’, B’’’, etc., for exteroceptive action effects, and numbers 1’’’, 2’’’, etc., for interoceptive and proprioceptive action effects), which are fed into a comparator (symbolized by the purple-colored hexagon). Predicted action effects are compared to actual action effects, giving rise (in case of discrepancy) to a prediction error (PE) which is fed back into the forward model so as to optimize its predictions. Predicted action effects are compared to desired action effects, in which case a PE is used to revaluate and adjust the chose action option, which is then fed into the forward model in its turn; the cycle continues until PE is minimized and the appropriate action can be executed. (Adapted from Ridderinkhof, 2014.)

Forward modeling and its computational bases have been developed extensively in the literature on motor control (e.g., Wolpert & Flanagan, 2001) and have recently been elaborated in the literature on predictive processing (e.g., Friston, 2012). They are introduced here in the emotion literature, providing a solid footing for the selection of actions to meet emotional motives, and hence indeed rendering such actions emotional in a meaningful way. The notion that emotion in action entails forward modeling of action effects in order to attain motives (as specified in terms of desired action effects) implies that emotional action may be considered as ideomotor action. Such a framing may help generate novel hypotheses, that allow for empirical tests (for a seminal example see Eder et al., 2015).

Multiple Emotions

The events we encounter often touch upon a multiplicity of concerns and elicit multiple appraisals, and hence multiple states of action readiness simultaneously. The emotional action impulse incited by one motive may be antagonized and checked by competing motives simultaneously present, given that the competitors are of sufficient strength (Franck, 1878; Michotte & Prüm, 1910). The suggestion of a cool beer may awaken desire, while simultaneously this suggestion may activate the memory of last night’s serious hangover and hence elicit aversion. Mixed emotions can be observed in nostalgia, consisting of pain moderated by the happiness that was, together with pleasure moderated by the regret that it had gone (Bellelli & Saldarelli, 1990).

Multiple emotions require coordination in the form of emotion regulation (Frijda et al., 2014). Some impulses are best held in check until more appropriate times: when holding a cup of hot coffee, an itch may well trigger a scratching impulse, but one would do better to put the cup on a nearby table before giving in to the urge. As a final example borrowed from Nico Frijda, one may consider it worth to suffer prolonged torture for not betraying a comrade, and even to commit suicide to prevent betraying.

Conclusions

In sum, emotional actions are instigated by events that, by processes of appraisal that assess the event’s relevance to the individual’s concerns, elicit some motive to establish or modify a specific state of the world. The ensuing state of action readiness drives and instigates emotional action that helps attain that state, or rather, helps reduce the discrepancy between the desired and current state. The prediction of the sensory consequences of the selected action option allows for the evaluation and fine-tuning of anticipated action effects vis-à-vis what the action should accomplish, which renders the emotional action impulsive yet purposive. Through efference copies or forward modeling, the pragmatic anticipation leads to a state of action readiness. As a result, the action is by definition emotional.

This article introduces the mechanism of forward modeling into the emotion literature, as part of a theoretical synthesis of often old concepts, integrated in IMPPACT (Ridderinkhof, 2014). We do not claim substantial differences between the key components introduced here, and how these component processes or constructs (and their underlying mechanisms) are dealt with in the relevant literatures. Rephrasing questions about emotional behavior in terms of underlying constructs and mechanisms of information processing may not in itself add much explanation. Our aim here was to evaluate whether, from an action control perspective, a meaningful integration and synthesis with emotion theory is feasible, at least at the level of the conceptual components. It is our hope that the present theoretical synthesis, and in particular its inclusion of forward modeling, may engender a deeper understanding of emotion in action (or at least a framework from which novel and testable predictions can be derived).