Brain substrates of behavioral programs associated with self-regulation.

The present paper proposes that four neuromodulator systems underpin highly generalized behavioral sets, but each targets either dorsomedial or ventrolateral cortical systems, where it produces its effects in either a proactive or reactive orientation to the environment. This way systems are discriminated that control reactive approach (dopaminergic), reactive avoidance (cholinergic), proactive behavior (noradrenergic), and withdrawal (serotonergic). This model is compared with models of temperament, affect, personality, and so-called two-system models from psychology. Although the present model converges with previous models that point to a basic scheme underlying temperamental and affective space, at the same time it suggest that specific additional discriminations are necessary to improve descriptive fit to data and solve inconsistencies and confusions. We demonstrate how proactive and reactive actions and controls can be confused, and that this has many potential implications for psychology and neurobiology. We uncover conceptual problems regarding constructs such as effortful control, positive affect, approach-avoidance, extraversion, impulsivity, impulse-control, and goal-directedness of behavior. By delineating those problems, our approach also opens up ways to tackle them.

Introduction

The present paper proposes that four neuromodulator systems underpin highly generalized behavioral sets, but each targets either dorsomedial or ventrolateral cortical systems, where it produces its effects in either a proactive or reactive orientation to the environment. This way systems are discriminated that control reactive approach (dopaminergic), reactive avoidance (cholinergic), proactive behavior (noradrenergic), and withdrawal (serotonergic). This model is compared with models of temperament, affect, personality, and so-called two-system models from psychology. Although the present model converges with previous models that point to a basic scheme underlying temperamental and affective space, at the same time it suggest that specific additional discriminations are necessary to improve descriptive fit to data and solve inconsistencies and confusions. We intend the model as a step toward a comprehensive scheme to make sense of multiple cognitive, motivational, and neural processes that must work together and also be distinguishable. The comprehensive scope of the model makes it useful for condensing the highly elaborative and often conflicting findings concerning motivational sets, cognitive controls, personality patterns, and their neuroanatomical and chemical substrates. In the discussion we demonstrate how proactive and reactive actions and controls can be confused, and that this has many potential implications for psychology and neurobiology.

In Section “The Proactive and Reactive Behavioral/Physiological Programs” we will start by describing separate brain systems for proactive and reactive modes of behavior control. Next, in Section “Motivational Aspects of Neuromodulator Systems”, we outline the neuromodulator systems that are, based for a large part on research with animals, hypothesized to be involved in motivation and behavior such as approach (the catecholamines, dopamine, and noradrenalin), avoidance (acetylcholine and noradrenalin), and withdrawal (serotonin). After that, in Section “Neuromodulation of Proactive and Reactive Behavioral Programs”, the different motivations and behaviors related to the neuromodulator systems will be united into a single framework with the distinction between proactive and reactive behavior control, and this framework or model will be compared to models of temperament and self-regulation. Finally, in Section “Conceptual Issues Raised by the Model” we will discuss important conceptual issues that are raised by the model.

The Proactive and Reactive Behavioral/Physiological Programs

Traditional hypotheses described ascending neuromodulatory systems as “state-setting” or “gating” systems, to regulate arousal or the readiness for cortical information processing (e.g., Pribram and McGuinness, 1975). Such descriptions have proven to be incomplete, and recent experiments indicate that in addition to the relatively slow, or tonic, changes in the activity of neuromodulator systems (over minutes), faster, transient, or phasic, components of activity are evoked by defined cognitive and behavioral activities. Therefore, in addition to the more global regulation of arousal, neuromodulators appear to influence and perhaps even initiate the processing of highly specific cognitive operations (Briand et al., 2007). We think both aspects of neuromodulator function are covered by a biologically plausible hypothesis that neuromodulators are involved in the activation of “behavioral programs” that orchestrate different aspects of behavioral and physiological control systems into a unified program adapted to a particular set of contexts and conditions. These aspects of control systems include arousal, information processing biases, action control, specific cognitive operations, and importantly, specific motivation (Tucker and Williamson, 1984).

In Section “Proactive and Reactive Behavioral Programs for Different Environmental Conditions” we will describe brain systems for reactive and proactive modes of behavior control. Because the distinction between dorsal proactive and ventral reactive systems is based for a large part on animal learning or memory research, we will discuss the relationship with memory systems in Section “The Proactive and Reactive Behavioral Programs and Memory”, and we will discuss the consequences for cognition in Section “The Proactive and Reactive Behavioral Programs and Cognition.” Building on the distinction between proactive and reactive systems, we will propose separate behavioral programs for proactive approach and avoidance (noradrenergic), reactive approach (dopaminergic), reactive avoidance (cholinergic), and withdrawal (serotonergic) in Section “Motivational Aspects of Neuromodulator Systems.”

Proactive and reactive behavioral programs for different environmental conditions

A prominent example of behavioral programs can be found in animal temperament research, where proactive and reactive temperaments are found across different vertebrate species and related to differences in neuromodulator and stress response systems. These temperaments seem to play a role in the population ecology of the species, proactive temperament being more adaptive in stable, high-predictability environments while reactive temperaments are more adaptive in changing, low-predictability environments (Koolhaas et al., 1999, 2007). High predictability environments are those wherein models can be constructed that predict which actions will be effective in a given context. Only in a predictable environment can behavior be guided by a context model; in a low-predictable environment behavior should be guided reactively by environmental feedback-control (Daw et al., 2005). Tucker and colleagues developed a theory of brain systems that control the proactive and reactive programs. Inspired by the work of Pribram and McGuinness (1975) and McGuinness and Pribram (1980), they first related the proactive or extraverted temperament to systems preferably modulated by noradrenalin and serotonin, and the reactive or neurotic temperament to systems preferably modulated by dopamine and acetylcholine (Tucker and Williamson, 1984).

Based on neurophysiological and animal learning research Tucker, Luu, and colleagues subsequently developed their theory further to incorporate dorsal and ventral limbic–thalamic–cortical control paths (Tucker et al., 1995; Tucker and Luu, 2007). In the mammalian brain, two separate control paths are routed from limbic networks through the frontal lobe to motor cortex (see Figure 1). A ventrolateral pathway proceeds from olfactory cortex through the orbital frontal lobe to lateral frontal cortex before reaching the ventral premotor and motor cortices (Goldberg, 1985; Passingham, 1987). This ventral pathway appears to provide greater external constraint (i.e., reactivity) to motor control, in which external cues set criteria for ongoing evaluation of the action progress. A mediodorsal pathway proceeds from the cingulate gyrus through medial frontal cortex to dorsolateral frontal cortex to the premotor and motor areas on the lateral convexity of the hemisphere (Goldberg, 1985; Passingham, 1987). This dorsal pathway appears to provide a projectional (i.e., proactive) mode of behavior control, in which context models and predictions guide the action toward a goal.

Figure 1

Left: primary direction of corticolimbic traffic for organizing output from limbic integration toward specific action modules in the motor cortex. Two separate control paths are routed from limbic networks through the frontal lobe to motor cortex. A ventrolateral pathway proceeds from olfactory cortex through the orbital frontal lobe to lateral frontal cortex before reaching the ventral premotor and motor cortices (ventral/bottom arrows). A mediodorsal pathway proceeds from the cingulate gyrus through medial frontal cortex to dorsolateral frontal cortex to the premotor and motor areas on the lateral convexity of the hemisphere (upper/dorsal arrows). Right: primary direction of corticolimbic traffic for integrating perception from specific modules in the sensory cortex (in this case the arrows start from the visual area) toward the limbic cortex shown for dorsal (upper arrows) and ventral (bottom arrows).

The proactive and reactive behavioral programs and memory

Drawing from Gabriel's (1990) analysis of the roles of the major limbic–thalamic–frontal circuits in animal learning, Tucker and colleagues theorized that the proactive mode of action regulation is linked closely to the operation of the mediodorsal corticolimbic pathway in the maintenance of the context for action and expectations based on these actions. Gabriel cites evidence that the dorsal pathway, with important control from the hippocampus, posterior cingulate, and dorsolateral cortex, maintains a cognitive representation of the context for action. This context model can be adjusted only gradually, through a process that may be described as context-updating (Tucker, 2001; Luu and Tucker, 2003a,b; Luu et al., 2004). By contrast, when events are discrepant with expectancies, either from stimulus-associations or context model, a more rapid, focused form of learning is engaged, in which previous associations are disrupted and a new set of contingencies may be attended effectively. Gabriel's (1990) research has suggested that this new learning in response to context violation requires the ventral circuit that involves the amygdala, ventral striatum, anterior cingulate cortex (ACC), mediodorsal nucleus of the thalamus, anterior temporal, insula, and the orbitofrontal and ventrolateral prefrontal cortices (Price, 1999; Bussey et al., 2001; Phillips et al., 2003a; Saleem et al., 2008). See Table 1 for an overview of central characteristics of the ventral and dorsal systems and their associated behavioral programs.

Table 1

Characteristics of the ventrolateral and mediodorsal systems.

	Ventrolateral system	Mediodorsal system
Origin	Paleocortical	Archicortical
Visceral functional base	Viscerosensory	Visceromotor
Adaptive in environment	Low-predictable/changing	High-predictable/stable
Cognitive mode	Object	Configural
Motor control	Feedback/reactive	Feedforward/proactive
Learning stage	Early	Late
Working memory bias	Redundancy	Habituation
Attention	Focused	Global
Neuromodulators	Dopamine, acetylcholine	Serotonin, noradrenalin

This model of learning bears resemblance to the model of memory systems proposed by O'Keefe and Nadel (1978) and Nadel (1992). According to those authors, several factors distinguish hippocampal based (“locale”) learning from non-hippocampally based (“taxon”) learning. Importantly, locale learning is assumed to be quite different from taxon learning with regard to the underlying systems of motivation that drive it. O'Keefe and Nadel argued that there is a fundamental connection between locale learning and exploration. The drive to acquire information, in the first instance about one's environment, is taken as the force underlying locale learning, just as it has to be in the proactive system to enable the construction of context models. The drives of reactive systems, such as hunger and thirst, are not considered to be important in this system, though information about location of food, water, mates, and safety might well be part of what is acquired. Taxon learning, on the other hand, is assumed to be motivated by the traditional drives (e.g., to exploit) emphasized by Hull (1943), and therefore to be dependent on the standard application of reinforcements. The locale system is the basis for providing the context within which context-free information from the taxon systems could be situated. In contrast with those similarities between respectively locale and taxon systems on the one hand, and proactive and reactive systems on the other hand, quite opposite to the model of Tucker and colleagues, O'Keefe and Nadel assume locale learning to be fast and all or none, while taxon is assumed to be slow and incremental. However, O'Keefe and Nadel acknowledge that there are examples of taxon learning that are very fast and can occur with but a single pairing. We think this seeming contradiction may be explained by the focus in the research of O'Keefe and Nadel on hippocampal function, which reflects only one part of the proactive system, and mediates only aspects of proactive learning function, such as perhaps an episodic buffer. O'Keefe and Nadel compared only one part of one system, the hippocampus, with everything that is non-hippocampal (Keren and Schul, 2009).

The association of the dorsal pathway with gradual learning and context model-guided action control suggests that this pathway may be relatively more efficient in retrieval compared to encoding. By contrast, the association of the ventral pathway with rapid learning and action control by external constraint suggests that this pathway may be relatively more efficient in encoding compared to retrieval. Indeed, recent fMRI studies support this. For instance, left ventrolateral prefrontal cortex was associated more with successful encoding, whereas left dorsolateral prefrontal cortex was associated more with successful retrieval (Prince et al., 2005; Kim et al., 2010). In a meta-analysis of five different fMRI studies of episodic memory, Daselaar et al. (2009); also Binder et al. (2010) and Kim et al. (2010) found that successful retrieval was associated with increased activity in posterior cingulate and precuneus, and posterior lateral parietal cortex, whereas successful encoding was associated with decreased activity in these regions. Encoding success and novelty detection overlapped in a ventrolateral anterior insula/inferior frontal gyrus (IFG) area (Kim et al., 2010).

The proactive and reactive behavioral programs and cognition

The posterior cingulate and Papez circuits of the dorsal networks support a positive memory bias that forms the context representation and expectancy for approach and reward (see Deakin, 2003). The fundamental motive tone for this control system leads actions to be initiated through proactive (i.e., context-predicted) control (Tucker et al., 1995). With preferential neuromodulation of the dorsal limbic and neocortical networks by noradrenalin (Foote and Morrison, 1987), the habituation bias integral to noradrenergic modulation (Tucker and Williamson, 1984) may be a mechanism through which global attention, configural working memory, and gradual learning are incorporated within the context model with minimal disruption of the representation (Luu and Tucker, 2003a,b). Noradrenalin is distributed through the dorsal pathways (Morrison and Foote, 1986), and regulated by those pathways (Arnsten and Goldman-Rakic, 1984). Indeed, noradrenergic and serotonergic antidepressants increase positive biases in the processing of emotional material (e.g., Harmer et al., 2004; Serra et al., 2006; Norbury et al., 2007) and appear to shift activity from ventral to dorsal networks (see Tucker and Luu, 2007; Carver et al., 2008; Tops et al., 2009).

By contrast, the recognition of discrepancy or threat, mediated by the ACC and IFG with inputs from reactive ventral limbic networks (Corbetta and Shulman, 2002; Sridharan et al., 2008), leads to a suppression of the context model, an engagement of focused attention, focus on and awareness of the present moment, and the capacity for rapid reorganization of behavioral contingencies. Neuromodulators of the ventral system provide a redundancy bias to working memory (Tucker and Williamson, 1984), focusing attention on potential discrepancies, threats, and rewards, and mediating the frustration that may guide further learning and adaptive control. This redundancy bias in working memory also supports perseverative cognition (Brosschot, 2010) such as apprehensive worry and rumination. In humans, in the more subtle actions of this regulatory influence, the vigilance for discrepant events leads to a focused, analytic mode of cognition that is congruent with highly constrained, feedback control of ongoing cognition and action.

In summary, focusing on the contrast between noradrenergic and dopaminergic modulation (Tucker and Williamson, 1984), the dorsal or noradrenalin system supports perceptual orienting, yet this system achieves its major attentional control through increasing habituation. The system responsible for augmenting neural activity in response to perceptual input thus simultaneously decrements further response to input to facilitate proactive and context model-guided control. The ventral or dopaminergic system, which increments brain activity to support motor output and readiness, also facilitates a tight momentary control of behavior by a continuous stream of external or internal stimuli, or inhibition of behavior during redundant processing of aspects of stimuli.

Although ventral and dorsal systems will usually work in parallel and in interaction for many tasks, stable, and dynamic biases toward one or the other system may explain temperamental and state variation, respectively, in tendencies toward reactive or proactive programs. This claim is supported by a recent fMRI study, in which subjects were scanned while they adopted either a reflective, extended self-reference linking experiences across time in memory (which may involve the dorsal system) or a momentary experiential self-reference centered on the present moment (ventral reactive). The experiential focus yielded reduced activity in dorsal system areas such as medial prefrontal cortex, posterior cingulate cortex and hippocampus, and increased engagement of ventral areas such as in the insula, secondary somatosensory cortex, ventrolateral prefrontal cortex and inferior parietal lobule. These effects were largest in subjects that were trained to develop focused attention on the present. Functional connectivity analyses further demonstrated a strong coupling between the insula and the ventromedial prefrontal cortex in untrained subjects that was uncoupled in the trained group (Farb et al., 2007). This decoupling may reflect increased ability in trained subjects to avoid rumination by disengaging attentional processes of self-referential elaboration (Farb et al., 2007). These results suggest a fundamental neural dissociation between two distinct forms of self-awareness consistent with the dorsal and ventral programs, which are habitually integrated but can nevertheless vary within and between individuals in relative activation. For a review of neuroimaging studies showing the role of the insula/IFG area in awareness of the moment and emotional intensity, see Craig (2009); for a review of neuroimaging studies of the relations between dorsal areas and reflective, planfull behavior vs. ventral areas and reactive behavior, see Carver et al. (2008).

Motivational Aspects of Neuromodulator Systems

The classic neuromodulator systems that project from the brainstem and basal forebrain, the noradrenergic, serotonergic, cholinergic, and dopaminergic systems, are very complex and each has been related to a multitude of functions at various levels (e.g., physiological, behavioral levels) that often seem unrelated. On the other hand, functions at different levels that seem unrelated at first sight may be part of a common behavioral and physiological program. For behavioral scientists who want to constrain their behavioral models with knowledge of brain function, characterization of behavioral programs in terms of associated motivational aspects seems to be most important. Although functions, modules and other units may be grouped into physiological and even behavioral programs without aspects of motivation being involved, the phylogenetically old brainstem neuromodulator systems seem plausible candidates in the search of basic motivated programs. Indeed, despite above complexities, for each of the neuromodulators there are reviews that suggest a function in aspects of motivation.

Considering elementary control functions that have been conserved through evolution as part of behavioral programs, we propose each of the brainstem neuromodulators to have neuromodulatory function that can best be conceptualized as a phylogenetically conserved drive, “functioning in a higher-order capacity to integrate a variety of behavioral functions” (Lucki, 1998). Each of the neuromodulator systems cover a large area of influence from a central point of origin, which is suitable for a general regulatory system (Clark, 1979). The projections of those systems apply a neurophysiological modulatory influence, and this has very general, vectoral drive properties, meaning that behavior is oriented in a certain direction. A general phylogenetically conserved drive or behavioral program theory of neuromodulator function in behavior can help to account for why each neuromodulator appears to influence so many behaviors, but also is an unlikely neurotransmitter to be the principle or sole mediator of any of these behaviors (Lucki, 1998).

The catecholamines dopamine and noradrenalin: different drives to approach

Activity in midbrain dopamine neurons has been shown to be related to signaling the rewarding value of events and actions (Schultz, 2002). Most midbrain dopamine neurons exhibit burst activity following delivery of primary rewards. They respond to both rewards (with activation) and punishments/reward omission (mostly with depression; Ungless et al., 2004). This dopaminergic activity, however, appears to depend on the predictability of the reward, such that unpredicted rewards elicit dopaminergic activation and an unpredicted non-reward induces a depression in dopamine activity, while fully predicted rewards do not elicit dopamine activity (Schultz, 2002). The dopaminergic response has been associated with the “wanting” or “drive” aspect of reward processing that facilitates approach, or frustration when attractive stimuli are out of reach, and not so much with the “liking” aspect (Berridge, 2007). For instance, in an fMRI study of cocaine-dependent subjects drug-induced “high” feeling was found to correlate negatively with activity in regions including the ventral striatum (nucleus accumbens), ACC and IFG while craving correlated positively with activity in these regions (Risinger et al., 2005).

The noradrenalin-driven locus coeruleus is believed to be aroused in active coping efforts to maintain or gain control (Henry, 1993) and in response to unexpected uncertainty (Yu and Dayan, 2005). High noradrenergic activity has been related to aggression and assertiveness (Haller et al., 1998; Tse and Bond, 2006), possibly because aggression is often involved in the (re)gaining of control and proactive coping (Henry, 1993). McClelland et al. (1980) proposed that brain noradrenalin mediates a power drive, i.e., a drive to gain control. The locus coeruleus modulates vigilance and the initiation and energizing of adaptive behavioral responses (Aston-Jones et al., 1994).

Because many readers will associate noradrenalin with stress-responding and anxiety, it is important to discuss the differences between the (pro)active avoidance behaviors that appear to be modulated by noradrenergic systems, and passive, inhibitory and reactive avoidance and withdrawal that will be discussed in the next two sections. In the present framework the association of noradrenergic function with active coping is explained by the guidance of avoidance behaviors by context models. The encoding of adaptive active coping behaviors in context models and the associated positive expectancy bias that these behaviors will be successful, facilitates adaptive, context-dependent proactive coping responses guided by those context models (Tucker et al., 1995; Koolhaas et al., 1999). This is associated with high arousal and alertness, but at the same time not directly controlled by environmental stimuli without involving the context model, and feedback from the stimulus environment should not interfere with the proactive control of motor responding; attention is modulated by the context model instead of exclusively focused on the threat stimulus. In these aspects the noradrenergic proactive avoidance responding is different from the reactive avoidance we will discus in Section “Acetylcholine: The Drive to Avoid.”

The proactive and reactive programs appear to have evolved because they are adaptive in stable, predictable, and in low predictability, changing environments, respectively (Koolhaas et al., 1999). The behaviors that are adaptive in stable vs. changing environments have also been characterized as “exploration” (essential in constructing context models to serve the proactive program) vs. “exploitation” (reactive appetitive), respectively (Cohen et al., 2007). Evidence suggests that dopaminergic systems mediate exploitation (Daw et al., 2006) while noradrenalin neurons in the locus coeruleus have been suggested to mediate shifts between exploration and exploitation (Usher et al., 1999). Alternatively, if noradrenalin in the locus coeruleus is involved in a proactive program, then it may apply network reset and modulate switching between goal-directed and exploratory behaviors, which is necessary for switching between proactive goal-behavior repertoires, so as to permit rapid behavioral adaptation to changing environmental imperatives and context switches. It may be that unexpected changes in the world – within the context of a proactive behavioral program – activate a noradrenergic interrupt signal in order to search for alternative behavioral repertoire within the proactive program (Usher et al., 1999; Yu and Dayan, 2005). On the other hand, locus coeruleus noradrenalin may be involved in switching between the proactive control phases of deliberation and post-decision implementation (Einhauser et al., 2010).

The dopaminergic and noradrenergic systems are different systems with different drive functions, which we characterize, following the discussion in the previous section, as “reactive approach” and “proactive approach and avoidance”, respectively. However, they seem similar in the way they oppose, and are opposed by, the serotonergic and cholinergic systems, and energize behavior (e.g., Mawson, 1999). We think these similarities together with other factors are the cause of conceptual confusion we return to in Sections “Neuromodulation of Proactive and Reactive Behavioral Programs” and “Conceptual Issues Raised by the Model.”

Acetylcholine: the drive to avoid

Especially in reactive systems, the drive to approach seems in natural opposition to a drive to avoid. Acetylcholine seems to be implicated in a drive to avoid, i.e., in anxiety. Brainstem cholinergic projections induce a rapid, transient elevation of vigilance level by their phasic response to novel, unfamiliar stimuli (Kayama and Koyama, 2003). Basal forebrain cortical cholinergic activity may foster the attentional processing of threat-related stimuli and associations, and thereby contribute to cortical/cognitive aspects of anxiety (Berntson et al., 2003). In healthy male volunteers the behavioral and cardiovascular sensitivity to a centrally active cholinergic stimulant correlated significantly with “irritability” and “emotional lability” as well as with habitually passive strategies in stress coping (Fritze et al., 1995). Acetylcholine has been proposed to signal expected uncertainty, coming from known unreliability of predictive cues within a context (Yu and Dayan, 2005) and hence may, like dopamine, bias behavior toward ventral system control that is adaptive in changing, low predictable environments.

Although the striatum has been strongly implicated in dopaminergic reward processing, it should be noted that evidence indicates that cholinergic and dopaminergic systems work together to produce the coordinated functioning of the striatum (Zhou et al., 2003). Especially in the striatum, the dense mingling of dopaminergic and cholinergic constituents enables potent interactions. Moreover, acetylcholine-mediated mechanisms might be of crucial importance in processing the cortical inputs to the striatum (Calabresi et al., 2000). A recent study found that cholinergic but not dopaminergic projections to the basal ganglia carry reward omission information (Joshua et al., 2008). A role of a cholinergic activation system in facilitating detection of stimuli associated with punishment, failure, or reward omission has been suggested previously (Gray, 1989; Boksem et al., 2006). Similarly, Sarter et al. (2006) review evidence indicating that increases in the activity of cortical cholinergic inputs represent a major component of the neuronal circuitry mediating increases in what they call “attentional effort.” These cortical cholinergic inputs mediate effortful cognitive control that is engaged when negative or aversive events and response outcomes signal that goals are not being achieved, making effort necessary. Such negative events may encompass aversive outcomes such as error detection, reward loss, and punishment (Sarter et al., 2006).

In light of the wide-spread cholinergic projections to the cortex and its involvement in many aspects of cognition, it is important to stress that a label such as “avoidance” may not do justice to the complexity of cholinergic functions. We only use labels such as approach and avoidance to stress motivational aspects and to position the programs in relation to papers in which these labels are often mentioned. We envision that cholinergic systems have a general function, perhaps comparable to noradrenalin, but complementary to it in the sense that it is associated with reactive control. The contrast between central noradrenergic and cholinergic function may be similar to their peripheral functions in the sympathetic and parasympathetic autonomic nervous system, respectively. The sympathetic system provides a global, feed-forward, proactive response to challenge, while the parasympathetic system fine-tunes the response, more tightly guided by detailed environmental feedback control (Porges et al., 1996). The reactive avoidance behavioral program is a program for low-predictable environments or circumstances, in which threats are judged to outweigh opportunities. This situation, and associated kind of control, may have been dominant in many species, environments, and periods in evolution, and this behavioral program may have developed into a very general and versatile program that is nevertheless biased toward reactive avoidance. Moreover, the more recent cognitive control elaborations of the ventral systems provide essential capabilities to present day humans.

In our model, reactive avoidance stands out from the other motivations in terms of motivational and affective biases: reactive avoidance is associated with negative affective biases (Tucker et al., 1995), while approach and (pro)active avoidance are facilitated by positive biases (Carver et al., 2000). Indeed, noradrenergic and dopaminergic neuromodulation seem to be characterized by high levels of subjective energy and social activity, positive emotional biases or defenses, and reward orientation and impulsivity; also serotonin has been related to positive emotional biases, positive social engagement, social potency, and aspects of dominance (see Nutt et al., 2007; Carver et al., 2008; Tops et al., 2009; Sections “The Catecholamines Dopamine and Noradrenalin: Different Drives to Approach” and “Serotonin: The Drive to Withdraw”). These biases are opposite to, or contrast as very different from, reactive avoidance or neuroticism. By contrast, while antidepressants are used that stimulate noradrenalin, serotonin or dopamine, recently anticholinergic antidepressants are developed (Howland, 2009). The cholinergic–adrenergic hypothesis of mood disorders states that depression would be the clinical manifestation of a state of cholinergic dominance, whereas mania would reflect noradrenergic dominance (Fritze et al., 1995; Howland, 2009). In conclusion, of the neuromodulators considered, only acetylcholine seems to fit the profile of a neuromodulator of a reactive avoidance program. However, in future work the distinction between proactive and reactive avoidance should be worked out further, so as to determine whether noradrenergic systems cannot account for (aspects of) reactive avoidance as well.

Serotonin: the drive to withdraw

We have proposed that serotonin functions as a neuromodulator of a drive to withdraw: a phylogenetically conserved motive to reduce the present or anticipated environmental stimulation mentally or behaviorally, such as by moving into an environment of lower stimulation levels (Tops et al., 2009; see also Lowry et al., 2009). Serotonin exerts an inhibitory influence on behavior (see Lucki, 1998) and decreases responsiveness to motivational stimuli, increasing restraint by allowing for responding to cues of longer-term outcomes and delay of gratification (Depue, 1995; Carver and Miller, 2006). Functional roles of serotonergic projections from the dorsal raphe nucleus to upper brain structures have been investigated by recording neural activity in this nucleus, and by observing effects of stimulation of this nucleus (Kayama and Koyama, 2003). According to these authors, action on upper brain is inhibitory in spite of waking-specific activity of the neurons. The serotonergic systems serves to dampen and oppose the actions of the dopaminergic, noradrenergic, and cholinergic systems, for example by promoting behavioral inhibition and cortical de-arousal (Robbins, 1997). Thus, serotonin promotes satiety, sleep, quiet non-aroused waking, parasympathetic activation, and the anti-stress relaxation response. Low serotonin and (hypothetically) frustration of the drive to withdraw will increase irritability (Tops et al., 2009).

Ellison (1979) showed that the low-serotonin animal can be thought of as being in a state of central functioning appropriate for any animal out in the environment, foraging for food: it is hyper-aroused, sensitive to stimulation and vigilant. Furthermore, Ellison suggested two antagonistic types of positive affect (drives): one which pulls the animal out of hiding into the environment by positively rewarding it when it engages in appetitive consummatory responses (catecholaminergic), and another which pulls it back into the security of the nest by satisfying a reciprocal set of needs (serotonergic). The positive affects that Ellison associated with serotonergic function were security and relaxation, which are proposed to serve functions of energy conservation and recuperation.

Another function of serotonin may lay in its relative promotion of dorsal systems and proactive behavior. It has been suggested that serotonin facilitates motor output, partly by suppressing ongoing processing of sensory input that might disrupt motor output, thereby effectively facilitating proactive action control (Jacobs and Fornal, 1995). We proposed that serotonin facilitates a mode of proactive function that guides behavior that is best performed without interference from high levels of unpredictable environmental stimulation (Tops et al., 2009). Most activities reflecting attachment behavior are most successfully maintained when the organism is relatively relaxed and free from challenge by the need for self-preservation. Serotonergic sanguinity and comfort may be important to the facilitation of social interactions by reducing the associated anxiety and inhibition. We proposed that serotonin is involved in the type of social interactions, in which immediate reward value is traded for delayed rewards. To do so, serotonin decreases both aversive and appetitive reactivity. The drive for these low-arousal social behaviors may have been derived from a serotonergic drive to withdraw into a safe place or comfort. We speculate that the development of phylogenetically more recent social and cognitive functions may have led to the increasing complexity and number of serotonergic system receptor sub-types and some co-selection of both ventral and dorsal elements in most recent sub-systems.

Other motivational functions of serotonin have been suggested in the literature. Serotonin has been suggested to be implicated in harm avoidance and anxiety (e.g., Deakin, 2003). However, in that literature the drive to avoid is not discriminated from the drive to withdraw. We argue elsewhere that serotonergic function is more likely to relate to a drive to withdraw than to a drive to avoid (Tops et al., 2009, also for references to other authors arguing against direct involvement of serotonin in a drive to avoid or anxiety). Another hypothesis, that serotonin decreases immediate stimulus reactivity and increases future orientation, which may promote behaviors that could be described as proactive avoidance (Carver et al., 2008, 2009), we think is compatible with the present one. However, we think a role in future orientation is derived from a phylogenetically more basic function in withdrawal: perceptions of environmental resource availability and feeding state motivate organisms to withdraw in order to conserve energy and recover, while in more recent evolutionary history additionally future oriented behaviors and cognitions may be facilitated (Russo et al., 2009; Tops et al., 2009).

Neuromodulation of Proactive and Reactive Behavioral Programs

We now turn to models of temperament and self-regulation. We do this because we believe that combining (a) the four behavioral programs related to the neuromodulators with (b) the division into proactive and reactive behavioral programs, produces a model that is very similar to a model that has recently been proposed to integrate literature on temperament and self-regulation systems, serotonin function, psychopathology, and neuroimaging (Carver et al., 2008, 2009). However, we think our behavioral programs approach has additional strengths compared to theirs. Specifically, we think our model uncovers important conceptual problems and also opens up ways to address those problems. Tackling conceptual issues will also help in comparing our approach to other research that suggested related distinctions between dorsal and ventral networks associated with top-down/goal-directed vs. stimulus-driven attention, executive control vs. salience-detection, attention vs. emotion, aspects of emotion perception, resting state connectivity, visceromotor vs. viscerosensory function and psychopathology, although it is beyond the space limits of this paper to address all those comparisons directly (Price, 1999; Corbetta and Shulman, 2002; Yamasaki et al., 2002; Phillips et al., 2003a,b; Fox et al., 2006; Seeley et al., 2007).

Figure 2 is a depiction of the two reactive motivational systems, i.e., the dopamine-linked reactive approach system and the acetylcholine-linked reactive avoidance system, and of the noradrenalin- and serotonin-linked proactive approach and avoidance system. Inspection of this figure may raise the question why it depicts separate reactive approach and reactive avoidance systems, but not separate proactive approach and proactive avoidance systems. One explanation for the lack of evidence for separate proactive approach and proactive avoidance systems may be that the involvement of the ventral systems in acute external reactivity and reactive responding to threat may induce time constraints that are best met by separate systems for the fundamentally different behaviors. By contrast, proactive approach and proactive avoidance may be guided by the same context models. Relatedly, proactive approach and avoidance may not be opposites as much as their reactive counterparts are: proactive avoidance of threat involves active problem-solving, a coping style that has even been termed “approach coping” (Roth and Cohen, 1986). Indeed, individuals who score high on measures of adaptive optimism that most likely reflect a bias toward proactive behavioral programs (see below) have been shown to confront both pleasant and unpleasant emotions and threat stimuli to serve active problem solving (Aspinwall et al., 2001). Finally, reactive approach and reactive avoidance behavioral programs may have evolved because they represent two alternative adaptive strategies in low-predictability environments. In such environments, reactive harm avoidance may evolutionary have been more adaptive for women who care for vulnerable children and family, whereas in such environments reactive approach may be more adaptive for men who hunt and compete for physical and social resources (Del Giudice, 2009), which may explain higher prevalence of internalization disorders and reactive avoidance temperament in women, and externalization disorders and reactive approach temperament in men (e.g., Oldehinkel et al., 2007).

Figure 2

Three temperamental influences (and associated neuromodulators) on behavior that reflect different behavioral programs. A reactive system for approaching rewards and a reactive system for avoiding threats or punishment both interact, collaborate and compete with, and are dampened by a proactive system guided by context models or effortful control. This figure is inspired by Figure 1 in Carver et al. (2008, 2009) who in turn noted inspiration by Rothbart and others. NA, noradrenalin; DA, dopamine; ACh, acetylcholine; 5HT, serotonin.

Interestingly, although Figure 2 shows the reactive behavioral programs and their interaction by a context-model-guided proactive program, this figure is actually adapted from a figure in Carver et al. (2008, 2009); compared to that figure, we added “proactive approach and avoidance” to describe the box that originally only said “(deliberative) effortful control.” Carver and colleagues presented this model to explain aspects of serotonin function and psychopathology. They based their perspective on a set of theories from cognitive, social, personality, motivational, and developmental psychology that are often called two-system or two-mode models. Such models converge on the idea that there exist two modes of processing information and regulating action, which operate simultaneously and in competition with each other: a lower-order system that responds quickly to associative cues of the moment and a higher-order system that responds more reflectively and planfully. However, for the terms they used in their figure (see Figure 2) they chose a particular temperament model of self-regulation from developmental psychology (Derryberry and Rothbart, 1997; Posner and Rothbart, 2007). Additionally, they discuss evidence that those systems involve ventral, respectively dorsal cortical areas. And similar to what we proposed, they suggest that serotonin shifts activity from ventral to dorsal systems, i.e., that low serotonin is linked to relative dominance of the reactive systems (Tucker and Luu, 2007; Tops et al., 2009). They also suggest, as we did, that the kind of behavior manifested when serotonin is low, depends on biases toward either reactive approach or reactive avoidance systems (also Prange et al., 1974). Starting from a different literature than we do, they arrive at a similar explanation for different forms of psychopathology that seem associated to low serotonin (Tops et al., 2009).

Research in children and adolescents, some of which based on the temperament questionnaires developed by Rothbart and colleagues, found support for the model in Figure 2. For instance, there is significant covariation between internalizing and externalizing behavior that is partly explained by individual differences in emotional reactivity, although there is also evidence that internalizing behavior is a protective factor against externalizing behavior (Rhee et al., 2007). This is consistent with internalizing and externalizing both being mediated by reactive systems and dampened by the proactive system or effortful control, but at the same time externalizing behavior is inhibited or opposed by the reactive avoidance system if punishments are involved (see below and Figure 3). In a large population sample (n = 2230) studied at preadolescence and again as adolescents (Oldehinkel et al., 2004, 2007), fearful and shy temperaments were associated with internalizing disorders, and frustration and high-intensity pleasure temperaments were associated with externalizing disorders. In adolescents both associations were attenuated by high levels of effortful control (Oldehinkel et al., 2007). Additionally, high frustration or irritability tended to be associated with both categories of disorders (Oldehinkel et al., 2004, 2007; Baldwin and Dadds, 2008). This fits with the hypothesized association of both categories of reactive temperament and disorders with low serotonin; low serotonin is associated with increased emotional reactivity, the evidence appearing strongest for irritability (Russo et al., 2009).

Figure 3

Different kinds of self-regulation after adding effortful control to the temperamental influences. We introduced an “effortful control” element to each program from Figure 2. Taking as example the self-regulation of reactive approach (i.e., impulsive reward responding), this model suggests that reactive emotional behavior can be regulated in at least three different ways: (1) by regulation from the proactive system; (2) by opposition from the other reactive system; (3) by effortful control (constraint) within the reactive system itself.

Conceptual Issues Raised by the Model

Although there is a lot of conceptual confusion and imprecision in the two-system model literature (see Keren and Schul, 2009, and the discussion below), most of the evidence reviewed by Carver et al. (2008, 2009) seems to support our model as well. However, there are a few differences in our models and approach. We claim that our model using behavioral programs points at conceptual problems that may affect the model of Carver et al., as well as other behavioral research. We will discuss a few examples below of issues raised by our model.

Higher-order effortful control or both reactive and proactive effortful control

Carver et al. (2008, 2009) conceptualize the proactive system as being at a “higher order” than the reactive system. This is similar to tendencies in two-system models and psychology in general to generate contrasts based on dichotomies such as cognitive–emotional, controlled–automatic, positive–negative, approach–avoidance, hot–cold etc. (Keren and Schul, 2009). However, the behavioral/physiological programs approach may trigger new insights because it suggests that elements that are not intuitively similar, connected, or at a similar level of complexity (perhaps even opposites in some ways) may nevertheless be associated with the same system: they may be associated with the same system because they are elements of the same program that is adaptive in a particular environment. In other words, criteria for elements to be associated to the same psychologically relevant system are derived from their role in an adaptive program instead of from intuitions of what is similar, compatible, or comparable.

For instance, although the reactive programs in the model of Tucker and colleagues include elements that appear more primitive (e.g., impulsive aspects of reactivity discussed below) and indeed may be of phylogenetically older origin and develop to completion at a younger age compared to the proactive system (Flechsig, 1901), Tucker and colleagues actually argue that in humans, in the ventral reactive regulatory influence, the working memory redundancy bias and vigilance for discrepant events leads to a focused, analytic mode of cognition (Tucker and Williamson, 1984). Indeed, absorption, which reflects reactive attention focused on the present and intense emotion, is associated in healthy individuals with increased working memory capacity and elaboration learning but relative performance deficiencies in tasks of memory for associative, context-dependent verbal material, visuospatial working memory, and executive control functions in terms of a heightened perseveration tendency and false positive errors (de Ruiter et al., 2006; Amrhein et al., 2008); we think this pattern fits with a relative bias toward reactive and away from the proactive program. We think that each program incorporated in its more recent elaborations elements of cognitive control. At the neurophysiological level this may often involve prefrontal cortex elaborations and different receptor subtypes (Durstewitz and Seamans, 2008). That is why we think detail and explaining power can be added to the model in Figure 2 by introducing an “effortful control” element to each program (Figure 3).

For comparison and consistency with Figure 2 and other literature we stayed with the term “effortful control” (Derryberry and Rothbart, 1997; Posner and Rothbart, 2007; Carver et al., 2008, 2009), although we suggest that subjectively effort may be experienced differently under each program. Sarter et al. (2006) review evidence indicating that increases in the activity of cortical cholinergic inputs in response to negative events and response outcomes represent a major component of the neuronal circuitry mediating increases in “attentional effort” or effortful cognitive control. Sense of effort and physical exertion relate negatively to extraversion and positively to neuroticism, anxiety, depression (Morgan, 1994), and insula activity (Williamson et al., 1999; de Graaf et al., 2004). These findings are consistent with acute effort sense being related to momentary awareness and affective intensity of the moment (Craig, 2009) in the reactive avoidance system.

Failure to discriminate reactive positive affect from proactive positive affect

In addition to suggesting that certain elements of programs that seem opposites actually may belong together within the same program, the present model also suggests the opposite: that other elements that are usually not discriminated actually belong to different programs that may even be seen as opposites in some regards. This may be true for two types of positive affect. Evidence suggesting that there are two types of positive affect associated with different programs comes from studies of the relationship between positive affect and cognition. Intuitively researchers have linked positive affect to reward processing and dopaminergic function (e.g., Watson et al., 1999). Other researchers have discussed appetitive or pregoal positive states and “wanting” as being different from consummatory or postgoal positive states and “liking” and argued that dopamine function is only involved in the former (e.g., Berridge, 2007). According to the present model, the reactive approach positive affect (“wanting”) of the ventral system would be associated with focused attention; however, there should be another kind of positive affect related to proactive approach and global attention (Table 1 and Section “The Proactive and Reactive Behavioral/Physiological Programs”). Moreover, whereas there is a positive affective bias associated with the proactive dorsal circuit, in moderate-temperament individuals in situations without strong positive incentives there is a negative affective bias associated with reactive ventral system function (Derryberry and Tucker, 1994). This negative bias to reactive function is supported by studies of dynamics of mood over the day, which show that while positive affect shows a tonic rhythm of increase in the morning and decrease in the evening, negative affect only shows reactivity to events (Watson et al., 1999). The circadian rhythm of positive affect may be modulated by noradrenalin, as noradrenalin has been associated with positive affect (Nutt et al., 2007), tonic activation (Kayama and Koyama, 2003)1 and circadian arousal regulation (González and Aston-Jones, 2006).

Studies generally support the model of Derryberry and Tucker (1994), finding that whereas negative affective states are associated relatively with increased local attention, positive affect is associated with more global attention (see Förster et al., 2006). Importantly, studies that specifically induced reactive approach-related positive affect (i.e., “wanting”, using pictures of desserts presented to hungry volunteers) found reduced global or increased local processing (Gable and Harmon-Jones, 2008; Harmon-Jones and Gable, 2009). The results specifically supports the present model, as it is the first to discriminate reactive approach from proactive approach, and it relates the former to focused attention and the latter to global attention. However, related distinctions between different types of positive affect have been made by others.

Drawing on the review by Derryberry and Tucker (1994), the broaden-and-build theory of Fredrickson (2004) appears to describe proactive system positive affect. According to this theory some positive emotions broaden an individual's momentary thought-action repertoire: joy sparks the urge to play, interest sparks the urge to explore, contentment sparks the urge to savor and integrate, and love sparks a recurring cycle of each of these urges within safe, close relationships. The broadened mindsets arising from these positive emotions are contrasted to the narrowed mindsets sparked by emotions associated with specific action tendencies, such as attack or flee (and reactive approach, we argue; also Gable and Harmon-Jones, 2008). Panksepp (1998) similarly discriminates a play positive affect system and a seeking emotive system involved in reward obtainment. These theories have in common with each other and the present model that they suggest that, in addition to reactive approach positive affect, there are kinds of positive affect that are adaptive in stable, predictable, or comfortable environments and allow for a broadening of attention and cognition (Panksepp, 1998; Carver, 2003; Fredrickson, 2004; Gable and Harmon-Jones, 2008). This broadening of attention in the proactive mode functions to guide play and explorative behavior and integration to construct context models, read contexts and to flexibly and optimally select and switch between context models.

Factor analyses of self-rated mood suggest that affective states can be described by a two-factor model (e.g., Watson et al., 1999). One factor or dimension has been labeled alternatively as “arousal–dearousal” or “engagement–disengagement”; the other dimension has been labeled “pleasantness–unpleasantness.” Alternatively, other authors argue that these factors are best rotated such that affective space is described by the dimensions “positive activation” and “negative activation.” Positive affectivity and negative affectivity are also found as traits that are related to extraversion and neuroticism, respectively, and were suggested to represent the subjective components of broader biobehavioral systems of approach and withdrawal, respectively (Watson et al., 1999; Carver et al., 2000). We think the engagement–disengagement dimension may reflect the balance, as modulated by serotonin, between reactive and proactive systems in Figure 2 (i.e., it may reflect the disengagement of reactive systems involved in the intense experience of the moment, for instance due to increased suppression by serotonin, effecting increased withdrawal), whereas the pleasantness–unpleasantness dimension may reflect the balance between reactive approach and reactive avoidance system activation. By contrast, the rotated solution of negative activation and positive activation may contrast reactive avoidance system activation with activations in reactive approach and/or proactive approach systems that are not discriminated. We think that the present model is consistent with this literature, given the limitations of the used measures and analyses (Carver et al., 2000).

In another approach to the issue of basic dimensions of affective dispositions, two models have been proposed, one by Tellegen (1985), composed of positive emotionality, negative emotionality, and constraint, and the other by Watson and Clark (1993), composed of positive temperament, negative temperament, and disinhibition. These models seem to fit the model in Figure 2 very well, assuming that constraint and inhibition reflect the regulation of the reactive approach and avoidance systems by the proactive system. However, in the next sections we will elaborate further on how our model points at potential confusion between different types of impulse control that may affect constructs such as constraint, impulsivity, and inhibition (Figure 3).

Failure to discriminate proactive and reactive approach, proactive, and reactive extraversion

Because positive affectivity is an important aspect of temperament and personality (Carver et al., 2000), if there are indeed different types of positive affect that belong to different temperament systems, then failure to discriminate them may impair models of temperament. It is thought that positive affect is an important aspect of temperament because it is functionally involved in approach motivation (Carver et al., 2000). Hence, the implications described above of failing to discriminate between reactive and proactive types of positive affect, translate in the realm of temperament and personality research to consequences of failing to discriminate between reactive approach and proactive approach, respectively. This distinction is usually not made in psychology (but see Braver et al., 2009). But there are reasons to believe that this is potentially an important distinction, and that recognition of this distinction may facilitate progress in behavioral- and neurosciences.

Remember that proactive and reactive temperament seems to characterize two basic behavioral programs that are recognizable across vertebrate species (Koolhaas et al., 1999). In humans the Big 2 of personality or temperament research are extraversion (positive affectivity) and neuroticism (negative affectivity). It would make sense to expect, as Tucker and colleagues suggested, that extraversion and neuroticism would reflect the proactive and reactive behavioral programs, respectively (Tucker and Williamson, 1984). However, in addition to indications that extraversion and its associated optimistic bias are indeed related to proactive approach and adaptive context model-guided behavior and lower risk of psychopathology, they are also linked to reactive approach and externalizing behavior (e.g., Taylor et al., 2003). Similarly, optimism may reflect optimism that context models remain efficient in guiding successful behavior, i.e., optimism about predictability, a form of optimism that is necessary for a bias toward proactive behavior; or reflect optimism that the individual has enough resources to control his environment (even when unpredictable) in a reactive, opportunistic way, a form of optimism and positive bias that characterizes narcissistic and the other disorders of the externalizing category. In terms of Figure 2 it appears that reactive avoidance may be contrasted with reactive approach, or at other times with proactive approach, or with a combination of proactive and reactive approach. In other words, it seems that similarities to the human observer between reactive and proactive approach have led to failure of measures of temperament to discriminate them. An observation that could discriminate reactive from proactive extraversion is that neurotic (i.e., reactive) extraverts but not low-neurotic (proactive?) extraverts showed response facilitation when rewards could be obtained (Reed and Derryberry, 1995).

Proactive approach and reactive approach may be hard to discriminate from each other, both by questionnaire measures and observational techniques, because the phenotypical differences between both may often be more subtle compared to the similarities. Like the similarities between noradrenergic and dopaminergic neuromodulation (Nutt et al., 2007), both seem to be characterized by high levels of subjective energy and social activity, positive emotional biases or defenses, and reward orientation and impulsivity; also serotonin has been related to positive emotional biases, positive social engagement, social potency and aspects of dominance (see Carver et al., 2008; Tops et al., 2009; and Section “Serotonin: The Drive to Withdraw”). Both are opposite to, or contrast as very different from, reactive avoidance or neuroticism.

Confusion between proactive and reactive impulsivity, between proactive and reactive impulse control

Impulsivity is a behavioral aspect that may be importantly implicated in the conceptual confusion we proposed above. Though widely used in research on personality, the impulsivity construct is far from being well defined. Different ways of measuring and operationalizing this construct have led to contradictory empirical results. Nor is it clear where impulsivity belongs in multidimensional personality models. Originally included by Eysenck in extraversion but later moved to his psychoticism construct, it is a facet of neuroticism in the NEO five-factor model, and correlates to both extraversion and neuroticism (see Estrella Romero et al., 2006; also Carver et al., 2000). Additionally, in the five-factor model conscientiousness or constraint is supposed to reflect impulse control. Impulsivity is part of reactive behavior controlled by the ventral systems. Although ventral effortful control and reactive harm avoidance can produce constrained, feedback control of ongoing cognition and action (Tucker and Williamson, 1984), the direct control of behavior by emotional stimuli of the moment produces emotional reactive, impulsive behavior. This emotionally reactive kind of impulsivity is seen in internalizing and externalizing disorders (see also Reed and Derryberry, 1995). However, aspects of proactive behavior can also be described as impulsive. The lack of feedback control of ongoing cognition and action in the proactive mode, and also its opposition of reactive avoidance and inhibition, can be experienced and labeled as impulsive. In fact, Tucker and colleagues used to stress this kind of impulsivity related to proactive, feedforward function in previous writings (Tucker et al., 1995).

Figure 3 suggests that reactive approach or impulsive reward responding can be regulated in at least three different ways: (1) by inhibition or regulation from the proactive system (proactive inhibition); (2) by inhibition or opposition from the reactive avoidance system if punishments are involved (reactive inhibition); (3) by effortful control or constraint within the reactive approach system itself. Additionally, proactive approach and impulsivity can be inhibited by reactive inhibition and possibly be regulated by effortful control. Research until now did not simultaneously take those different forms of inhibition of approach into account, but usually focused on dichotomies.

Proactive and reactive goal-directed behavior

A final issue of conceptual confusion we would like to address within our framework is the relationship of behavioral programs with “goal-directedness.” In two-system models the cognitive control-type system is often described as supporting goal-directed behavior, while in the reactive-type system the reactivity is interpreted as being opposite to internal goal-directed. However, in our approach, drives and motivation are central to all four behavioral programs. The difference is how they are expressed, e.g., in proactive or reactive behavior. In a proactive mode, context models can assist in directing behavior toward goals. In a reactive mode, goals and motivational stimuli can be held active by the redundancy bias and actually lead to perseveration or obsessional behavior (Tucker and Williamson, 1984).

Summary

The present model suggests a reorganization of conceptual space in psychology. It suggests that certain behavioral characteristics that are usually contrasted with each other may actually belong to the same behavioral control system. At the same time, it suggests that some other characteristics belonging to different control systems are typically not discriminated. We think confusion in psychological conceptualizations arose because of two features of brain organization that cause (1) relationships between characteristics to not confirm to intuition and (2) difficulty in discriminating between characteristics that share psychologically salient features. Such problems can only be solved when conceptualization is constrained by a model that takes such features of brain organization into account. We discussed several aspects in which our model differs from, or is similar to other models. However, instead of attempting to define the definitive, ultimate model, our main point is that there are two aspects of brain organization that we think may help developing models of behavioral control that may overcome limitations of previous models.

The first feature of brain organization we think is important is the organization into behavioral/physiological programs. The behavioral/physiological programs approach may trigger new insights because it suggests that elements that are not intuitively similar, connected or at a similar level of complexity, or even seem opposites in some ways, may nevertheless be associated with the same system. They may be associated with the same system because they are elements of the same program that is adaptive in a particular environment. In other words, criteria for elements to be associated to the same psychologically relevant system are derived from their role in an adaptive program instead of from intuitions of what is similar or comparable. As an example, we discussed the tendency in psychology to group elements together that appear to reflect higher levels of complexity, sophistication and cognitive control, and contrast these with elements that appear to reflect lower levels of complexity, sophistication and cognitive control. However, the behavioral programs approach suggests that each program is likely to incorporate elements from the higher and lower levels, even when those elements seem opposites in some regards. For instance, reactive systems, that may at first thinking appear more primitive and not reflexive, may nevertheless be associated to high cognitive controls such as focused attention and a redundancy bias in working memory; at the same time, proactive systems as well may be associated with types of behavioral impulsivity and higher cognition.

The second feature of brain organization is the organization into dorsomedial systems that control behavior in a proactive way guided by context models, and ventrolateral systems that control behavior in a reactive way guided by environmental feedback. Without a model stressing the importance of this distinction, this distinction may be less salient and obvious in observed behavior and psychological experience, compared to other features that proactive and reactive programs may have in common. We discussed the example of proactive and reactive forms of positive affect that are typically not discriminated. Because positive and negative affectivity are core aspects of other constructs such as temperament, extraversion, approach-avoidance, and impulsivity, failure to discriminate between reactive and proactive affect affects those domains as well. In the discussion we did not elaborate on the distinction between proactive and reactive forms of avoidance, although we think this is important as well, and should be addressed in future work.

We do not advocate simply reorganizing psychological models around terms such as “proactive” and “reactive.” These terms are just as vulnerable to spurious associations and intuitions as other psychological terms. Rather, we advocate thinking in terms of brain systems controlling behavior guided by context models and other systems controlling behavior in an active, vigilant momentary attentional mode closely tied to motor readiness. Next, further elaboration on this basic distinction between underlying mechanisms is needed, in terms of their interactions, dynamics, and incorporation into more specialized behavioral programs and asymmetric hemispherical functions (Tucker et al., 1995; Tops and Boksem, 2010). For determining the characteristics of each behavioral program it will be important to delineate the specific environments and circumstances for which they evolved, and to go back-and-forth between psychological and brain models and evidence.

For the present model to be tested optimally, new measures should be developed that address the conceptual problems we discussed and specifically target the systems described. For this, careful thinking is necessary about how to catch often subtle differences at the behavioral level and how to catch the essence of systems. However, if the central points of this paper prove right, than this endeavor would likely improve progress and consistency of results in many important areas of psychology and psychopathology research.

Conflict of Interest Statement

The research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.