Prefrontal/accumbal catecholamine system processes high motivational salience.

Motivational salience regulates the strength of goal seeking, the amount of risk taken, and the energy invested from mild to extreme. Highly motivational experiences promote highly persistent memories. Although this phenomenon is adaptive in normal conditions, experiences with extremely high levels of motivational salience can promote development of memories that can be re-experienced intrusively for long time resulting in maladaptive outcomes. Neural mechanisms mediating motivational salience attribution are, therefore, very important for individual and species survival and for well-being. However, these neural mechanisms could be implicated in attribution of abnormal motivational salience to different stimuli leading to maladaptive compulsive seeking or avoidance. We have offered the first evidence that prefrontal cortical norepinephrine (NE) transmission is a necessary condition for motivational salience attribution to highly salient stimuli, through modulation of dopamine (DA) in the nucleus accumbens (NAc), a brain area involved in all motivated behaviors. Moreover, we have shown that prefrontal-accumbal catecholamine (CA) system determines approach or avoidance responses to both reward- and aversion-related stimuli only when the salience of the unconditioned stimulus (UCS) is high enough to induce sustained CA activation, thus affirming that this system processes motivational salience attribution selectively to highly salient events.

Incentive motivation and mesoaccumbens

In the last two decades motivation theory has reached developments of paramount importance for psychology and neuroscience. Incentive motivation theory was a crucial crossroads along the way the led to such important developments. Incentive motivation concepts rose in the 1960s when several new realizations about brain and motivation led many psychologists and behavioral neuroscientists to reject simple drive and drive-reduction theories. Specific alternative theories were developed in the form of incentive motivation theories (Bolles, 1972; Bindra, 1978; Toates, 1986, 1994; Panksepp, 1998; Berridge, 2001). Three biopsychologists made major incremental contributions to its development. Bolles (1972) proposed that individuals were motivated by incentive expectancies, not by drives or drive reduction. Incentive expectancies, that Bolles called S–S* associations, were essentially learned expectations of a hedonic reward, indistinguishable from cognitive predictions. Accordingly, a predictive neutral stimulus (S), such as a light or a sound, became associated by repeated pairing with a hedonic reward that followed (S*), such as a palatable food. The S caused an expectancy of the S*. The S was, in terms of Pavlovian learning processes, a conditioned stimulus (CS or CS+), and the S* an unconditioned stimulus (UCS).

Bindra (1974, 1978) acknowledged that expectations might well be important to cognitive strategies to obtain the reward, but suggested that a CS for a reward actually evokes the same incentive motivational state normally caused by the reward itself, as a consequence of classical conditioning. The learned association does not simply cause expectation of the reward. It also causes the individual to perceive the CS as a hedonic reward, and lets the CS elicit incentive motivation just as would the original hedonic reward. That means that the CS takes on specific motivational properties that normally belong to the S* itself, and these motivational properties are specifically incentive properties. Note that this was true not only for reward S*, but also for painful S* motivation, that would be based on fear or punishment properties. Toates (1986) modified the Bolles–Bindra views suggesting that physiological depletion states could enhance the incentive value of their goal stimuli. This leads to a multiplicative interaction between physiological deficit and external stimulus, which determined the stimulus' incentive value. However, physiological deficit signals do not drive motivated behavior directly, but they are able to magnify the hedonic impact and incentive value of the actual reward (S*), and also the hedonic/incentive value of predictive stimuli for the reward (CSs). Around 1990 the incentive salience model was proposed (Berridge et al., 1989; Berridge and Valenstein, 1991) that followed Bindra–Toates rules for incentive conditioning but identifies separable brain substrates for “liking” a reward versus “wanting” the same reward. “Liking” is essentially hedonic impact—the brain reaction underlying sensory pleasure-triggered by immediate receipt of reward, for instance, a sweet taste (unconditioned “liking”).

“Wanting,” or incentive salience, is the motivational incentive value of the same reward (Berridge and Robinson, 1998), the incentive motivational value of a stimulus, not its hedonic impact. The important point is that “liking” and “wanting” normally go together, but they can be split apart under certain circumstances, especially by certain brain manipulations. “Liking” without “wanting” can be produced, and so can “wanting” without “liking.”

Motivation can be conceptually described as a continuum along which stimuli can either reinforce or punish responses to other stimuli. Behaviorally, stimuli that reinforce are called rewarding and those that punish aversive (Skinner, 1953). Reward and aversion describe the impact a stimulus has on behavior, and provided of motivational properties, thus able to induce attribution of motivational salience.

The incentive salience model underscored the main role of dopamine (DA) function as brain mechanism of motivational processes. Indeed DA suppression leaves individuals nearly without motivation for any pleasant incentive at all: food, sex, drugs, etc., (Ikemoto and Panksepp, 1999; Naranjo et al., 2001; Berridge, 2004; Salamone et al., 2005). Thus, disruption of mesolimbic DA systems via neurochemical lesions of the DA pathway that projects to nucleus accumbens (NAc) or by receptor-blocking drugs, dramatically reduces incentive salience or “wanting” to eat a tasty reward, but does not reduce affective facial expressions of “liking” for the same reward (Pecina et al., 1997; Berridge and Robinson, 1998).

DA has a crucial role in motivational control. One type of DA neuron encodes motivational value, excited by rewarding events and inhibited by aversive or stressful events (Bromberg-Martin et al., 2010; Cabib and Puglisi-Allegra, 2012, for reviews). These neurons support brain systems for seeking goals, evaluating outcomes, and value learning. Indeed, most DA neurons are activated by reward-predicting stimuli and code bidirectional reward prediction errors (i.e., better than expected/worse than expected) in humans, monkeys, and rats (Ikemoto and Panksepp, 1999; Ikemoto, 2007; Schultz, 2007). Although discrete aversive stimuli such as air puffs, hypertonic saline, and electric shock induce activating responses in a small proportion of DA neurons in awake animals (Guarraci and Kapp, 1999; Joshua et al., 2008; Matsumoto and Hikosaka, 2009), most DA neurons are depressed by aversive stimuli (Ungless et al., 2004; Jhou et al., 2009). This response variability indicates that recorded cells are part of different, independent circuits (Margolis et al., 2006; Ikemoto, 2007; Bromberg-Martin et al., 2010). A second type of DA neuron encodes motivational salience, excited by both rewarding and aversive events (Bromberg-Martin et al., 2010).

Evidences suggest that different groups of DA neurons convey motivational signals in distinct manners (Matsumoto and Hikosaka, 2009) and the mesocorticolimbic DA system may be comprised of distinct circuits, each modified by distinct aspects of motivationally relevant stimuli, based on DA projections to NAc medial shell mediating positive stimuli, on DA projections to mpFC affected by aversive stimuli, and projections to NAc lateral shell affected by both rewarding and aversive stimuli, presumably reflecting saliency (Lammel et al., 2011). It has been shown how the VTA DA neurons may employ the convergent encoding strategy for processing both positive and negative experiences, intimately integrating with cues and environmental context (Wang and Tsien, 2011).

The mesolimbic dopaminergic system, that projects from the ventral tegmental area (VTA) neuronal cell bodies rostrally to the NAc, is a primary link in the reward pathway (Wise, 1996, 2004). However, DA release is not necessary for all forms of reward learning and may not always be “liked” in the sense of causing pleasure, but it is critical for causing goals to become “wanted” in the sense of motivating actions to achieve them (Robinson and Berridge, 1993, 2003; Berridge and Robinson, 1998; Palmiter, 2008).

A line of evidence supporting a role for DA in the motivational properties of stimuli comes from the place-conditioning paradigm (Mucha and Iversen, 1984; van der Kooy, 1987; Carr et al., 1989). This paradigm treats the increase in amount of time spent in an environment that has been paired with an UCS (either drugs or natural reinforces) as an index of the stimulus's reward properties. By contrast, if animals are repeatedly exposed to an environment paired with an aversive stimulus they will avoid the environment. In the first case we speak of conditioned place preference (CPP), in the second of conditioned place aversion (CPA). DA antagonists administered before each conditioning session with amphetamine block amphetamine-conditioned place preferences (Nader et al., 1997 for review). These results are not interpretable in terms of a general learning deficit because animals have been shown to be able to form normal CS-US associations in place conditioning with other US (Shippenberg and Herz, 1988). These findings suggest that normal DA transmission is necessary for the rewarding properties of stimuli to occur.

If the dopaminergic pathway from the VTA to the NAc is a primary link in the pathways mediating the motivational properties of stimuli (Tsai et al., 2009; Adamantidis et al., 2011), then examples of DA-independent reward should be nonexistent. There are a number of examples, however, of stimuli that possess reinforcing properties independent of DA. Thus, behavioral pharmacological experiments indicate that although increasing mesolimbic DA transmission plays an important role in the reinforcing effects of abused substances, there are also DA-independent processes that contribute significantly to the reinforcing effects of these compounds (Joseph et al., 2003; Pierce and Kumaresan, 2006 for review). For instance, DA antagonist pretreatment or 6-OHDA lesions of the NAc have been reported to have no effect on morphine or heroin self-administration (Ettenberg et al., 1982; Pettit et al., 1984; Dworkin et al., 1988), and on ethanol oral self-administration (Rassnick et al., 1993). A lack of dopaminergic involvement in cocaine place preferences (Spyraki et al., 1982; Mackey and van der Kooy, 1985) has been reported following systemic or intra-accumbens administration (Koob and Bloom, 1988; Hemby et al., 1992; Caine and Koob, 1993). Under some conditions, DA-independent opiate place preferences has been demonstrated (Mackey and van der Kooy, 1985; Bechara et al., 1992; Nader et al., 1994). Moreover, DA-deficient mice display a robust conditioned place preference for morphine in specific experimental conditions (Hnasko et al., 2005), and DA is not involved in opiate naive state (Laviolette et al., 2004; Vargas-Perez et al., 2009). A DA-independent reward mechanism for caffeine has been shown (Sturgess et al., 2010).

Dopamine D2 receptor knockout mutation in C57BL/6 mice failed to block ethanol-conditioned place preferences in ethanol-dependent and withdrawn mice (Ting-A-Kee et al., 2009). In a more “naturalistic” conditions operant place conditioning by male chemo-signal in female mice was not affected by D1 or D2 receptor antagonists (Agustin-Pavon et al., 2007). It is worth noting that VTA-mediated but DA-independent positive reinforcement has been demonstrated (Fields et al., 2007).

These examples of DA-independent motivated behavior seriously question the original DA hypothesis that suggested DA is a final common pathway in the processes mediating reinforcement.

Prefrontal-accumbal catecholamine system

About a couple of decade ago, research has pointed to prefrontal catecholamine (CA) regulation of mesoaccumbens DA transmission in response to pleasant or aversive stimuli (Le Moal and Simon, 1991). In particular, DA transmission in subcortical structures, such as the NAc, seems to be modulated by the DA mesocortical system in an inhibitory way (Ventura et al., 2004, for review), thus strongly suggesting that the mesoaccumbens DA response is inversely related to the mesocortical DA response.

Mesoaccumbens DA transmission has been suggested to be regulated by prefrontal transmission via glutamatergic projections (Carr and Sesack, 2000, for review), through activation of excitatory prefrontal-cortical projection to the VTA (Sesack and Pickel, 1990), and/or through activation of a corticoaccumbens glutamatergic projection (Taber and Fibiger, 1995). Thus, besides possible direct cortico-accumbal circuit, a cortico-(VTA)- accumbal DA network involving different brain areas, such as amygdala (Jackson and Moghaddam, 2001; Mahler and Berridge, 2011), has been proposed to have an important role on accumbal DA modulation.

In late nineties, a French study (Darracq et al., 1998) showed that prefrontal cortical norepinephrine (NE) had a pivotal role in the increased accumbal DA release induced by systemic amphetamine administration. Till that moment, brain noradrenergic system involvement in behavior control was mostly focused on Locus Coeruleus (LC) functions (Aston-Jones et al., 1999) or on emotional memory regulation by the amygdala (McGaugh, 2006). The pioneer work of Darracq and coworkers, suggested implicitly that DA transmission in the NAc could be controlled by and directly related to NE in the medial prefrontal cortex (mpFC). This view, together with the established inhibitory role of prefrontal DA on dopaminergic activity in the accumbens, suggested a possible opposite action of the two amines in the prefrontal cortex on subcortical DA transmission.

Experimental evidence from our lab on mice of C57BL/6 (C57) and DBA/2 (DBA) inbred strains supported this hypothesis. Comparative studies of neurotransmitter activity and behavior in different genetic backgrounds make available a major strategy for investigating the neural basis of drug effects related to individual differences. Mice of the DBA background have been shown to be poorly responsive to the enhancing extracellular DA induced by the psychostimulant in the NAc (shell) as well as to the stimulating/reinforcing effects of amphetamine, that are dependent on increased accumbal DA release. The opposite occurs in mice of the C57 background, which have been shown to be highly responsive to stimulating/reinforcing effects of amphetamine, as shown by increased locomotor activity or to amphetamine-induced CPP (Zocchi et al., 1998; Cabib et al., 2000). In C57, Amphetanine produces low mpFC DA and high DA in the NAc, the opposite occurs in DBA mice that show lower locomotor activity than C57and no CPP or even CPA. Moreover, selective DA depletion in the mpFC of DBA mice makes this strain similar to highly responsive C57 mice leading to high DA outflow in the NAc and hyper locomotion. Nevertheless, no differences in structure or expression of DA transporter in the NAc between C57 and DBA strains have been reported (Womer et al., 1994). These results showed that the different effects of amphetamine on accumbal DA outflow in the two backgrounds do not depend on differences in DAT related mechanisms. However, microdialysis experiments showed that amphetamine increased NE and DA outflow in the mpFC of C57 and DBA mice in a different way. While C57 showed higher NE increase than DA, DBA mice present an opposite pattern, thus indicating that NE/DA ratio induced by amphetamine is higher in C57 versus DBA. Since DA is inhibitory on DA NAc, while NE was suggested to be enabling (Darracq et al., 1998), we hypothesized that imbalanced NE/DA in the mpFC controlled DA in the NAc and related behavioral outcomes, making the C57 strain more responsive than DBA. Such a hypothesis was confirmed by subsequent experiments showing that selective prefrontal cortical NE depletion abolished the effects of amphetamine on DA in accumbens and CPP in C57 mice (Ventura et al., 2003), while selective prefrontal DA depletion (sparing NE) led to DA outflow in the NAc and behavioral outcomes in DBA mice entirely similar to those of C57 (Ventura et al., 2004, 2005).

These data suggested strongly that DA in the NAc is controlled by prefrontal cortical NE that enables it, and by DA that inhibits it. Moreover, our data indicated that prefrontal NE transmission is critical for attribution of motivational salience, as demonstrated by the impairment of amphetamine-induced CPP in mpFC NE depleted C57 mice (Ventura et al., 2003).

However, evidence in the literature (Ventura et al., 2002 for review) and results on stress obtained in our lab on C57 and DBA mice had shown that this was true also for aversive experiences (restraint, Forced Swimming), at least as far prefrontal DA control over DA in the NAc was concerned. Indeed, we found that restraint stress produced inhibition of mesoaccumbens DA release accompanied by a very fast and strong activation of mesocortical DA metabolism in C57 mice, and the opposite in mice of the DBA strain, showing a genetic control over the balance between mesocortical and mesoaccumbens DA responses to stress (Ventura et al., 2001). Moreover, C57 mice but not mice of the DBA strain showed an extremely high level of immobility on their first experience with the forced swimming test (FST) as well as immediate and strong activation of mesocortical DA metabolism and inhibition of mesoaccumbens DA metabolism and release. In addition, the behavioral and the mesoaccumbens DA responses to FST in C57 mice were reduced and reversed, respectively, by selective dopamine DA depletion in the mpFC (Ventura et al., 2002).

Prefrontal NE transmission was known to play a critical role in regulating many cortical functions, including arousal, attention, motivation, learning, memory, and behavioral flexibility (Sara and Segal, 1991; Tassin, 1998; Feenstra et al., 1999; Arnsten, 2000; Robbins, 2000; Bouret and Sara, 2004; Dalley et al., 2004; Mingote et al., 2004; Tronel et al., 2004; Aston-Jones and Cohen, 2005; Rossetti and Carboni, 2005; Lapiz and Morilak, 2006; van der Meulen et al., 2007; Robbins and Arnsten, 2009). Moreover, both rewarding/reinforcing and aversive stimuli have been shown to increase NE release in pFC (Finlay et al., 1995; Dalley et al., 1996; Goldstein et al., 1996; Jedema et al., 1999; Kawahara et al., 1999; McQuade et al., 1999; Feenstra et al., 2000; Page and Lucki, 2002; Morilak et al., 2005; Feenstra, 2007). These evidences suggested that CA prefrontal transmission could control DA in the accumbens also in stressful conditions, a hypothesis that deserved to be assessed. This was done by two independent laboratories and published in 2007. These studies showed that novel stressful experiences enhance DA release in the NAc through activation of prefrontal cortical alpha-1 adrenergic receptors (ARs) by high levels of released NE (Nicniocaill and Gratton, 2007; Pascucci et al., 2007). Indeed, the experience with a novel stressor promotes a rapid, massive, and transient increase in NE release within the mpFC which parallels the enhancement of mesoaccumbens DA release (Pascucci et al., 2007). A selective depletion of prefrontal cortical NE prevents both the cortical NE response and the increase in accumbal DA, leaving stress-induced enhancement of prefrontal cortical DA release as well as basal CAs levels unaffected (Pascucci et al., 2007). Moreover, applications of the alpha-1 AR selective antagonist benoxathian into the mpFC inhibits stress-induced DA release in the NAc dose-dependently (Nicniocaill and Gratton, 2007). Pascucci et al. (2007) also confirmed that stress-induced enhanced NAc DA release is constrained by activation of mpFC DA. Indeed, either DA depletion (Deutch et al., 1990; Doherty and Gratton, 1996; King et al., 1997; Pascucci et al., 2007) or blockade of D1 receptors by infusion of a selective antagonist in the mpFC (Doherty and Gratton, 1996) enhances stress-induced DA release in the NAc. It is known that DA in the mpFC exerts an inhibitory influence on DA release in the NAc and depletion of mesocortical DA facilitates stress-induced activation of mesoaccumbens DA release (Deutch et al., 1990; Doherty and Gratton, 1996; King et al., 1997). However, our results demonstrated that during novel stressful experiences the mpFC determines mesoaccumbens DA response through the opposing influences of NE and DA. Our data could explain why stress may be involved in different pathological conditions. Indeed, the balanced action of the two CAs in the mpFC may be required for healthy coping, whereas unbalanced action may promote hyper- or hypo-responding by mesoaccumbens DA, leading to different and even opposite behavioral disturbances.

The opposite influence exerted by mpFC NE and DA on DA transmission in the NAc during stressful experiences points to possible opposite modulation of frontal cortical glutamate (GLU) by the two CAs. Since blockade of mpFC alpha-1 ARs or D1 receptors has opposite effects on stress-induced GLU increase (Lupinsky et al., 2010), it is likely that frontal cortical NE and DA exert opposite effects on mpFC output, possibly through glutamatergic stimulation of GABA interneurons in the mpFC (Del Arco and Mora, 1999; Homayoun and Moghaddam, 2007).

The involvement of alpha1-ARs in the prefrontal NE control of DA release in the NAc during stress is consistent with evidence that a sustained increase of prefrontal cortical NE (as the one induced by stress) is capable to activate these low-affinity receptors subtypes, while mild increase is able to activate high affinity alpha2- or beta1- ARs (Ramos and Arnsten, 2007). However, the main role of alpha1-ARs in the mesoaccumbens DA activation by stress or by amphetamine (Darracq et al., 1998; Ventura et al., 2003; Nicniocaill and Gratton, 2007), and the crucial role of prefrontal NE in attribution of motivational salience to stimuli related to amphetamine, as shown by CPP study in the mouse (Ventura et al., 2003), point to a main role of these receptors in motivated behavior and coping. mpFC and NAc receive DA afferents from different populations of VTA DA cells and these are controlled by different circuits (Joel and Weiner, 1997; Carr and Sesack, 2000; Lewis and O'Donnell, 2000; Margolis et al., 2006; Lammel et al., 2008; Tierney et al., 2008). VTA also receives afferents from the central nucleus of amygdala (CeA); the inhibition of CeA, and hence of its inhibitory input to VTA, leads to an increase of NAc DA (Ahn and Phillips, 2003; Phillips et al., 2003a), suggesting that this input is part of a double inhibition mechanism (Fudge and Haber, 2000; Ahn and Phillips, 2002; Floresco et al., 2003; Fudge and Emiliano, 2003). NE afferents in the mpFC originate from the relatively small group of cells of LC (Aston-Jones et al., 1999; Valentino and van Bockstaele, 2001; Berridge and Waterhouse, 2003). LC receives strong convergent projections from the orbito-frontal and cingulate cortex, which have been suggested to drive transitions between phasic and tonic modes in NE neurons to fit the behavioral/cognitive states with environmental conditions (Aston-Jones and Cohen, 2005). LC activity is also modulated by CeA (Curtis et al., 2002) through innervation of the pericoerulear region (Berridge and Waterhouse, 2003) and through the excitatory corticotropin-releasing hormone (Van Bockstaele et al., 2001; Bouret et al., 2003; Jedema and Grace, 2004). NE has different effects on target cortical areas depending on its concentration and on the distribution of alpha1 and alpha2 receptors (Briand et al., 2007; Arnsten, 2009). Indeed, different levels of tonic neuromodulator release affects receptors that are differentially located among cortical layers, so that a neuromodulator may differently affect its target subregions depending on the receptors it activates.

The evidences considered till now indicate that a prefrontal CA system controls DA release in the NAc, a sub-cortical area known to be involved in all motivated behavior, independently of the valence of the stimuli or experiences. Thus, a similar prefrontal-accumbal regulation has been shown for rewarding (amphetamine) or aversive (stress) stimuli. Further studies provided substantial support to this view, through experimental evidences that prefrontal cortical NE is crucial in the effects of other addictive drugs, of palatable food, and of aversive pharmacological or physical stimuli. Moreover, they demonstrated that prefrontal NE through its action on NAc DA is essential in attribution of motivational salience in specific conditions, as it will be shown in the next paragraph.

Prefrontal NE- accumbal DA in motivational salience attribution to both appetitive- and aversion-related stimuli

Other addictive drugs, in addition to amphetamine, increase DA release in the NAc through prefrontal NE, as shown by experiments based on intracerebral microdialysis in the mouse and on selective NE depletion in the mpFC. Selective NE depletion was carried-out by the neurotoxin 6-hydroxydopamine and pre-treatment with the selective DA transporter blocker GBR-12909 that produced about 90% NE afferents destruction, with no significant effects on DA. To avoid substantial changes in receptor regulation, neurochemical and behavioral testing were carried out within one week from surgery. Morphine (Ventura et al., 2005), Cocaine (Ventura et al., 2007), ethanol (Ventura et al., 2006, in preparation) have been shown to induce dose-dependent increase of NE in the mpFC and a parallel increase of DA in the NAc. Selective prefrontal NE depletion abolished the outflow increase of both prefrontal NE and DA in the NAc, thus confirming the crucial role of NE in mpFC in accumbal DA activation induced by different classes of drugs of abuse. It is worth noting that all the assessed drugs increased DA outflow in the mpFC, that was not affected by NE depletion. However, one may hypothesize that, based on the known inhibitory role of prefrontal DA on DA release in the NAc observed in animals receiving drugs (e.g., amphetamine) or stress, the failure of DA increase in the NAc of NE mpFC depleted subjects receiving drugs was due to the prevalent inhibitory action of prefrontal DA in the absence of NE. Such a view, would affirm the crucial “promoting” role of prefrontal NE on accumbal DA, pointing, however, to a complementary role of DA in mpFC that would exert an inhibitory role leading to “flatting” accumbal DA when cortical NE is depleted. This possibility was ruled-out by complementary experiments showing that concomitant depletion of NE and DA in mpFC does not change the impaired accumbal DA release in mice receiving AMPH in comparison with animals subjected to selective NE depletion. A body of evidence suggests that DA in the prefrontal cortex is co-released with NE from noradrenergic terminals (Devoto et al., 2001, 2002). Moreover, it has been reported that DA in this brain area is normally cleared by NE transporter (Tanda et al., 1997; Moron et al., 2002). Different set of data obtained in both mice and rats, showed a lack of effects of the NE depletion on basal extracellular DA, suggesting that the likely reduction of DA released from destructed noradrenergic terminals is compensated by augmented availability of DA due to its reduced uptake from these terminals (Ventura et al., 2005; Pascucci et al., 2007). However, NE-depleted mice showed an increase of morphine-induced DA release similar to that exhibited by Sham animals, thus suggesting that prefrontal noradrenergic and dopaminergic projections are functionally uncoupled. In agreement with this observation, selective prefrontal NE depletion in rats did not affect stress-induced DA release and selective DA depletion did not affect stress-induced NE release. Taken together, these data indicate that, in both reinforcing (morphine injection) and aversive (stressful situation) conditions, NE and DA release in the mpFC are independent.

This evidence suggests that NE is a common regulating element responding to different class of stimuli to induce DA activation in the NAc, irrespective of the specific pharmacological or physiological properties of stimuli. Possible network elements have been mentioned before and will be further considered. Here, it is worth to point out that different class of pleasant stimuli as well as aversive experiences such stress are likely to activate a common prefrontal cortical-subcortical network.

The role of mesoccumbens DA system in motivation is well-established. However, whether also a system, involving prefrontal NE and accumbal DA, has a role, needs experimental support. To study incentive learning and incentive motivation, place conditioning is commonly exploited in rats and mice, but in the last species is prevalent since operant procedures that are mostly used for studying drug self-administration in rats, present a number of difficulties in mice. Anyway, this method allows attribution of motivational salience to stimuli related to either pleasant (appetitive) or aversive stimuli (US). In the first case pairings between stimuli and environment (CS) lead to place preference (CPP), while in the second produce place aversion (CPA). The process of attribution of motivational salience is measured by the preference (or the aversion) shown when a subject has to choose between the environment previously paired with the US and a neutral environment (Tzschentke, 1998; Mueller and Stewart, 2000). This method is also useful to assess relapse to previously preference (or aversion) after extinction, and is a choice method in modeling addiction (Lu et al., 2003; Shaham et al., 2003). Indeed, a before mentioned study had shown that selective prefrontal cortical NE depletion besides impairing amphetamine-induced DA outflow increase in the NAc, impaired CPP induced by the stimulant. These effects were not due to motor deficits, or learning impairments, since depleted animals were not different from sham controls in motor behavior, and, most importantly, were able of associative learning as shown by avoidance test (Ventura et al., 2003).

Moreover, these results indicate that intact prefrontal cortical NE is necessary for CPP induced by morphine, cocaine, or ethanol as well as for reinstatement (relapse) of extinguished morphine-induced CPP, and for ethanol intake in a choice test. Thus, they demonstrate that prefrontal NE is crucial for DA release in the NAc induced by addictive drugs and for attribution of motivation salience to drug-related stimuli.

However, the results concerning aversive experiences demonstrate that the noradrenergic control of accumbal DA activation is evident also for stress, suggesting a common network involved in processing both pleasant (rewarding) and aversive stimuli. To assess this hypothesis we planned two experiments. In the first one we observed that a pharmacological aversive stimulus such lithium chloride administered systemically in mice induced a clear-cut increase of NE in the mpFC and DA in the accumbens that was abolished by selective prefrontal NE depletion. Moreover, lithium induced a CPA that was abolished by prefrontal NE depletion, thus confirming that prefrontal NE is crucial for attribution of motivational salience to stimuli related to aversive experience (Ventura et al., 2007).

Next step was suggested by preliminary results obtained when we decided to assess the role of prefrontal-accumbal CA system in attribution of motivational salience to natural non-pharmacological stimuli. Previous data in the literature allowed to hypothesize that appetitive or aversive stimuli produce a graded activation of prefrontal noradrenergic transmission, thus more salient is a stimulus stronger the prefrontal NE release will be (Feenstra et al., 2000; Ventura et al., 2008 for review). If this were the case, then prefrontal NE release could be considered an index of stimuli salience. To further support that prefrontal NE-accumbal DA system is crucial for attribution of motivational salience also for aversive stimuli we used as aversive non-pharmacological experience a stressor (intermittent lights) that could be graded in order to provide parallel effects to those of pleasant (rewarding) stimuli like palatable food before described. In place conditioning preliminary tests, in which the two stressors were compared we observed that they differ in the conditioned aversive effects, the pulsating intermittent lights being more aversive than intermittent non-pulsating lights. This result paralleled the effects of the two aversive conditions on prefrontal cortical NE release. Both lighting conditions increased prefrontal NE release, but pulsating lighting produced more pronounced increase than non-pulsating lighting. Moreover, noradrenergic response in mpFC was paralleled by graded increase of DA in the NAc (Ventura et al., in preparation).

Then, we assessed if appetitive non-pharmacological stimuli, used as US in place conditioning, required intact prefrontal NE-accumbal DA functioning for attribution of motivational salience. We have observed that mice preferred white chocolate (WCh) to Milk (MCh)-chocolate in a free choice test, a preference that was confirmed in a CPP paradigm where mice chose the environment paired with WCh in comparison with that paired to MCh-chocolate. Consistently, intracerebral microdialysis showed that exposure to WCh intake produces a higher NE release in mpFC than MCh (Ventura et al., 2008, in preparation) accompanied by a more sustained DA outflow in the NAc. These results demonstrate that prefrontal NE and accumbal DA respond to different salient stimuli, either pleasant or aversive, in a graded manner.

Incentive motivation theory has pointed to the major role of the motivational status of the organism (hungry, thirsty, tired, alert etc.) when it is confronted with a stimulus or experience. Stress has received much attention in motivation related studies, particularly those concerning addiction models, for neuro-adaptation it can produce in the brain systems involved in the response to drug priming, incentive learning processes, and relapse. We wondered if pre-exposure to stressful experience could affect the “perceived” salience of the stimulus and the response of prefrontal-accumbal CA system, and if such changes could affect attribution of motivational salience in our experimental conditions. We used a food-restriction regimen as chronic stress that was also shown to change the behavioral response to amphetamine and, affect attribution of motivational salience in mice (Cabib et al., 2000; Guarnieri et al., 2011). Food-restriction (FR) led to higher NE release in mpFC and higher DA release in the NAc in comparison with control mice. This increase was similar to that shown by free-fed (Non-FR) mice exposed to WCh, thus showing that the state of the organism, as expected, influenced the response to appetitive stimuli. This effect may be ascribed obviously to the food deprivation that would make it more palatable. However, our data indicate that FR regimen is an environmental condition that affects the perceived salience, independently of food-related mechanism. Indeed, we observed that FR made the effects induced by the less salient stressor (intermittent light) similar to the effects produced in non-FR mice by the more salient stressor (pulsating intermittent light). This means that FR is capable to increase the salience of both pleasant (rewarding; food) and aversive (stressful lighting) stimuli, regardless of hunger-related mechanisms. Note that in additional experiments Sham and NE depleted mice subjected to a different no food-related chronic stressful experience (social isolation) showed similar effects to those of FR animals, thus indicating that the effect of prefrontal NE depletion on MCh-induced CPP cannot be ascribed to the homeostatic response to dietary restriction (Ventura et al., 2008). Food-restriction can also be considered as leading to a generalized drive effect (Niv et al., 2006; Phillips et al., 2007) that would “energize” motivation. This mechanism seems to depend on deprivation states. Our results, however, indicate that a generalized drive effect produced by food-restriction regimen before exposure to specific stimuli affects not only appetitive alimentary stimuli but also aversive stimuli. In fact, aversive effects of intermittent light are stronger in food-restricted than in free-fed mice. Thus, a generalized drive effect should involve common neural mechanisms regulating both appetitive and aversive experiences.

Taken together, these results show that the prefrontal-accumbal CA response is an index of the emotional/motivational impact of differently salient stimuli depending on the stimuli characteristics or on the state of the organism. The graded response of prefrontal NE was in agreement with previous results and suggested to us to ascertain the role of prefrontal-accumbal CA system in attribution of motivational salience related to differently salient stimuli. Using the experimental paradigms of other studies on the same topics, we assessed the effects of selective prefrontal NE depletion on the CA response and on attribution of motivational salience measured by place conditioning. Surprisingly, we observed that NE depletion abolished the increase of prefrontal cortical NE release and of accumbal DA, consistently with previous experiments. However, it prevented place preference (CPP) in animals exposed to WCh and in food-restricted (FR) animals exposed to milk chocolate (MCh; both conditions of high salience) but not in non-FR (Free-fed) animals exposed to MCh (low salience). Moreover, it prevented place aversion (CPA) in animals exposed to intermittent pulsating light (IPL) and in the FR animals exposed to intermittent light (IL; high salience) but not in non FR animals exposed to IL (low salience; Figure 1).

Figure 1

Effects of prefrontal cortical norepinephrine depletion on conditioned place preference (CPP) induced by chocolate (milk chocolate in control, MCh; milk chocolate in food restricted MCh+FR; white chocolate in control, WCh) and conditioned place aversion (CPA) induced by light (intermittent light in control, IL; intermittent light in food restricted, IL+FR; intermittent-pulsating light in control; IPL) [Sham, norepinephrine-depleted in medial prefrontal cortex (mpFC NE Depleted)]. All data are expressed as mean (sec ± SE) time spent in Paired, and Unpaired chambers. *P < 0.05 in comparison with the Unpaired chamber; #P < 0.005 in comparison with the Unpaired chamber (from Ventura et al., 2008, modified).

These results show that that pFC NE depletion affects attribution of motivational salience only when the salience of the UCS is high enough to induce sustained CA activation, thus indicating that prefrontal-accumbal CA system is involved in processing motivational salience attribution selectively when intense motivational salience is processed. Salience refers to the capability of stimuli to be arousing (Horvitz, 2000). Salient stimuli cause the reallocation of available cognitive resources in order to produce an attentional or a behavioral switch (Zink et al., 2006). The more salient the stimulus, the more likely it will lead to an attentional or a behavioral switch. Recent reports in humans have shown that striatum has a major role in prompting the reallocation of resources to salient stimuli (Zink et al., 2003, 2006). However prefrontal cortex, due to its “supervisory” functions, has an unquestionable central role in attentional and motivational processing of salient stimuli.

Moreover, data indicate that ventral striatum (or NAc) and prefrontal cortex constitute a common substrate for processing both rewarding and aversive stimuli (Berridge and Robinson, 1998; Darracq et al., 1998; Becerra et al., 2001; Jensen et al., 2003; Kensinger and Schacter, 2006; Borsook et al., 2007), and neuroimaging studies in humans suggest that different areas of prefrontal cortex (O'Doherty et al., 2001; Small et al., 2001; Killgore et al., 2003; Wang et al., 2004) and of striatum (Jensen et al., 2003; Zink et al., 2006; Borsook et al., 2007) are activated by natural positive or negative salient stimuli. Again, we have previously demonstrated that intact NE prefrontal transmission is necessary for motivational salience attribution to both natural (in food restricted animals) and pharmacological reward related stimuli as well as to pharmacological aversion related stimuli through modulation of DA in NAc (Ventura et al., 2007). Therefore, it is likely that the effects of prefrontal NE depletion on CPP and CPA in animals exposed to highly salient stimuli depend on the impaired response of prefrontal-accumbal CA system, whose activation by unconditioned rewarding and aversive highly salient stimuli is a substrate for motivational salience. However, other brain areas and neurotransmitters are likely to be engaged. Thus, since amygdala is involved in pavlovian conditioning of emotional responses and plays a specific role in modulating memory for arousing experiences (Balleine, 2005; Balleine and Killcross, 2006; McGaugh, 2006), and given the complex anatomical and functional connections between this brain area and prefrontal cortex (Cardinal et al., 2002; Holland and Gallagher, 2004; Roozendaal et al., 2004) a role of a prefrontal cortex-amygdala system in the effects of highly salient stimuli reported here must be considered (Belova et al., 2007).

Conclusions

Attribution of motivational salience is related to the salience of an UCS (Dallman et al., 2003; Pecina et al., 2006). Thus, the more salient an UCS the more likely a neutral (to-be-conditioned) stimulus will be associated with it through motivational salience attribution. Prior experience is a major determinant of the motivational impact of any given stimulus (Borsook et al., 2007) and emotional arousal induced by motivational stimuli increases the attention given to stimuli influencing both the initial perceptual encoding and the consolidation process (Anderson et al., 2006; McGaugh, 2006). We provided evidence that prefrontal-accumbal CA transmission is necessary for motivational salience attribution to both reward- and aversion-related stimuli only under those conditions able to induce stronger increases of CA outflow in response to highly salient unconditioned natural stimuli, independently of valence.

Thus, selective prefrontal NE depletion abolished the place conditioning induced by highly salient stimuli (i.e., WCh and IPL) in control animals and by mildly salient stimuli (i.e., MCh and IL) in stressed groups, but had no significant effects in control animals exposed to mild salient stimuli. These results demonstrate that prefrontal-accumbal CA transmission is necessary for the acquisition of conditioned properties to stimuli paired with highly salient natural rewarding or aversive events in a place-conditioning procedure. Many different factors have major regulatory role in motivated behaviors, including the internal variables of the organism (i.e., motivational state, stress response) and stimulus properties (i.e., salience or intensity), both of which affect motivational salience attribution processes (Berridge and Robinson, 1998; Richard and Berridge, 2011). It has been recently proposed that appetitive and aversive brain systems act in a “congruent manner for processes sensitive to affective intensity (salience) but not valence” (Belova et al., 2007), thus suggesting that a common neural system might be involved in processing stimuli salience, irrespectively of valence. Moreover, arousing pleasant or aversive stimuli that elicit valence-specific responses have been suggested to enhance attention and memory formation through a common, valence-insensitive pathway (Belova et al., 2007) and prefrontal cortex has been involved in processing both rewarding and aversive stimuli (Rolls, 2000; O'Doherty et al., 2001; Killgore et al., 2003; Ventura et al., 2007).

Dopaminergic transmission within NAc is considered to mediate the hedonic impact of reward or some aspects of reward learning (Everitt and Robbins, 2005 for review). Our results, in agreement with a different view (Berridge and Robinson, 1998), show that DA transmission in NAc plays a role in both positively and aversively motivated behavior; most importantly, however, they demonstrate that this motivational process is governed by prefrontal cortical NE.

Norepinephrine in mpFC might activate mesoaccumbens DA release through excitatory prefrontal cortical projection to VTA DA cells (Sesack and Pickel, 1992; Shi et al., 2000) and/or through corticoaccumbal glutamatergic projections (Darracq et al., 2001). Moreover, a role for mpFC projections to the LC in exerting an excitatory influence can be envisaged because this nucleus has been shown to activate VTA DA neurons (Grenhoff et al., 1993; Jodo et al., 1998; Liprando et al., 2004), which could lead to increased DA release in NAc. However, since amygdala is involved in Pavlovian conditioning of emotional responses and plays a specific role in modulating memory for arousing experiences (Balleine and Killcross, 2006; McGaugh, 2006), and given the complex anatomical and functional connections between this brain area and prefrontal cortex (Cardinal et al., 2002; Roozendaal et al., 2004), a role of the prefrontal cortex-amygdala system in the effects of the highly salient stimuli reported here must be considered (Belova et al., 2007; Mahler and Berridge, 2011).

Note that NAc and dopaminergic transmission are considered to play major role in motivation processes besides the role that DA plays in other aspects of incentive motivation and instrumental learning (Salamone et al., 2005). Indeed, based on the view that is doubtful that accumbens DA performs only one function, substantial evidences support the hypothesis that DA is involved in the exertion of effort or effort-related decision-making (Salamone et al., 2007; Bardgett et al., 2009), that is not incompatible with the involvement of this system in instrumental learning, incentive motivation, or pavlovian-instrumental transfer. Animal and human studies seem to converge in that, together with animal studies focused on effort related functions of accumbal DA, clinical findings are consistent with the hypothesis that DA systems are involved in behavioral activation, pointing to a striking similarity between the brain systems implicated in effort-related processes in animals and those involved in energy dysfunctions in humans (Salamone et al., 2007). According to this view NAc functioning is to be considered, along with prefrontal cortex and the amygdala, as a component of the brain circuitry regulating effort-related functions. In this framework, the prefrontal/accumbal CA system we have envisaged may conceivably be a part of a complex network involving cortical and subcortical brain areas involved in regulation of effort-related functions controlling motivation outcomes, and possibly linking salience intensity to effort intensity. In our view the impact of salient stimuli is crucial in the processes that lead to attribution of motivational salience, due to the salience perceived. That means that the impact of stimuli produce an emotional response that tunes the association processes leading to motivation outcome, thus pointing to the basic role of emotional salience as the individual is exposed to the UCS. The prefrontal/accumbal system we propose to control motivational salience processes depending on salience intensity should be considered included in complex networks regulating perceived emotion (Phillips et al., 2003b). Emotion perception, according to the appraisal theories (Arnold, 1960; Lazarus, 1991) has been suggested to stems from three processes: the identification of the emotional significance of a stimulus, the production of an affective state in response to the stimulus, and the regulation of the affective state. As shown by human and animal literature (Phillips et al., 2003b for review), these processes depend on different brain emotional systems involving brain areas including mesencephalic, cortical, and subcortical such as the amygdala, insula, ventral striatum, ventral, and dorsal anterior cingulate gyrus, the septo-hippocampus system, the prefrontal cortex, all characterized by reciprocal functional relationships (Salzman and Fusi, 2010). The septo-hippocampus system has been considered as a general-purpose comparator, with a central role in determining the extent of conflict between different goal-directed behaviors (Gray and McNaughton, 2000). Amygdala has a well-known role in emotion and in memory consolidation processes depending on emotional arousal. Recently a role of this area in decision-making has been envisaged. Indeed, the amygdala can evoke conditioned responses capable of exerting a dominant effect on choice, and perceived emotional values in Pavlovian conditioning are exploited by instrumental (habit-based and goal-directed) learning mechanisms through connectivity with other brain regions such as the striatum and prefrontal cortex (Seymour and Dolan, 2008).

It is worth noting that “values” are affected by the action of stress hormones, such as glucocorticoids, on amygdala, and these effects control memory consolidation pointing to a link between emotional salience and the strength of memories (Roozendaal, 2000; Setlow et al., 2000; McGaugh, 2005). Moreover glucocorticoids have been shown to be biological substrate of reward (Piazza and Le Moal, 1997) and substantial evidences show that they play a role in the modulation of both appetitive and aversive emotional memories indicating that modulation of appetitive and aversive discrete-cue learning may be subserved by a common mechanism (Zorawski and Killcross, 2002).

We have provided evidence that prefrontal-accumbal CA system is involved in processing motivational salience attribution selectively when intense motivational salience is processed, thus pointing to an allegedly different neural system involved in the attribution of motivational salience related to mildly salient stimuli. Our results are consistent with those that had shown DA transmission is not always involved in motivation (Nader et al., 1997, for review). In contrast with the DA hypothesis that is based on a single-system model of reward, a non-deprived/deprived model has been proposed in the late nineties that claims that two separate neurobiological reward systems can be double dissociated, each of which makes a significant contribution to motivated behavior depending on deprivation state. Note that the model is supported by experiments where drug naive animals are considered similar to food-sated (i.e., non-deprived), differently from drug-dependent animals in withdrawal or food-restricted animals that are considered deprived (Nader et al., 1997; Laviolette et al., 2004). The model has two important implications. First, the relationship between the two systems appears to be mutually exclusive. A state of deprivation inhibits the non-deprived system [involving the peduncolo-pontine nucleus (TPP)]. Thus, the differential activation of the two systems is predicated specifically on whether animals are, for instance, in a state of withdrawal or not (Nader et al., 1997). The second implication is that a state of deprivation engages a second neurobiologically distinct motivational system, a component of which is DA.

The obvious question that arises from this model is whether all motivated behaviors can be considered to have a non-deprived and a deprived component. As it was questioned by the proponents (Nader et al., 1997, for review): “Do some stimuli only work through one of the two systems?” Although this discussion is out of the aim of our present work, we can't help noting the parallelism between our findings on prefrontal-accumbal CA system and the non-deprived/deprived system, in that our system is crucial in attribution of motivational salience when the stimuli salience is high and characterized by high emotional impact (either positive or negative). In this case, another system involved in low salience processing is inhibited or “off-line,” and this system that is on-line when low salience is processed (and that we did not envisaged yet), parallels the non-deprived system, characterized by low emotional impact. Our results also suggest strongly that, like suggested for the non-deprived/deprived model, the system processing high salience (prefrontal-accumbal CA system) and the supposed one involved in low salience are mutually exclusive. In terms of the neural dynamics involved in the selective and exclusive engagement of these systems, we can tentatively pose that the graded increase of NE outflow in mpFC depending on low or high salience of stimuli, may involve different AR-subtypes, that, in turn, depending on a given threshold level of released NE, will engage different circuits, and, in the case of high salience, including DA in the NAc. This is the aim of ongoing experiments that will possibly elucidate this critical question.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.