Endocannabinoids and the processing of value-related signals.

Endocannabinoids serve as retrograde signaling molecules at many synapses within the CNS, particularly GABAergic and glutamatergic synapses. Synapses onto midbrain dopamine (DA) neurons in the ventral tegmental area (VTA) make no exception to this rule. In fact, the effects of cannabinoids on dopamine transmission as well as DA-related behaviors are generally exerted through the modulation of inhibitory and excitatory afferents impinging onto DA neurons. Endocannabinoids, by regulating different forms of synaptic plasticity in the VTA, provide a critical modulation of the DA neuron output and, ultimately, of the systems driving and regulating motivated behaviors. Because DA cells exhibit diverse states of activity, which crucially depend on their intrinsic properties and afferent drive, the understanding of the role played by endocannabinoids in synaptic modulations is critical for their overall functions. Particularly, endocannabinoids by selectively inhibiting afferent activity may alter the functional states of DA neurons and potentiate the responsiveness of the reward system to phasic DA.

Survival of individuals and conservation of the species strictly depend on the drive by which animals seek natural reinforcers such as food, water, sex, and maternal care. Dopamine (DA) neurons in the ventral tegmental area (VTA), lying centrally amidst the limbic circuit and the basal ganglia, have adapted to organize several aspects of motivated behaviors (Ungless, 2004; Fields et al., 2007) and to provide the behavioral flexibility necessary for the organism to adjust itself into a constantly changing environment.

To carry out this task, midbrain DA neurons exhibit diverse states of activity that crucially depend on their intrinsic properties, afferent drive, and retrograde messengers (reviewed in Marinelli et al., 2006a; Sesack and Grace, 2010; Morikawa and Paladini, 2011).

In vivo, along with a regular, rhythmic discharge similar to that observed in vitro (Grace and Onn, 1989; Khaliq and Bean, 2010), spontaneously active DA neurons exhibit an irregular, single-spike firing mode, and a bursting pattern. It is the switch between these activity states that is believed to differentially regulate DA output in downstream structures, with single-spiking determining extracellular, “tonic” DA levels and burst firing leading to transient synaptic “phasic” DA levels (reviewed in Grace et al., 2007). Notably, “phasic” DA release is thought to represent the behaviorally relevant signal sent to postsynaptic targets to indicate reward-related cues and facilitate goal-directed action (see Grace et al., 2007 for a review and references therein).

Firing pattern of DA neurons is correlated with specific behavioral stimuli: these cells fire in bursts and release high DA levels especially when a reward is unexpected or larger than expected (Schultz, 1998, 2002, 2006; Schultz and Dickinson, 2000). After training, when reward delivery is reliably preceded by a sensory cue, DA neurons fire in bursts in response to the conditioned stimulus (Schultz, 1998). On the other hand, these cells pause their firing when no reward or punishment are delivered (Schultz, 1998). Under this aspect, DA neurons have been postulated to encode reward prediction error (Bayer and Glimcher, 2005; Schultz, 2006) and DA to be a powerful learning signal, evolutionary adapted to energize the organism toward natural reinforcers. Phasic depression of firing rate and DA release, on the other hand, can be interpreted as a signal for reversal learning (Schultz, 1998; Robbins and Arnsten, 2009; Kehagia et al., 2010), essential for the animal to attain that degree of behavioral plasticity necessary in an ever-changing environment. Deficits in reversal learning impair the ability of the animal to devaluate reward-related cues when they no longer predict reward and to avoid idle perseveration into unproductive behaviors (Kehagia et al., 2010). This neural circuit can be hijacked by drugs of abuse, which are able to become major reinforcers and overcome the natural ones. Additionally, addicting drugs strongly impair reversal learning and maintain perseveration of drug taking despite of negative and unpleasant consequences of drug use (Dagher and Robbins, 2009).

This dynamic behaviorally driven regulation of DA neuron activity must rely on efferent inputs arising from other brain regions directly involved by sensory, motor, and cognitive information. Hence, both burst firing and pauses depend on the balance between excitatory and inhibitory inputs impinging on DA neurons and interacting with the intrinsic pacemaker currents observed in vitro (Lobb et al., 2010; Morikawa and Paladini, 2011). Excitatory projections arise mainly from the glutamatergic prefrontal cortex and bed nucleus of the stria terminalis, and the glutamatergic–cholinergic pedunculopontine and laterodorsal tegmental nuclei. Inhibitory inputs to the VTA arise from neighboring GABAergic interneurons (Grace and Onn, 1989; Bayer and Pickel, 1991; Omelchenko and Sesack, 2009) and from the basal ganglia (Marinelli et al., 2006a; Sesack and Grace, 2010; Morikawa and Paladini, 2011), but not from the nucleus accumbens, whose GABA neurons target non-DA VTA neurons (Xia et al., 2011). A major GABAergic inputs to DA neurons arises also from a region located at the caudal tail of the VTA, termed rostromedial tegmental nucleus (RMTg). The RMTg has been recently discovered as an important source of GABAergic projections to midbrain DA neurons (Jhou et al., 2009; Kaufling et al., 2009; Lecca et al., 2011a,b; Figure 1).

Figure 1

The rostromedial tegmental nucleus (RMTg)–VTA pathway in aversion and reward mechanisms: possible role of endocannabinoids. (A) γ-Aminobutyric (GABA) neurons in the RMTg receive a major glutamatergic (GLU) input from lateral habenula (LHb), show phasic activation after aversive stimuli and inhibit ventral tegmental area (VTA) dopamine (DA) neurons. The inset displays oscilloscope traces recorded from a DA neuron inhibited for ∼ 100 ms following RMTg stimulation. The graphs represent rate histograms recorded from RMTg and DA neurons and show increase or decrease in firing rate, respectively, following a painful stimulus (paw pinch). Rate histograms and oscilloscope traces are adapted from Lecca et al. (2011a,b). We hypothesize that pauses in DA neuron firing rate, which are known to encode reward omission and punishment, might be mediated by the RMTg and regulated by endocannabinoids (i.e., 2-arachidonoylglycerol, 2-AG) activating presynaptic CB1 receptors. DA neurons receive also important glutamatergic afferents arising from various brain regions. Glutamatergic terminals are also endowed with presynaptic CB1 receptors. (B) Acute drug administration or reinstatement of drug self-administration excite DA neurons and DA release in terminal regions. On the other hand, addicting drugs also inhibit RMTg neurons (Lecca et al., 2011a) and/or suppress RMTg-induced inhibition of DA neurons (Lecca et al., 2011b), contributing to their excitation via a disinhibition mechanism. The oscilloscope traces in the inset show that RMTg-induced inhibition of the same DA neuron as in (A) is attenuated following cannabinoid administration (from Lecca et al., 2011b). Hence, we hypothesize that endocannabinoids might modulate the strength of these inhibitory afferents via long- or short-term forms of synaptic plasticity (such as depolarization-induced suppression of inhibition, DSI or long-term depression of GABA afferents, LTDGABA, respectively). The picture is further complicated, however, by the fact that endocannabinoids also modulate glutamatergic afferents, inducing functionally opposite forms of synaptic plasticity (such as depolarization-induced suppression of excitation, DSE or long-term depression of glutamatergic afferents, LTD).

Endocannabinoids and Synaptic Plasticity in the VTA

How incoming inputs are integrated and reverberate into specific firing patterns also depends on retrograde transmitters such as endocannabinoids (Melis and Pistis, 2007).

Endocannabinoids are unconventional lipid neurotransmitters/neuromodulators – synthesized on demand from membrane phospholipids – whose functions include retrograde signaling in the brain by modulating and/or mediating several types of synaptic plasticity (Kano et al., 2009). The best characterized endocannabinoids are anandamide (Devane et al., 1992) and 2-arachidonoylglycerol (2-AG; Mechoulam et al., 1995; Sugiura et al., 1995). Once released, they activate presynaptic type 1 cannabinoid receptors (CB1) and inhibit neurotransmitter release influencing both short- and long-term forms of synaptic plasticity.

Similarly to other neurons, DA neurons use endocannabinoids as retrograde neurotransmitters (Melis and Pistis, 2007). These molecules, with particular regard to 2-AG, and endocannabinoid-like N-acylethanolamines (NAEs; see below), have been demonstrated to be efficient means to dampen the activity of external afferents and permit DA neurons to fine tune their own function (Melis et al., 2004a,b, 2010; Melis and Pistis, 2007) to optimally respond to behavioral stimuli.

Regarding endocannabinoid-mediated synaptic plasticity, the best studied synapse in the VTA is the excitatory afferent arising from rostral/cortical regions. Hence, VTA DA neurons have been shown to release endocannabinoids to decreased glutamate release (Melis et al., 2004a,b, 2006; Marinelli et al., 2006b; Haj-Dahmane and Shen, 2010; Kortleven et al., 2011). Particularly, to date, the endocannabinoids that have been identified as fine modulators of excitatory synaptic transmission within the VTA are mainly 2-AG and N-arachidonoyl-dopamine (NADA; Melis et al., 2004a; Marinelli et al., 2006b; Haj-Dahmane and Shen, 2010; Kortleven et al., 2011). Notably, anandamide – though present within the midbrain (Marinelli et al., 2003; Melis et al., 2006) – plays a role as endovanilloid on ionotropic transient receptor potential vanilloid type 1 (TRPV1; Marinelli et al., 2003) rather than as an endocannabinoid on CB1 (Melis et al., 2006). In contrast, the endocannabinoid NADA, which can act depending on the conditions on either CB1 or TRPV1 at both inhibitory and excitatory synapses, can only be detected upon K+-induced depolarization (Marinelli et al., 2006b), thus raising issues on its role under physiological conditions.

Conversely, 2-AG appears as the key signaling molecule released on demand by VTA DA neurons. Hence, 2-AG mediates depolarization-induced suppression of excitation (Melis et al., 2006), a form of short-term plasticity that might serve to limit pathological excitation of VTA DA neurons, such as under ischemic–reperfusion injury (Melis et al., 2006). Additionally, behaviorally relevant patterns of synaptic activity such as a brief burst (2 s, 5 Hz) of excitatory synaptic activity onto DA neurons activate mGluR1 and increase intracellular Ca2+ levels, thus leading to 2-AG release, which transiently and selectively reduces excitatory inputs and DA neuron spike and burst probability (Melis et al., 2004a; Pillolla et al., 2007). Hence, it was suggested that 2-AG operates locally within the VTA as a device for DA neurons to switch their firing pattern and activity in response to stimuli (Melis et al., 2004a). Consistently, 2-AG synthetic enzyme sn-1-diacylglycerol lipase-α has been found to be widely expressed in the VTA on DA neurons in proximity of both glutamatergic and GABAergic synapses (Mátyás et al., 2008). Accordingly, different groups have shown that 2-AG plays a role in diverse forms of long-term synaptic plasticity expressed by VTA DA neurons (Pan et al., 2008a; Haj-Dahmane and Shen, 2010; Kortleven et al., 2011). Remarkably, 2-AG mediates long-term depression (LTD; Haj-Dahmane and Shen, 2010), and inhibits long-term potentiation (LTP; Kortleven et al., 2011) at excitatory synapses, consistently with its role in short forms of plasticity at these synapses. Additionally, 2-AG mediates LTD at GABAergic synapses (LTDGABA; Pan et al., 2008a,b, 2011).

New Endocannabinoid-Like Players in the VTA: OEA and PEA

Along with anandamide, the enzyme N-acylphosphatidylethanolamine-hydrolyzing phospholipase D (NAPE-PLD) also generates other NAEs, such as the anorectic and lipolytic oleoylethanolamide (OEA) and the anti-inflammatory palmitoylethanolamide (PEA). Although OEA and PEA are not endocannabinoids, but ligands of peroxisome proliferator-activated receptors (PPAR), they are considered endocannabinoid-related molecules (Pistis and Melis, 2010). In fact, they share with anandamide both the anabolic and degradative pathway (Lambert and Di Marzo, 1999), and they also compete with anandamide for its hydrolysis by the enzyme fatty acid amide hydrolase (FAAH), thereby causing an indirect activation of other receptors and the so called “entourage effect” (De Petrocellis et al., 2001; Di Marzo et al., 2001; Jonsson et al., 2001; Smart et al., 2002).

Emerging evidence suggests that OEA and PEA exert their actions in the VTA (Melis et al., 2008, 2010). Particularly, they decrease spontaneous activity of VTA DA cells and the number of spontaneously active DA neurons through a rapid non-genomic mechanism downstream to activation of type alpha-PPAR (PPARα; Melis et al., 2010). In 2008, the discovery that OEA and PEA block nicotine-induced excitation of VTA DA neurons both in vivo and in vitro (Melis et al., 2008) shed light into their physiological role as negative modulators on nicotinic acetylcholine receptors containing β2 subunits (β2*-nAChRs; Melis et al., 2010). These effects are blocked by the tyrosine kinase inhibitor genistein (Melis et al., 2008), thus suggesting the phosphorylation of β2*-nAChRs as a plausible underlying mechanism of NAE actions (Melis et al., 2010; Pistis and Melis, 2010).

Despite the interest, the potential physiological significance, and the implications for NAE actions on VTA DA neurons, many questions remain unanswered. First, OEA and PEA levels have neither been measured in the VTA nor compared with other related lipid molecules. Second, it is not known whether these molecules are released on demand by VTA DA neurons during physiological or pathological circumstances. Third, it is not clear whether they help in controlling the state of β2*-nAChRs or the number of functional surface expressed β2*-nAChRs. Since NAEs are found in all mammalian tissues (Hansen et al., 2001; Hansen and Diep, 2009), one could expect OEA and PEA to be constitutively present in the VTA. Additionally, it could be speculated that their synthesis and/or release occurs on demand upon cholinergic receptor activation, given their role as important negative modulators of β2*-nAChRs. In a similar fashion, NAEs are produced by cortical neurons in primary cultures (Stella and Piomelli, 2001). If so, under these conditions, in the VTA acetylcholine and NAEs might control each other in a negative feedback mechanism, where OEA and PEA negatively modulate β2*-nAChRs downstream to PPARα activation, and their biosynthesis is increased by cholinergic agonists (Pistis and Melis, 2010).

Given that β2*-nAChRs play a crucial role in mediating the switch from “basal” to “excited” states of VTA DA neurons (Mameli-Engvall et al., 2006), we currently hypothesize that engagement of PPARα by NAEs, by making DA neurons less sensitive to external information, might translate into prevention of an erroneous attribution of saliency to otherwise irrelevant stimuli. Thus, engagement of PPARα by NAEs may have protective effects and confer VTA DA neurons with the exquisite resilience from excitotoxicity, which – together with individual differences in ion channel makeup – might grant them less vulnerability to metabolic dysfunction.

Role of Endocannabinoid-Mediated Synaptic Modulation on DA Neurons in Goal-Directed Behavior/Reward Seeking

Despite a great deal of research, how – and if – endocannabinoid-mediated modulation of DA neurons translates into behavior is still to be established. Among neurotransmitters/neuromodulators involved in the different phases of compulsive seeking of natural or drug-induced reward, the endocannabinoid system has gained particular attention. Indeed, pharmacological manipulation of endocannabinoids can influence reward-seeking behavior, but if the DA system is directly involved is less clear. For example, blockade of CB1 receptors with rimonabant inhibits nicotine-, alcohol-, and cocaine-induced phasic DA release in the ventral striatum measured with in vivo fast-scan voltammetry (Cheer et al., 2007). Earlier work by Cohen et al. (2002) showed that rimonabant inhibits nicotine-induced DA release in the nucleus accumbens, as measured by brain microdialysis, and nicotine self-administration. Despite evidence on drug-induced DA release, electrophysiological studies failed to demonstrate effects of CB1 antagonists on drug-induced excitation of DA neurons (Melis et al., 2000, 2008), with the exception of alcohol (Perra et al., 2005). Thus, it can be inferred that endocannabinoids modulate drug-induced DA release at the synaptic level, without affecting firing rate of DA cells. This piece of evidence is at odds with the important modulatory role of endocannabinoids on synaptic afferents to DA neurons. One explanation is that endocannabinoids might be primarily involved in the effects of chronic drug administration, by modulating multiple long-term, rather than short-term, forms of synaptic plasticity, i.e., facilitating LTDGABA (Pan et al., 2008a,b, 2011) or LTD on DA neurons (Haj-Dahmane and Shen, 2010), or inhibiting LTP (Kortleven et al., 2011). How these functional opposing effects of endocannabinoids on long-term synaptic plasticity combine with each other, and, more importantly, in which phase of the addiction process they are engaged remains to be established.

One possibility is that a persistent LTP at excitatory synapses onto DA cells such as that induced by cocaine (Chen et al., 2008) would lead to 2-AG-mediated LTD at the same synapses to protect DA cells from overexcitation. However, under these conditions, 2-AG rather mediates LTDGABA (Pan et al., 2008b), thus raising questions on its own role. Hence, it seems unlikely that 2-AG would act simultaneously at both inhibitory and excitatory synapses, although most of these latter are equipped with the key molecular players required for 2-AG signaling (Mátyás et al., 2008; Kortleven et al., 2011). Alternatively, one can suggest that cocaine-induced released 2-AG, by removing the inhibitory inputs, can induce DA cells to burst (F. Georges, personal communication), consistent with the disinhibition bursts produced by removal of GABA (Lobb et al., 2010). With this in mind, one would expect that only VTA DA neurons encoding strong reward salience may use 2-AG to escape from GABAergic inhibition and enhance their burst firing (Riegel and Lupica, 2005; Mátyás et al., 2008; Figure 1B). Given that GABAergic afferents onto DA neurons arise from different districts, the precise distribution pattern of key molecular players needed for 2-AG actions can provide the correct framework to understand 2-AG contribution in the context of drug addiction.

Among behaviors related to natural or drug-induced reinforcement, reinstatement (equivalent to relapse in humans) is most sensitive to CB1 receptor activation/blockade. Indeed, following extinction of drug-seeking behaviors such as drug self-administration, CB1 receptor antagonists have been reported to reliably block cue-induced or drug-induced reinstatement of drug-seeking behavior (Fattore et al., 2003, 2005, 2007, 2011; Le Foll and Goldberg, 2005; Ward et al., 2009; Schindler et al., 2011; Yu et al., 2011). Vice versa, CB1 agonists reinstate drug-seeking behaviors (Justinova et al., 2008; Scherma et al., 2008; Gamaleddin et al., 2012). One hypothesis is that supramaximal stimulation of CB1 receptors, such as that attained by exogenous cannabinoid agonists, might desensitize or occlude 2-AG-mediated short- or long-term forms of synaptic plasticity primarily on glutamatergic afferents to DA neurons (such as short-lasting suppression of excitation or LTD) by which DA cells might be sensitized to priming with drugs or with drug-associated cues.

Despite the remarkable effect on reinstatement, consensus is emerging in the literature on the lack of effect of (endo)cannabinoids on extinction of learned appetitively motivated tasks (Hernandez and Cheer, 2011). On the other hand, endocannabinoid mechanisms are strongly engaged in extinction of negatively motivated behavior (Marsicano et al., 2002; Lutz, 2007). These pieces of evidence lead us to theorize that endocannabinoids might also participate in the modulation of neural circuits underlying behavioral responses to aversive stimuli. Although amygdala and hippocampus play a major role in aversion mechanisms, DA neurons or their afferents might be also involved. Among afferents to DA neurons, the RMTg plays a pivotal role in processing aversive and appetitive stimuli (Jhou et al., 2009). In fact, RMTg neurons are excited by noxious stimuli and inhibited by rewarding stimuli (Jhou et al., 2009; Hong et al., 2011). These cells form inhibitory synapses with DA neurons (Kaufling et al., 2010; Balcita-Pedicino et al., 2011) and activation of the RMTg inhibits DA neurons both in the rat (Lecca et al., 2011a,b) and in the monkey (Hong et al., 2011). Pauses in DA neuron firing, which encode lack of reward or punishment, might be mediated by the RMTg (Figure 1A) and be regulated by endocannabinoids. In support to this idea, cannabinoids were shown to depress firing rate of RMTg neurons by reducing the strength of excitatory postsynaptic currents (Lecca et al., 2011a; Figure 1B). Moreover, they depress inhibitory inputs on DA neurons arising from RMTg stimulation (Lecca et al., 2011b). Although the expression of CB1 receptors either in the RMTg neurons’ cell body or terminals remains to be demonstrated, electrophysiological evidence supports the notion that the endocannabinoid system might modulate these afferents and finely adjust DA neuron responses to punishment/aversion or reward omission (Figures 1A,B). Consistently, cannabinoids disrupt reversal learning by conferring behavioral inflexibility and increasing perseveration errors (Egerton et al., 2005; Hill et al., 2006; Harte and Dow-Edwards, 2011; Sokolic et al., 2011). On the other hand, RMTg terminals might be a major target for endocannabinoid-mediated short- and long-term forms of synaptic plasticity (depolarization-induced suppression of inhibition, DSI, or LTDGABA, respectively). Hence, by depressing these important inhibitory afferents, endocannabinoids might indirectly excite DA neurons and/or sensitize them toward behavioral or drug-related stimuli, this mechanism resulting into an enhanced stimulus-evoked firing activity and DA release in terminal regions (Figure 1B).

Concluding Remarks

Further studies on the knowledge of the anatomy, physiology, biochemistry, and pharmacology of these circuits might contribute to new therapeutic strategies for treatment of psychiatric disorders characterized by a dysregulation of the endocannabinoid system in the reward DA circuitry.

It appears, therefore, compelling to investigate whether diverse sets of synapses, most likely arising from extrinsic sources, are differently equipped/enriched with the discrete players of 2-AG signaling machinery. One interesting possibility could be that in the VTA, similarly to the amygdala (Yoshida et al., 2011), a unique molecular convergence of 2-AG signaling molecules would occur at specific synapses, resulting in a targeted control of the excitability of DA neurons. If this is the case, many of the unraveled issues regarding (endo)cannabinoid-mediated effects on DA neurons and the dependent behavior would be solved.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.