The role of dopamine in risk taking: a specific look at Parkinson's disease and gambling.

An influential model suggests that dopamine signals the difference between predicted and experienced reward. In this way, dopamine can act as a learning signal that can shape behaviors to maximize rewards and avoid punishments. Dopamine is also thought to invigorate reward seeking behavior. Loss of dopamine signaling is the major abnormality in Parkinson's disease. Dopamine agonists have been implicated in the occurrence of impulse control disorders in Parkinson's disease patients, the most common being pathological gambling, compulsive sexual behavior, and compulsive buying. Recently, a number of functional imaging studies investigating impulse control disorders in Parkinson's disease have been published. Here we review this literature, and attempt to place it within a decision-making framework in which potential gains and losses are evaluated to arrive at optimum choices. We also provide a hypothetical but still incomplete model on the effect of dopamine agonist treatment on these value and risk assessments. Two of the main brain structures thought to be involved in computing aspects of reward and loss are the ventral striatum (VStr) and the insula, both dopamine projection sites. Both structures are consistently implicated in functional brain imaging studies of pathological gambling in Parkinson's disease.

Gambling as a disorder of reward and punishment processing

Pathological gambling can be conceptualized as a disorder of reward and punishment processing, whereby the gambler selects an immediate but risky opportunity to obtain money over the larger, more probable opportunity to save money (Ochoa et al., 2013). Indeed, gambling is typically conceptualized as a disorder of impulsivity, in which decision-making is rash and relatively uninfluenced by future consequences. Pathological gamblers demonstrate increased impulsivity and increased delayed discounting on laboratory measures (Verdejo-Garcia et al., 2008). The coupling of increased reward seeking behavior with insensitivity to negative consequences may explain the persistence of gambling in the face of overall monetary losses (Vitaro et al., 1999; Petry, 2001b; Cavedini et al., 2002). This conceptual framework is similar to that used in drug addiction, where seeking immediate gains while minimizing potential risks is ubiquitous. Hallmarks of addiction include cravings or compulsions, a loss of control, and continued engagement in behaviors that maintain the addiction despite repeated negative consequences (American Psychiatric Association, 2000). Similarly, pathological gambling can be referred to as a behavioral addiction because it shares many common features with drug-addiction, such as compulsion and loss of control over one’s behavior, as well as continuation of the behavior in the face of negative consequences (Grant et al., 2006; Goodman, 2008). Pathological gamblers exhibit uncontrollable cravings, tolerance, habituation, and withdrawal symptoms, similar to those of drug addicts (Wray and Dickerson, 1981; Castellani and Rugle, 1995; Duvarci and Varan, 2000; Potenza et al., 2003). Moreover, both pathological gambling and substance abuse are associated with the same specific personality traits, namely sensation seeking and impulsivity (Zuckerman and Neeb, 1979; Castellani and Rugle, 1995), which index heightened arousal to potential rewards and reduced self-control and inhibitory function. The high comorbidity between substance dependence (drugs and alcohol) and pathological gambling (Petry, 2001a; Petry et al., 2005), and evidence for common genetic factors, point to the two disorders having overlapping etiologies (Slutske et al., 2000; Goodman, 2008).

One useful model views reward and punishment learning as inherent components in the decision-making process. Decision-making can be broken down to the weighing of the probability and value of reward against potential costs (e.g., negative consequences). Other factors such as outcome ambiguity and variance (sometimes referred to as risk) also affect individual choices (Huettel et al., 2006), but here we will only consider potential gains and losses as determinants of decision-making while gambling. We will also take “risk” to mean the potential loss attached to any choice. Risk, as so defined, increases with the magnitude and probability of potential losses. In fact, risk-taking may be seen as an indicator of the balance existing between computations of potential gains and losses. Two of the main brain structures thought to be involved in these computations are the ventral striatum (VStr) and the insula, both dopamine projection sites. Both have been linked to computations of value, with the VStr being especially responsive to reward prediction error (RPE), encoding gain anticipation positively and loss anticipation negatively (Rutledge et al., 2010; Bartra et al., 2013), and the insula responding predominantly to losses and loss anticipation in some studies (Knutson and Greer, 2008) or to both positive and negative outcomes in others (Campbell-Meiklejohn et al., 2008; Rutledge et al., 2010). Bartra et al.’s meta-analysis (Figure 1) suggests that the insula encodes arousal or salience as opposed to value, as it responds positively to both gains and losses. This meta-analysis also raises the possibility of a greater role for the insula in the assessment of risk and losses than gains (compare panels A and B in Figure 1). Alteration of the balance between these gain and loss anticipation systems may underlie the inappropriate choice behaviors that occur in disorders such as addiction, gambling and impulse control disorders.

Figure 1

Meta-analysis of fMRI studies of value (taken from Bartra et al., . The authors extracted peak coordinates of activation from 206 published fMRI studies that investigated value computations. (A) Significant clustering of positive responses. (B) Significant clustering of negative responses. (C) Conjunction maps, showing regions with significant clustering for both positive and negative responses. (D) Results of a between-category comparison, showing regions with significantly greater clustering for positive than negative effects. (E) Detail of the striatum, illustrating overlap between the conjunction map (Panel C) and the difference map (Panel D). These data demonstrate the relative response of anterior insula, striatum and ventromedial PFC to positive and negative value.

Recent research suggests that differences in brain function, structure, and biochemistry are present in those who develop gambling problems, with dopamine being a common etiological factor. Imaging studies have demonstrated an increase in mesolimbic dopamine release during gambling tasks in healthy subjects (Thut et al., 1997; Zald et al., 2004; Hakyemez et al., 2008). However it should be noted that unpredictable reward tasks have the ability to cause a suppression and enhancement of dopamine transmission in different regions of the striatum (Zald et al., 2004; Hakyemez et al., 2008). Earlier research on pathological gamblers suggested altered dopaminergic and noradrenergic systems, as found through a decrease in concentration of dopamine and an increase in cerebrospinal fluid levels of 3,4-dihydroxyphenyl-acetic acid and homovanilic acid (Bergh et al., 1997). Pathological gamblers have also been reported to have higher cerebrospinal fluid levels of 3-methoxy-4-hydroxyphenylglycol, a major metabolite of norepinephrine, as well as significantly greater urinary outputs of norepinephrine in comparison to controls (Roy et al., 1988), indicative of a functional disturbance of the noradrenergic system. In addition there is evidence that genetic polymorphisms affecting dopaminergic neurotransmission act as risk factors for problem gambling (Lobo and Kennedy, 2006).

Dopamine in reinforcement

Considerable evidence from animal studies, implicating dopamine in behavioral reinforcement, provides a neurobiological substrate that could encompass processing of natural rewards, such as food and sex, as well as drugs of abuse and pathological gambling (Di Chiara and Imperato, 1988; Wise and Rompre, 1989; Wise, 1996, 2013). The observations of Schultz and others (Schultz et al., 1998; Schultz, 2002) confirmed a role for dopamine neurons in response to rewards; however the current model of dopamine signaling can be traced to a seminal paper by Montague, Dayan and Schultz (Schultz et al., 1997), where it was argued that the firing pattern of dopamine neurons did not signal reward per se, but a RPE signal, similar to those used in machine learning. This finding, along with evidence that dopamine could modulate synaptic plasticity (Calabresi et al., 2007; Surmeier et al., 2010) led to the theory that dopamine acts as a learning (or reinforcement) signal that shapes future motivated behavior. Subsequent research has shown that dopamine may also encode predictions about upcoming rewards and reward rate, thus acting as a value signal in the mesocortical and mesolimbic dopaminergic pathways (Montague and Berns, 2002).

The main projection site of dopamine neurons is the striatum, whose connectivity to frontal, limbic and insular cortex, provides a mechanism whereby dopamine can act as a prediction error signal driving both “Go” learning, which relates to actions with positive outcomes, and “No Go” or avoidance learning, which relates to actions that lead to punishment or an absence of reward. First, dopamine signaling operates in two modes (Grace, 2000): slow constant release of dopamine regulates tonic levels, which mostly signal via dopamine D2 receptors on striatal medium spiny neurons; phasic bursts of dopamine firing lead to large increases in synaptic dopamine which signal via both the D1 and D2 receptor systems. D1 receptors have low affinity for dopamine (Marcellino et al., 2012) and only respond to large increases in synaptic dopamine released during phasic dopamine neuron bursts that reflect positive RPEs, supporting learning to approach rewarding stimuli (Frank, 2005). Dopamine D2 receptors, on the other hand, have a higher affinity for dopamine, allowing them to respond to tonic dopamine signaling, and to detect transient reductions in tonic dopamine levels that follow pauses in dopamine neuron firing during negative RPEs. This facilitates learning to avoid negative outcomes (Frank, 2005). The cortico-striatal system can be divided into a direct and an indirect pathway (Figure 2), which have opposite effects on the thalamus and hence cortex (Albin et al., 1989). In the dorsal striatum, receptors are segregated, with the D1 receptors within the direct pathway, related to action selection, while the D2 receptors control response inhibition within the indirect pathway (Mink, 1996). This separation allows dopamine to drive both reward (increases in dopamine signaling a better outcome than expected) and punishment (reductions in tonic dopamine indicated a worse outcome than expected). Frank proposed a model in which phasic dopamine bursts following rewards promote positive reinforcement while reductions in tonic dopamine levels lead to negative reinforcement, each controlled by the D1/direct pathway and the D2/indirect pathway, respectively (Cohen and Frank, 2009). This computational model suggests that the RPE dopamine signal promotes learning from positive outcomes via stimulation of D1 receptors, whereas learning to avoid negative outcomes is mediated via disinhibition of indirect pathway striatal neurons secondary to a reduction of D2 receptor stimulation during dopamine pauses (Cohen and Frank, 2009). A negative outcome (punishment or lack of an expected reward) leads to pause in the firing of dopamine neurons, which then leads to a transient reduction in tonic dopamine. It should also be noted that D2 receptor stimulation reduces excitability of neurons in the indirect pathway (Hernandez-Lopez et al., 2000), therefore, reductions in D2 receptor signaling have the effect of activating the inhibitory “No Go” pathway. This allows for bidirectional positive and negative reinforcement signaling by dopamine neurons. Support for this model has been provided by numerous experiments. Parkinson’s disease patients show enhanced positive learning when on their medications, but improved negative learning while off medication (Frank et al., 2004). Pharmacological manipulations also support the model (Frank and O’Reilly, 2006; Pizzagalli et al., 2008). The striatal release of dopamine is linked to associative learning and habit formation via control of corticostriatal synaptic plasticity, which is affected in an opposite manner by D1 and D2 signaling (Shen et al., 2008). D1 dopamine receptor signaling promotes long-term potentiation (Reynolds et al., 2001; Calabresi et al., 2007), whereas D2 receptor signaling promotes long-term depression (Gerdeman et al., 2002; Kreitzer and Malenka, 2007). Note that this model has been tested most thoroughly at the level of the striatum. Multivariate analysis of fMRI data shows that reinforcement and punishment signals are ubiquitous in the brain, most notably in the entire frontal cortex and striatum (Vickery et al., 2011). Less is known about the information signaled by dopamine projections to brain areas other than the striatum, such as frontal cortex, insula, hippocampus and amygdala, or how the RPE signal is used by these areas.

Figure 2

Basal ganglia model. A possible model whereby basal ganglia compute the utility of gains and losses via two segregated pathways in the corticostriato-thalamocortical circuit. Striatal output neurons of the direct pathway express D1 receptors and project to the internal globus pallidus (GPi) and the substantia nigra pars reticulata (SNr), and has an action selection effect on cerebral cortex. Striatal output neurons in the indirect pathway express D2 receptors and reduce the tonic inhibition of the external globus pallidus (GPe) on the GPi/SNr, which leads to action inhibition in the cortex. D1 receptors respond mainly to phasic (high concentration) dopamine signaling due to their low affinity for dopamine. D2 receptors have high affinity for dopamine and respond to lower tonic dopamine levels. Excitatory projections in green, inhibitory in red.

Striatum and monetary reward

In human functional neuroimaging studies, changes in brain activation have been demonstrated consistently in response to monetary rewards (Thut et al., 1997; Elliott et al., 2000; Knutson et al., 2000; Breiter et al., 2001; O’Doherty et al., 2007). Further, studies have teased apart the different brain areas involved in the various components of monetary reward, such as anticipation, feedback, winning and losing. There seems to be a specialization within dopamine projection sites in relation to monetary reward: anticipation of monetary reward increases activation in the VStr, which includes the nucleus accumbens, while rewarding outcomes increase activation in the ventral medial prefrontal cortex, dorsal striatum, and posterior cingulate, with deactivation in the aforementioned regions during reward omission (Elliott et al., 2000; Breiter et al., 2001; Knutson et al., 2001b; Tricomi et al., 2004). Neuroimaging experiments in humans suggest that VStr activity strongly correlates with expected value, as well as magnitude and probability (Breiter et al., 2001; Knutson et al., 2001a, 2005; Abler et al., 2006; Yacubian et al., 2006; Rolls et al., 2008). Work by D’Ardenne et al. (2008) supports a role for the mesolimbic dopamine system in monetary RPE signaling. Activation of the ventral tegmental area, the origin of the mesolimbic dopamine circuit, reflected positive RPEs, whereas the VStr encoded positive and negative RPEs. Similarly, Tom et al. (2007) showed that VStr activity reflected potential monetary gains and losses bidirectionally. This study also demonstrated that these neural signals reflected individual variations in loss aversion, the tendency for losses to be more impactful than potential gains. Finally, the influential actor-critic model (Sutton and Barto, 1998) proposes that the VStr uses prediction errors to update information about expected future rewards while the dorsal striatum uses this same prediction error signal to encode information about actions that are likely to lead to reward. This distinction has found support from fMRI experiments (O’Doherty et al., 2004; Kahnt et al., 2009). Interestingly, the ability to update behavior in response to RPE was shown to correlate with functional connectivity between dorsal striatum and dopaminergic midbrain (Kahnt et al., 2009). The imaging studies mentioned here support the theory of dopamine as a RPE signal, at least in its striatal projection.

Insula and risk

The insula is frequently activated in functional neuroimaging experiments (Duncan and Owen, 2000; Yarkoni et al., 2011). Functionally it can be divided into three distinct subregions: a ventroanterior region associated with chemosensory (Pritchard et al., 1999) and socio-emotional processing (Sanfey et al., 2003; Chang and Sanfey, 2009), a dorsoanterior region associated with higher cognitive processing (Eckert et al., 2009), and a posterior region associated with pain and sensorimotor processing (Craig, 2002; Wager et al., 2004). Different functional insular areas project to different striatal targets: the VStr receives insular projections primarily related to food and reward, whereas the dorsolateral striatum receives insular inputs related to somatosensation (Chikama et al., 1997).

The insular cortex is involved in decision-making processes that involve uncertain risk and reward. Specifically, fMRI studies have reported insular cortex involvement in risk-averse decisions (Kuhnen and Knutson, 2005), risk avoidance and the representation of loss prediction (Paulus et al., 2003), monetary uncertainty (Critchley et al., 2001), and encoding a risk prediction error (Preuschoff et al., 2008). Patients with insular cortex damage place higher wagers in comparison with healthy participants and their betting is less sensitive to the odds of winning, with high wagers even at unfavorable odds (Clark et al., 2008). Other research suggests that optimum decisions involving risk depend on the integrity of the insular cortex, showing that insula lesion patients have altered decision-making involving both risky gains and risky losses (Weller et al., 2009) (However see Christopoulos et al., 2009). Specifically, insula damage was associated with a relative insensitivity to expected value differences between choices. Previous research has shown that there is a dissociation between insula and VStr, with VStr activation preceding risk-seeking choices, and anterior insula activation predicting risk-averse choices (Kuhnen and Knutson, 2005) suggesting that the VStr represents gain prediction (Knutson et al., 2001a), while anterior insula represents loss prediction (Paulus et al., 2003). While imaging studies also demonstrate a more general role of the anterior insula in signaling the valence (positive or negative) of potential rewards (Litt et al., 2011; Bartra et al., 2013) the lesion data argue that the anterior insular cortex has a role in risk evaluation, specifically in making risk-averse decisions. Indeed, in healthy subjects, the insula is part of a value network that appears to track potential losses in a way that correlates with individual loss aversion level (Canessa et al., 2013). It is possible that an imbalance between prefrontal-striatal circuitry and insular-striatal circuitry may lead to suboptimal choices when weighing potential gains and losses, as observed in pathological gamblers (Petry, 2001a; Goudriaan et al., 2005).

Pathological gambling among patients with Parkinson’s disease

Pathological gambling was first reported in the context of Parkinson’s disease and dopamine replacement therapy in 2000 (Molina et al., 2000). The lifetime prevalence of pathological gambling in the general public is approximately 0.9 to 2.5% (Shaffer et al., 1999). In Parkinson’s disease, the prevalence rates are higher, from 1.7 to 6.1% (Ambermoon et al., 2011; Callesen et al., 2013). The risk factors associated with the occurrence of pathological gambling in Parkinson’s disease are young age of Parkinson’s disease onset, a personal or family history of drug or alcohol abuse, depression, and relatively high impulsivity and novelty seeking personality scores (Voon et al., 2007b). Interestingly, these are similar to the risk factors for drug addiction and pathological gambling in the general population. Also, there have been reports of addiction to L-dopa in certain patients (e.g., Giovannoni et al., 2000), a phenomenon that had already been noted in the 1980s. It was perhaps initially surprising to find that Parkinson’s disease patients can become addicted to their own medication or develop behavioral addictions because they were thought to not possess the personality type typical of addicted individuals. They are generally described as industrious, punctual, inflexible, cautious, rigid, introverted, slow-tempered, with lack of impulsiveness and novelty seeking, and they have low lifetime risks for cigarette smoking, coffee drinking, and alcohol use predating Parkinson’s disease onset (Menza et al., 1993; Menza, 2000).

Dopamine replacement therapy has been implicated in the development of pathological gambling in Parkinson’s disease (Gschwandtner et al., 2001; Dodd et al., 2005) and a remission or reduction of pathological gambling is typically noted after reduction or cessation of dopamine agonist medication (Gschwandtner et al., 2001; Dodd et al., 2005). A broader set of behavioral addictions termed impulse control disorders, including but not limited to pathological gambling, compulsive sexual behavior, and compulsive buying, have been reported in association with dopamine replacement therapy (Weintraub et al., 2006; Voon et al., 2007a; Dagher and Robbins, 2009). Dopamine agonists (pramipexole, ropinirole and pergolide) appear to pose a greater risk than L-Dopa monotherapy (Seedat et al., 2000; Dodd et al., 2005; Pontone et al., 2006). Reducing the dopamine agonist and increasing L-Dopa to achieve same motor response abolished pathological gambling in affected individuals (Mamikonyan et al., 2008), while a cross-sectional study of over 3000 Parkinson’s disease patients found that taking a dopamine agonist increased the odds of developing an impulse control disorder by 2.72 (Weintraub et al., 2010). Finally, these side-effects of dopamine agonist therapy have been recently noted in other diseases, such as restless leg syndrome, fibromyalgia and prolactinomas (Davie, 2007; Driver-Dunckley et al., 2007; Quickfall and Suchowersky, 2007; Tippmann-Peikert et al., 2007; Falhammar and Yarker, 2009; Holman, 2009). It should be noted however that some studies have reported behavioral addictions and/or impulsivity and compulsivity in association with high-dose L-Dopa monotherapy (Molina et al., 2000), deep brain stimulation for Parkinson’s disease (Smeding et al., 2007), and in drug naïve Parkinson’s disease patients (Antonini et al., 2011), all in the absence of dopamine agonists. Nonetheless, the clinical evidence overwhelmingly supports the theory that dopamine agonism at the D2 receptor family is sufficient to cause impulse control disorders.

Brain imaging studies

Neurotransmitter imaging

Positron emission tomography (PET) imaging allows for changes in endogenous levels of dopamine to be inferred from changes in the binding of the [11C]raclopride to the dopamine D2 receptors. The first [11C]raclopride PET study in this area was on Parkinson’s patients with dopamine dysregulation syndrome. Dopamine dysregulation syndrome is characterized by the compulsive taking of dopaminergic drugs, which is often comorbid with impulse control disorders (Lawrence et al., 2003). Patients with dopamine dysregulation syndrome exhibited enhanced L-Dopa induced VStr dopamine release compared to similarly treated Parkinson’s disease patients not compulsively taking dopaminergic drugs (Evans et al., 2006). This was the first study to provide evidence for sensitization of mesolimbic dopamine circuitry in Parkinson’s disease patients prone to compulsive drug use. Subsequent studies have supported a relative hyperdopaminergic state in Parkinson’s disease patients with pathological gambling. Three studies mapping the concentration of dopamine reuptake transporters (DAT) have shown reduced levels in the VStr of Parkinson’s disease patients with impulse control disorders compared to unaffected patients (Cilia et al., 2010; Lee et al., 2014; Voon et al., 2014). Unfortunately the finding is non-specific, as reduced DAT concentration can index either reduced nerve terminals (and reduced dopamine signaling) or reduced DAT expression (and therefore increased tonic dopamine levels). Supporting the latter hypothesis, impulse control patients demonstrate reduced [11C]raclopride binding in the VStr compared to Parkinson’s controls (Steeves et al., 2009), which is also consistent with elevated tonic dopamine in this group. Note, however that this result failed to be replicated in a similar study (O’Sullivan et al., 2011).

However, these two [11C]raclopride PET studies reported a greater reduction of VStr binding potential (an index of dopamine release) during gambling (Steeves et al., 2009) and following reward-related cue exposure (images of food, money, sex) compared to neutral cues (O’Sullivan et al., 2011) in Parkinson’s disease patients with impulse control disorders compared to unaffected patients. This suggests an increased responsiveness of striatal reward circuitry to gambling and reward-related cues in those patients with impulse control disorders. In O’Sullivan et al. (2011) dopamine release was only detected in the VStr and only when subjects received a dose of oral L-Dopa just prior to scanning, consistent with post-mortem data in Parkinson’s disease showing that brain dopamine levels are much lower in dorsal than VStr (Kish et al., 1988). These results are therefore consistent with the sensitization hypothesis proposed by Evans et al. (2006). More recently it was reported that Parkinson’s disease patients with pathological gambling have a reduced concentration of dopamine autoreceptors in the midbrain (Ray et al., 2012), which is known to correlate with elevated dopaminergic responsivity and increased impulsivity (Buckholtz et al., 2010). Finally, in Parkinson’s disease patients, dopamine synthesis capacity, as measured by [18F]DOPA PET, correlates with a personality measure of disinhibition, itself a risk factor for pathological gambling and other addictions (Lawrence et al., 2013). In summary, PET studies provide converging evidence of heightened dopaminergic tone and increased dopamine response to reward cues as the underlying vulnerability in Parkinson’s disease patients who develop pathological gambling during dopamine agonist treatment.

Functional magnetic resonance imaging

Parkinson’s disease patients with pathological gambling show enhanced hemodynamic responses to gambling-related visual cues in the bilateral anterior cingulate cortex, left VStr, right precuneus and medial prefrontal cortex (Frosini et al., 2010). This is in line with similar experiments in pathological gambling without Parkinson’s disease (Crockford et al., 2005; Ko et al., 2009) and drug addiction (Wexler et al., 2001), supporting the view that impulse control disorders in Parkinson’s disease may be conceptualized as behavioral addictions.

Parkinson’s disease patients with an impulse control disorder show diminished BOLD activity in the right VStr during risk taking and significantly reduced resting cerebral blood flow in the right VStr compared to their healthy disease counterparts (Rao et al., 2010). Similarly, it was found that Parkinson’s disease patients with impulse control disorders showed a bias toward risky gambles compared to control patients, and that dopamine agonists enhanced risk taking while decreasing VStr activity (Voon et al., 2011). The authors suggested that dopamine agonists may decouple brain activity from risk information in vulnerable patients, thus favoring risky choices. Another fMRI study reported that, relative to Parkinson’s controls, impulse control disorder Parkinson’s patients had decreased anterior insular and orbitofrontal cortex RPE signals. They also showed that dopamine agonists increased the rate of learning from gain outcomes, and increased striatal RPE activity, suggesting that dopamine agonists may skew neural activity to encode “better than expected” outcomes in Parkinson’s disease patients susceptible to impulse control disorders (Voon et al., 2010).

While differences in striatal dopamine signaling may distinguish Parkinson’s disease patients who do and do not develop pathological gambling, the mechanism of action by which dopamine agonists change risk assessment remains unclear. Dopamine agonists change the way in which the brains of healthy individuals respond to the anticipation and feedback of rewards. During reward feedback, administration of a single dose of pramipexole to healthy adults caused decreased VStr activity in a lottery game (Riba et al., 2008). Similarly, there was reduced VStr activation when Parkinson’s patients received a dose of L-Dopa compared to placebo (Cools et al., 2007). This pattern of hypoactivation is reminiscent of that found in pathological gamblers without Parkinson’s disease (Reuter et al., 2005): during a simulated gambling task, pathological gamblers showed decreased activation with respect to controls in the ventromedial prefrontal cortex and the VStr. Severity of gambling was negatively correlated with the BOLD effect in the VStr and ventromedial prefrontal cortex, suggesting that hypoactivity is a predictor of gambling severity. As noted above, impulse control disorder Parkinson’s patients were found to have diminished resting perfusion as well as diminished BOLD activity during risk taking in the VStr compared to Parkinson’s controls (Rao et al., 2010). These studies suggest that dopamine agonists cause individuals to seek rewards and make risky choices (Riba et al., 2008), in the face of suppressed VStr response to rewards.

It should be noted however that reduced VStr activation in fMRI experiments does not necessarily indicate reduced dopaminergic signaling. There is evidence to support relatively spared mesolimbic dopamine signaling as the risk factor for pathological gambling in Parkinson’s disease. First, the repeated taking of a dopaminergic medication for the treatment of Parkinson’s disease could lead to sensitization of dopamine signaling. VStr sensitization has been shown following repeated amphetamine administration in humans (Boileau et al., 2006). Moreover, in Parkinson’s disease the ventral portion of striatum is relatively spared by the disease compared to the dorsal areas (Kish et al., 1988), and thus dopamine replacement therapy, while correcting the dopamine deficiency in the dorsal striatum to normal levels, has the potential to raise dopamine levels in the VStr circuit to higher than optimal levels (Cools et al., 2007). This “overdose” theory was first proposed by Gotham et al. (1988) to explain the fact that L-Dopa administration to Parkinson’s disease patients, while improving some cognitive deficits, could also cause specific impairments in other fronto-striatal cognitive tasks. In the case of impulse control disorders, we propose that excessive dopaminergic stimulation in the VStr obscures the dips in dopamine signaling related to negative prediction errors.

The insula has also been implicated in imaging studies of pathological gambling in Parkinson’s disease. In an fMRI study, Ye et al. (2010) found that during the anticipation of monetary rewards, a single dose of pramipexole (compared to placebo) increased the activity of the VStr, enhanced the interaction between the VStr and the anterior insula, but weakened the interaction between the VStr and the prefrontal cortex, leading to increased impulsivity. Cilia et al. (2008) found Parkinson’s patients with pathological gambling showed resting over-activity in brain areas in the mesocorticolimbic network, including the insula. In an fMRI study, relative to Parkinson’s controls, impulse control disorder patients had decreased anterior insular and orbitofrontal cortex activity (van Eimeren et al., 2009; Voon et al., 2010). Finally, in a study of Parkinson’s disease patients with and without hypersexuality, a single dose of L-Dopa abolished the normal insular deactivation seen in response to erotic pictures, only in the hypersexual patients (Politis et al., 2013). Taken together these results may suggest an imbalance between the prefrontal-striatum connectivity and insula-striatum connectivity, favoring the influence of potential gains over that of potential risks (losses) in decision-making.

Risk taking and loss aversion

An influential framework for studying risky decision making is prospect theory, developed by Kahneman and Tversky (1979). A key finding of their work is loss aversion, a tendency for losses to loom larger than potential gains, and for individuals to typically forego risky choices when less valuable safer alternatives exist. For example most people will reject the offer of a coin flip unless the potential gain is considerably larger than the potential loss. Impulsiveness, at least in a gambling context, can be characterized as a reversal of loss aversion, and an over-weighing of potential rewards relative to losses. It remains to be seen whether loss aversion results from asymmetrical weighting of gains and losses along a single value axis (Tom et al., 2007), or from a competitive interaction between separate systems for gains and losses (Kuhnen and Knutson, 2005; De Martino et al., 2010). Possibly, both models are correct: recent fMRI evidence (Canessa et al., 2013) shows bidirectional responses to losses and gains in the VStr and ventromedial prefrontal cortex (positive for gains) and the amygdala and insula (positive for losses). In both cases, there is greater activation to potential losses, correlating with individual loss aversion measured using prospect theory (Kahneman and Tversky, 1979). However, there are also brain regions that respond uniquely to potential losses, namely the right insula and the amygdala, once again reflecting individual variation in loss aversion (Canessa et al., 2013). In sum, a network of regions centered on VStr, insula and amygdala seems to compute gain and loss anticipation in a way that typically results in loss aversion. Interestingly these structures, along with dorsal anterior cingulate, form an intrinsic connectivity network as identified by resting state fMRI. This network is thought to be involved in detecting and processing emotionally salient events (Seeley et al., 2007).

Loss aversion can be explained on an emotional basis, with both potential gains and losses influencing behavior via different emotions (Loewenstein et al., 2001), namely motivation on the gain side and anxiety for losses. Such a model might tie the former to the nucleus accumbens and the latter to the amygdala and insula. In either case, it is conceivable that individuals who are relatively less loss averse may also be at risk for impulsive behaviors such as drug addiction and gambling, due to relative under valuation of losses, although surprisingly this has yet to be formally tested.

There is some evidence implicating the striatum in reversal of normal loss aversion in pathological gamblers. Loss of striatal dopamine neurons in Parkinson’s disease is associated with reduced risk-taking behavior compared to control subjects (Brand et al., 2004; Labudda et al., 2010), while chronic administration of dopamine agonists, especially in high doses, reverses this tendency and promotes risky behavior and impulsivity (Dagher and Robbins, 2009). In the healthy brain, acute administration of D2 dopamine agonists may also cause an increase in risky choices in humans (Riba et al., 2008) and rats (St Onge and Floresco, 2009). Acute D2/D3 receptor stimulation has been found to produce complex changes in the value of losses judged worth chasing (chasing being the continued gambling to recover losses) (Campbell-Meiklejohn et al., 2011). Taken together, this suggests dopamine, acting on the striatum and possibly other mesolimbic structures, may modulate loss aversion. Two studies in Parkinson’s disease patients not affected by impulse control disorders found that a single dose of the dopamine agonist pramipexole reduced loss prediction error coding in the orbitofrontal cortex in one case (van Eimeren et al., 2009) and the orbitofrontal cortex and insula in the other (Voon et al., 2010). In sum, tonic dopamine activity appears to reduce loss prediction signaling, and may therefore reduce loss aversion.

We propose a general framework based on prospect theory, in which the anticipation of potential losses and rewards is computed, possibly in separate brain regions initially, and integrated to compute a decision value (Figure 3). We speculate that gain anticipation might be computed in the ventral medial prefrontal cortex, based on numerous imaging studies implicating this area in computation of value (Kable and Glimcher, 2007; Plassmann et al., 2007; Bartra et al., 2013). As reviewed above, the amygdala and insula may be involved in computing loss anticipation. A possible site for the final computation of value, at least for the purpose of updating choices and action plans, is the striatum, which has fairly direct access to brain regions involved in action planning (van der Meer et al., 2012). The striatum has inherent roles in both response-reward associations (dorsal striatum) (Alexander and Crutcher, 1990) and creating stimulus-reward contingencies (VStr), which afford it the unique opportunity for computation of value (Packard and Knowlton, 2002). Striatal value signals can promote reinforcement processes leading to the updating of future actions, strategies and habits, mediated by the dorsal striatum, while also driving appetitive reward seeking behavior via the VStr. For a review of the role of the striatum in value coding see Knutson et al. (2008); Bartra et al. (2013). The balance between gain and loss evaluation systems may be modulated at least in part by dopamine. We propose a model in which tonic dopamine, acting via the indirect basal ganglia pathway (Figure 2) regulates inhibitory control manifesting as loss aversion. Here lower levels of tonic dopamine would be associated with increased loss aversion. Conversely, phasic dopamine, acting via the direct pathway, would increase the value of gains. This is based on the finding that young healthy subjects given a single dose of the dopamine agonist cabergoline show reduced learning in response to gains (positive feedback), due presumably to a presynaptic effect (in low doses, cabergoline, a D2 agonist, reduces phasic dopamine neuron firing via actions on the high affinity D2 autoreceptor, located pre-synaptically on dopamine neurons) (Frank and O’Reilly, 2006). Conversely, haloperidol, a D2 antagonist, increased learning from gains, probably due to its ability to enhance phasic dopamine firing. With respect to Parkinson’s disease, if a patient has an individual vulnerability to undervalue losses, then dopamine agonist therapy, which tonically stimulates D2 receptors and blocks sensing of the phasic dopamine dips associated with negative rewards, (Frank et al., 2004, 2007), could result in even lower loss aversion. One interpretation is that the intensity of phasic activity sets the gain on the value of potential rewards, while the tonic stimulation of D2 receptors blocks the negative feedback associated with losses.

Figure 3

A model of decision-making based on prospect theory. (A) The utility of potential gains and losses is given by the following equation: u(x) = (x) for potential gains and u(x) = −λ · (−x) for losses (Kahneman and Tversky, 1979). When the loss aversion parameter λ is greater than 1 the function is steeper in the loss domain, implying loss aversion. In this model the utility of gains and losses is computed by different neural networks and combined at some point. We list regions that may be implicated in the calculation. (B) Dopamine may influence the shape of the utility function for gains and losses, by affecting any of the parameters α, β or λ to regulate the degree of loss aversion. Tonic and phasic dopamine may modulate gain and loss calculation via the direct and indirect basal ganglia pathways (Figure 2). The balance of tonic and phasic dopamine signaling could regulate the balance between action selection and inhibition, regulating the current level of loss aversion.

Parkinson’s disease patients show enhanced positive learning when on dopaminergic medications, and improved negative learning while off medication, compared to age-matched controls (Frank et al., 2004). Treatment with dopamine D2 agonists is now accepted as the cause of impulse control disorders in Parkinson’s disease, in which problem gambling is phase locked to medication use. In the model proposed here, D2 stimulation would reduce loss aversion via the indirect corticostriatal pathway. We suggest that under D2 agonist treatment, these patients have a tendency to undervalue losses and be more risk seeking. This is consistent with the observation that Parkinson’s disease patients’ deficits in risky decision making is dominated by impaired ability to use negative feedback (Labudda et al., 2010). The effect on gain, risk, and loss processing of dopamine signaling in other parts of the mesolimbic and mesocortical system, notably the vmPFC, OFC, insula and amygdala, remains to be investigated in greater depth.

Loss tolerance profile may also be affected by norepinephrine signaling. In healthy volunteers, a single dose of the centrally acting beta blocker propranolol reduced the perceived magnitude of losses (Rogers et al., 2004) and normal variations in norepinephrine reuptake transporter in the thalamus, as assessed by PET, correlate with loss aversion (Takahashi et al., 2013). An explanation for this is that norepinephrine increases the arousal response to potential losses, and low norepinephrine signaling may therefore reduce loss aversion. While norepinephrine neurons are also affected in Parkinson’s disease, their role in the motivational and impulsive aspects of the disease have yet to be investigated (Vazey and Aston-Jones, 2012).

Conclusion

The causal association between dopamine D2 receptor agonism and impulse control disorders in Parkinson’s disease has implications for addiction more generally. First, not all individuals develop addictive syndromes following dopamine replacement therapy; those who do appear to have relatively preserved dopamine signaling in the mesolimbic pathway, possibly through a combination of their specific pattern of neurodegeneration, sensitization and pre-morbid vulnerability (as evidenced by the fact that a family history of addiction is a risk factor). It is conceivable that enhanced mesolimbic transmission is also a risk factor in the general population (Buckholtz et al., 2010). Second, it is clear that D2 receptor agonism alone is sufficient for the development of the addictive syndrome. While combined D1/D2 agonists such as L-Dopa may themselves be addictive (Lawrence et al., 2003), D2 agonists are not typically administered compulsively; rather, they have the ability to promote other addictions such as pathological gambling (O’Sullivan et al., 2011). This is supported by animal experiments (Collins and Woods, 2009), computational neuroscience models (Cohen and Frank, 2009), and molecular biology evidence (Shen et al., 2008) suggesting that D1 receptor stimulation is reinforcing while D2 receptor stimulation inhibits the inhibitory indirect pathway. We suggest that D2 agonism, in vulnerable individuals, has the effect of “releasing the brake” on reinforcement systems, thus facilitating the development of impulse control disorders. The time-locked nature of the D2 effect, and the fact that addictive behaviors typically resolve upon discontinuation of the dopamine agonist, is consistent with the theory that tonic dopamine has an invigorating effect on reward seeking behavior (Niv et al., 2007; Dagher and Robbins, 2009).

We note however that other mechanisms besides dopamine-mediated disruption of responses to reinforcing events and stimuli may play a role. For example, Averbeck et al. (2014) have proposed that Parkinson’s disease patients with impulse control disorders are uncertain about using future information to guide behavior, which could lead to impulsivity (a tendency to privilege immediate action). Also, frontal lobe deficits (Djamshidian et al., 2010) could also lead to impulsivity through impaired self-control. These mechanisms need not be mutually exclusive.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.