Maladaptive reward-learning and impulse control disorders in patients with Parkinson's disease: a clinical overview and pathophysiology update.

Impulse control disorders (ICD) in Parkinson's disease (PD) are a disabling non-motor symptom with frequencies of 13-35% among patients receiving dopamine replacement therapy. ICD in PD is strongly associated with dopaminergic drug use, especially non-ergot dopamine agonists (DA). However, individual susceptibility and disease-related neural changes are also important contributors to the development of ICD. Discrepancies between nigrostriatal and mesolimbic dopaminergic degeneration and non-physiological administration of dopaminergic drugs may induce abnormal 'hyperstimulation' of the mesolimbic system, which alters reward-learning behaviors in PD patients. In addition, DA can make patients more impulsive during decision-making and seek risk-taking behaviors. DA intake is also related to the biased representation of rewards. Ultimately, loss of negative feedback control due to dysfunctional frontostriatal connections is necessary for the establishment of ICD in PD. The subsequent behavioral and neural changes are affected by PD treatment and disease progression; thus, proper treatment guidelines for physicians are needed to prevent the development of ICD. Future studies aimed at producing novel therapeutics to control the risk factors for ICD or treat ICD behaviors in PD are warranted. This review summarizes recent advances from epidemiological and pathophysiological studies on ICD in PD. Management principles and limitations of current therapeutics are briefly discussed.

INTRODUCTION

Parkinson’s disease (PD) is the second most common neurodegenerative disorder in the elderly population. Clinical features of PD are characterized by a progressive motor syndrome of resting tremor, rigidity, bradykinesia, and postural instability; however, it is often accompanied by a variety of non-motor symptoms, including sleep, sensory, psychiatric, cognitive, and autonomic disturbances. Although PD was first described in 1817, levodopa was introduced in late 1960s and since this time, chronic dopamine replacement therapy has become the gold standard medical treatment of PD.

In spite of the remarkable anti-parkinsonian efficacy of levodopa in PD, abnormal psychiatric and behavioral symptoms were observed in PD patients soon after its introduction[1-7]. Typically those symptoms included psychosis, delusion, paranoia, hypomania, mood cycling, anxiety, aggression, impulsive behavior, hypersexuality, agitation, and restlessness[8]. Some patients were engaged in meaningless repetitive tasks which resembles the “punding” described in cocaine abusers[7]. Others exhibited mood swings based on the medication-ON and -OFF states or showed compulsive drug seeking behaviors[9]. Giovannoni et al.[9]. described these behavioral disturbances as a hedonistic homeostatic dysregulation syndrome, which was later to be re-defined as dopamine dysregulation syndrome (DDS)[10]. These kinds of behaviors eventually led to financial and social problems and were disastrous to patients and their families[10]. Impulse control disorders (ICD) are one of these psychiatric and behavioral disturbances in PD. Its relationship to dopaminergic drugs has been recognized after numerous reports on pathological gambling in PD patients taking dopamine agonists beginning in the 1990s[11,12]. ICD-like behaviors in PD patients also included compulsive shopping, hypersexuality, and binge eating that met the DSM-IV criteria of ICD[13]. Systematic surveys have shown that all of these behaviors were highly prevalent in PD patients when compared with the general population[14,15]. The prevalence of ICD in PD patients taking drugs is approximately 13.6% in North America[14] and 10.1% in South Korea[15]. Approximately one quarter of patients with ICD have two or more behaviors[14,15]. The prevalence of behavioral disturbances, including both compulsive drug use (DDS) and punding-like behaviors, is reported to be up to 15.5–35% in studies using the newly validated screening tool ‘QUIP’ for ICD in PD[16-19]. ICD often co-exists with DDS but can be present in isolation.

The characteristic features of ICD in PD are often related to simple actions that do not require complex cognitive thought processing. They are linked to immediate rewards and repetitive in nature with insuppressible internal urges. Thus, ICD-related behavioral disturbances in PD are currently not only considered to be confined to pathological gambling, shopping, hypersexuality, and binge eating but also include a broad spectrum of disorders that are obsessive compulsive in nature and related to impaired impulse control, such as kleptomania, trichotillomania and problematic Internet use[20]. Repetitive and compulsive behaviors, known as ‘punding’, can include habitual but non-goal oriented-behaviors, such as hobbyism, cleaning, repairing, compulsive writing, and categorizing information, artistic drawing, craft-making, singing, playing a musical instrument, playing cards, and fishing[21]. These behaviors often co-exist with ICD in PD patients. Another characteristic behavior is the excessive intake of levodopa beyond the required dose which is necessarily accompanied by drug-seeking behavior typically observed in drug addicts[21]. Daily levodopa intake exceeds 3000 or 4000 mg despite severe dyskinesia and DDS has been observed in nearly all cases[13]. Punding can be frequently observed in patients with excessive levodopa intake behavior[22], suggesting a shared pathophysiological mechanism for these compulsive motoric phenomenon[23]. Because of the rapid increase in knowledge about behavioral disturbances in PD, physicians and researchers can confuse terminologies. In general, ‘ICD’, ‘punding’, and ‘excessive intake of levodopa’ are terminologies focused on the specific type of behavioral disturbances, while ‘DDS’ indicates a clinically significant problematic state that is caused by a ‘levodopa addiction’. Predictive factors for DDS and punding in PD remain largely unknown, and the management of these symptoms remains a challenge. Unlike DDS and punding, there have been recent advances in our understanding of the risk factors and the pathophysiology for ICD; thus, this review focuses on the ICD in PD.

RISK FACTORS OF ICD IN PD

Other than dopaminergic drug therapy, epidemiologic studies have reported that a younger age at onset, left sided dominance of parkinsonian symptoms, and male sex are possibly related to ICD risk (Table 1). However, a recent large-scale case-control study that matched medication dosages failed to confirm the significance of all of these factors[24].

Table 1.

Risk factors for impulse control disorders in patients with Parkinson’s disease

Factors
Drugs	Dopamine agonist, high dose, oral non-ergot drugs
Personality traits	Novelty seeking trait, impulsivity, obsessive-compulsive trait
Psychiatric symptoms	Depression, anxiety, aggression, irritability, alexithymia
Past history	Smoking, alcohol use disorder, addiction or substance use disorder
Family history	Alcohol use disorder, substance use disorder
Genetic predisposition	DRD3, GRIN2B, HTR2A
Dopaminergic system	Low dopamine transporter densities at ventral striatum
Clinical features of PD*	Young age at onset, male gender, predominant parkinsonism on left side

controversial.

PD: Parkinson’s disease, DRD3: dopamine receptor D3 gene, GRIN2B: glutamate N-methyl-D-aspartate receptor type 2B gene, HTR2A: serotonin receptor type 2A gene.

Dopamine agonists

Based on current epidemiological research, dopamine agonists are the strongest risk factor for ICD in PD. The first report on ICD was in patients using pramipexole[25], a non-ergot dopamine agonist with a relatively high affinity to limbic dopamine receptor D3 over D2, which raised the possibility of new class dopamine agonists having the ability to induce ICD. ICD was also reported in patients with restless leg syndrome[26], progressive supranuclear palsy[27], and fibromyalgia[28] treated with dopamine agonists that show a high affinity for D3. Because the mesolimbic system plays a central role in the pathophysiology of ICD and non-ergot dopamine agonists can hyper-stimulate limbic dopamine receptors, these drugs are considered to have a high probability for inducing ICD. However, subsequent studies showed that the class of dopamine agonists does not matter. ICD typically occurs in dopamine agonist users of any kind at a frequency 10-fold higher than non-users[15,29]. The risk of ICD increases with higher dosages[15,30]. In a prospective cohort study, the incidence of ICD increased based on the duration of dopamine agonist exposure[31]. However, there was a huge variability during the follow-up[31], and controversies remained regarding the ‘dosage effect’ of dopamine agonists[23]. A recent study suggested that another non-ergot dopamine agonist, the rotigotine transdermal patch, causes ICD less frequently[32]. Large systematic studies are needed to confirm these findings.

Individual susceptibility to ICD

Not all patients who use dopamine agonists develop ICD, and some patients who do not take dopamine agonist develop ICD[15]. Thus, individual susceptibility likely plays a significant role in the appearance of ICD in PD (Table 1).

Personality traits

Impulsivity and novelty seeking traits are consistently reported as risk factors for ICD[24,33,34]. Depression, anxiety, aggression, irritability, obsessive-compulsive traits, and alexithymia are also potential risk factors[24,35,36]. These personality traits are also closely related to addictive disorders in the general population. One study reported that the degree of impulsivity in drug-naïve PD patients was lower than healthy controls, and the frequency of ICD in drug naïve PD patients was not different from that of healthy controls[37]. Thus, these data suggest that dopamine replacement therapy can make patients more impulsive than their premorbid state.

Individuals with a past history of smoking, alcoholism or drug abuse, and a family history of alcohol or drug abuse are vulnerable to ICD after the initiation of dopaminergic treatment[24,34,38].

Genetic susceptibility

Impulse control disorder is considered to be a behavioral addiction and vulnerability to this disorder involves complex traits. These traits typically show a common susceptibility to both drug addiction and ICD. Many genetic and family studies have been conducted in drug addicts or pathologic gamblers from the non-PD population, and candidate genes code receptors, transporters, and enzymes involved in the dopamine, serotonin and glutamatergic systems in the brain[39]. Unfortunately, there have been only a few studies regarding the genetic susceptibility to ICD in PD. Among the dopamine receptor genes, the most frequently investigated is DRD2. In the general population, addictive disorders are significantly influenced by the presence of a DRD2 Taq1A variant that is linked to low receptor availability[40,41]. In contrast, two studies in the PD population consistently reported that this variant was not associated with ICD[42,43]. The different genetic influences between ‘ICD in PD’ and ‘addiction in the general population’ may be related to a different pathophysiology or false negative results in PD ICD studies because of small sample sizes or population stratification.

Lee et al.[42] reported that ICD is significantly associated with being a carrier of the DRD3 gene S9G variant and glutamate N-methyl-D-aspartate receptor type 2B gene C366G variant regardless of clinical status on dopamine agonist use, duration of treatment, PD onset and current patient age. The D3 receptor is predominantly expressed in the mesolimbic system and is thought to exert inhibitory actions[44,45]. An alternative splice variant in this receptor is associated with high novelty responding[46], and the upregulation of D3 expression is associated with behavioral sensitization to ethanol in animal experiments[47]. Another interesting finding is that the serotonergic system may also have a role in the appearance of ICD. In a PD population treated with low doses of dopaminergic drugs, a serotonin receptor type 2A gene T102C variant was associated with a dose-dependent increased risk of ICD[48]. Further studies are needed to explore the genetic mechanisms behind ICD in PD.

Disease-related neural changes in PD

The mesolimbic dopaminergic system plays a central role in the induction and establishment of addictive behaviors[49]. Thus, the chronic exogenous administration of dopaminergic drugs may induce ICD by aberrant stimulation of this system. A remaining question is whether the same dosages of drugs induce ICD in PD patients more frequently than healthy controls. A study using dynamic dopamine transporter (DAT) positron emission tomography (PET) scanning recently revealed that the mesolimbic to nigrostriatal DAT binding potential ratios were higher in PD when compared with normal controls. The ratios were approximately 3 times higher than those of healthy controls due to the profound degeneration of nigral dopaminergic neurons projecting to striatum in PD patients[50]. Thus, PD patients could be easily affected by exogenous dopaminergic drugs, and the effect could be synergistically increased if those drugs have high affinities to limbic dopamine receptors.

PATHOPHYSIOLOGY OF ICD IN PD

The pathophysiological mechanisms of ICD in PD are not fully understood; however, it may be a phased process, as compared to levodopa-induced dyskinesias which is supposed to undergo priming-induction-establishment processes. The priming process may begin with characteristic pathological features of PD and the chronic administration of dopaminergic drugs. The major feature of this process would be alterations in reward-learning behaviors. Some patients who have particular susceptibility to ICD may go on to induction process and during this process dopamine agonists possibly exert a central role. Lastly, ICD would be established by impairments in inhibitory networks and behavioral monitoring systems related to dysfunctional fronto-striatal connections.

Altered reward reinforcement learning in PD patients

Dopamine in the mesolimbic system has a significant role in motivation and learning behaviors. It acts as a pleasurable neurotransmitter and mediates teaching signals during reward-reinforcement learning processes[51]. Dopamine also represents incentive sailence[52]. In the ventral striatum, dopamine release is discretely coded to the probability and uncertainty of rewards[53]. There is a phasic dopamine surge in response to unexpected rewards, whereas there is phasic dopamine suppression in the absence of expected rewards[53]. In parallel, there is tonic dopamine release during the expectation of rewards with the highest degree to the highest uncertainty[53]. Although these observations are obtained from primate experiments, dopamine release in the human brain is thought to be regulated in a similar way. Thus, modulation of dopaminergic signals can affect reward-seeking behaviors in humans. For example, subjects treated with L-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with dopamine receptor blockers[54]. PD patients usually undergo several challenges to their reward-learning machinery during the course of the disease, including the progressive loss of the dopaminergic system, non-physiological administration of exogenous dopamine and dopamine receptor agonists, and excessive dopamine levels caused by high dose medications. In combination, these conditions alter the physiological regulation of dopamine release that occurs during the reward-reinforcement learning process. As a result, PD patients have different learning and reward seeking behaviors from healthy controls. PD patients showed exactly opposite learning patterns during their medication-ON and -OFF states[55]. PD patients achieved more efficient learning by positive reinforcement during their medication-ON state (carrot), whereas their best performance through negative feedback (stick) was during their medication-OFF state[55]. The learning pattern during the medication-OFF state was also observed in drugnaïve PD patients[56] and even in SNCA duplication carriers (genetic carriers for familial PD)[57]. The sensitivity to rewards and punishment becomes more disrupted in PD patients on chronic dopaminergic treatment. Briefly, they develop increased sensitivity to rewards and insensitivity to punishments, a phenomenon demonstrated by performance differences during modified and original versions of the Iowa Gambling Tasks[58].

Biased representation of rewards on risk taking behaviors and impulsive decision making in PD patients with ICD

It seems that dopamine agonists enhance a deviated learning pattern in PD patients who are beginning to express ICD. Dopamine agonists enhance the rate of gain-specific learning and increase striatal activity to δ of prediction error observed in patients with pathological gambling or problem shopping[59]. Thus, PD patients with ICD can experience a persistent “better than expected” outcome while taking dopamine agonists[59].

Dopamine agonists also enhance risk taking behaviors in PD patients with ICD. While taking dopamine agonists, these patients have a bias towards risky choices independent of the effect of loss aversion[60]. Voon et al.[60] has shown that neural activity in brain areas associated with risk representation, such as the ventral striatum, orbitofrontal cortex and anterior cingulate cortex, are decreased in these patients. In a study using the Balloon Analogue Risk Task[61], resting state regional blood flow at ventral striatum was decreased in PD patients with ICD when compared with those that did not have ICD; in addition, there was no activation in the right ventral striatum during risk taking (with unknown probability of the risk). Pathological gamblers in the general population showed a similar activity pattern of relatively diminished ventral striatal activity during simulated gambling[62].

Impulsive decision making is a typical feature of people with ICD. This feature seems to be closely related to the action of dopamine agonists. Administration of dopamine agonists is significantly associated with greater impulsive choices, faster reaction time, faster decision conflict reaction time and executive dysfunction in PD patients with ICD[63].

Plastic changes in the presynaptic and postsynaptic dopaminergic systems and sensitization in PD patients with ICD

Several functional imaging studies have shown that dopaminergic neural systems undergo plastic changes as ICD develops in PD patients (Table 2). The postsynaptic D2 receptor availability at the ventral striatum is lower in PD patients with ICD than those without ICD[64]. This characteristic resembles the low D2 receptor availability observed in drug addicts[65]. On the other hand, presynaptic DAT binding in the ventral striatum is relatively reduced in PD patients with ICD compared with those without ICD[50,66,67], and reduced DAT binding predicts the future risk for ICD[68]. There are only three studies that explored extrastriatal dopaminergic systems. One study used a [11C]FLB-457 {(S)-N-[(1-ethyl-2-pyrrolidinyl)methyl]-5-bromo-2,3-dimethoxybenzamide} PET[69] to explore postsynaptic fibers, and the other studies used N-(3-[18F]fluoropropyl)-2β-carbomethoxy-3β-(4-iodophenyl) nortropane ([18F]FP-CIT) PET[50] and [18F]fluorodopa PET[70] to explore presynaptic fibers. In PD patients with pathological gambling, the D2-autoreceptor occupancy in the midbrain was reduced and the D2/D3 receptor occupancy in the orbitofrontal and anterior cingulate cortex was low[69]. The ventromedial prefrontal DAT uptake and medial orbitofrontal decarboxylase activity were relatively high in patients with ICD at resting state[50,70]. PD is a progressive neurodegenerative disorder; therefore, these binding differences may be secondary to differences in the degree of degeneration[71]. However, studies on dopamine releasability suggest that these findings may be related to a neural plastic change towards the induction of ICD. Cue-induced dopamine release in the ventral striatum is greater in PD patients with ICD than those without ICD[64,72]. Interestingly, an experiment on PD patients with DDS revealed that ventral striatal dopamine release was more sensitized in these subjects, and the magnitude of dopamine release correlated with “wanting” but not “liking”[73]. Thus, incentive sensitization is the pathophysiological mechanism of DDS and ICD-like behaviors in PD, and mesolimbic dopamine release is important for the motivational aspect of these behavioral disturbances. A similar pattern of incentive sensitization is observed in drug addicts[74]. Thus, DDS in PD may share a common mechanism with addictive disorders in the general population.

Table 2.

Suggested synaptic plastic changes in the mesolimbic and mesocortical dopaminergic systems in PD patients with impulse control disorders

	Ventral striatum	Mesocortex
D2 auto-Rc	Probably low	-
DAT	Low	High
D2/D3 Rc occupancy	High	Low
Synaptic DA	High	Low
DA release	Sensitized	Unknown

PD: Parkinson’s disease, DA: dopamine, Rc: receptor, DAT: dopamine transporter.

Dysfunctional fronto-striatal networks in PD patients with ICD

The orbitofrontal and anterior cingulate cortex play a role in punishment-based decision making, the suppression of previously rewarded behavior (i.e., negative feedback) and the monitoring of functions to avoid negative consequences[75]. An early study with [18F]fluorodeoxyglucose PET showed a deactivation of these areas in PD patients during Iowa Gambling Task[76], suggesting dysfunctional limbic frontal networks are present in PD patients. A later study using [15O]H2O PET clearly demonstrated apomorphine-induced deactivation of these two areas during a gambling task in PD patients with pathological gambling[77]. Disconnection between the striatum and lateral orbitofrontal and anterior cingulate cortex was also reported in PD patients with pathological gambling by a path modeling analysis[78]. These findings suggest a dysfunctional inhibitory fronto-striatal network is another important feature of PD patients with ICD. A dysfunctional prefrontal cortex is a hallmark of addicts in the general population[79] that leads to impaired response inhibition and salience attribution in addiction cycles of intoxication, binging, withdrawal and craving[79].

MANAGEMENT OF ICD IN PD

Currently, there is no treatment guideline established for the management of ICD in PD. However, the principle of proper management is to treat the disorder as a behavioral complication of dopaminergic therapy. Although we may encounter conflicts in individual patients between the parkinsonian motor symptoms and behavioral problems, attempts to discontinue or decrease the dosages of dopamine agonists or switching to the use of levodopa are necessary. Comorbid psychiatric symptoms, such as depression and anxiety, should not be neglected. Proper treatments often require multidisciplinary approaches, including psychiatric consultation.

One study reported that all of their 18 subjects experienced full or partial remission of ICD symptoms after discontinuation or decreasing the dose of dopamine agonists[80]. However, there is a risk of dopamine agonist withdrawal syndrome (DAWS)[81]. DAWS is similar to a drug-withdrawal syndrome and is characterized by anxiety, panic attacks, depression, dysphoria, agitation, insomnia, irritability and drug cravings that are often accompanied by autonomic symptoms, such as orthostatic hypotension, diaphoresis and nausea. DAWS was reported in up to 20% of patients taking high doses dopamine agonists[81]. In addition, patients may develop severe apathy after withdrawal of dopaminergic drugs because of underlying disease-associated mesolimbic denervation[82].

One cross-over trial of amantadine showed a promising effect on reducing the severity of gambling behaviors in PD[83]; however, the results were refuted by other investigators and controversy remains as to the effect of amantadine. Actually the frequency of ICD behavior was significantly higher in amantadine users when compared with non-users in two large cohorts (relative risk for ICD is approximately 1.7)[84,85].

Deep brain stimulation (DBS) is an attractive therapeutic option in severe cases. However, the collective literature on DBS showed that some patients improve but others do not[86-93]. Unexpectedly, some patients develop ICD after DBS, despite dosage reduction or no intake of dopaminergic drugs[94-96]. Although these DBS studies are not specifically designed to investigate the effect of DBS on ICD, it is raised why de novo ICD cases occur after DBS[97]. Frank et al.[98]. assessed decision-making in PD patients using a computer game and found that DBS of the subthalamic nucleus makes patients more impulsive in high-conflict decisions, whereas dopaminergic drugs interfere with the ability to learn from negative experience. The subthalamic nucleus has been suggested to act on response inhibition as a central brake[99], which may be either a ‘proactive inhibition’ or a ‘reactive inhibition’. One study showed that subthalamic nucleus stimulation was associated with alterations in brain areas involved in both mechanisms[100], and the ventral portion of the subthalamic nucleus is specifically responsible for response inhibition[99]. To control motor symptoms, dopaminergic drug therapy and DBS would be a necessary evil for PD patients; thus, close monitoring for the appearance of ICD is required during the course of the disease.

Intrajejunal continuous levodopa infusion has been shown to be effective in 8 patients with severe motor complications, ICD and dopamine dysregulation syndrome[101]. In this small open label trial, all types of ICD behaviors, except for punding, improved after 6 months of treatment[101]. This treatment appears attractive because both motor complications and psychiatric symptoms can be controlled. However, large-scale double-blind studies are needed before any conclusions can be made. Recent trials on cognitive behavioral therapy[102] and on the administration of opioid antagonists[103,104] suggest a new treatment option; however, further studies are needed to confirm the efficacy of these therapies.

CONCLUSIONS

Impulse control disorder is a relatively common behavioral complication of dopamine replacement therapy in PD and is quite disabling to patients and their caregivers. In addition to dopaminergic drugs, individual susceptibility and disease-related neural changes contribute to the appearance of ICD. Based on the pathophysiological mechanisms of ICD in PD, establishment of proper treatment guidelines to prevent ICD and development of therapeutics to control risk factors of ICD and other problematic behaviors are warranted.