Disrupted reward processing in Parkinson's disease and its relationship with dopamine state and neuropsychiatric syndromes: a systematic review and meta-analysis.

BACKGROUND: Neuropsychiatric symptoms are common in Parkinson's disease (PD) and predict poorer outcomes. Reward processing dysfunction is a candidate mechanism for the development of psychiatric symptoms including depression and impulse control disorders (ICDs). We aimed to determine whether reward processing is impaired in PD and its relationship with neuropsychiatric syndromes and dopamine replacement therapy.
METHODS: The Ovid MEDLINE/PubMed, Embase and PsycInfo databases were searched for articles published up to 5 November 2020. Studies reporting reward processing task performance by patients with PD and healthy controls were included. Summary statistics comparing reward processing between groups were converted to standardised mean difference (SMD) scores and meta-analysed using a random effects model.
RESULTS: We identified 55 studies containing 2578 participants (1638 PD and 940 healthy controls). Studies assessing three subcomponent categories of reward processing tasks were included: option valuation (n=12), reinforcement learning (n=37) and reward response vigour (n=6). Across all studies, patients with PD on medication exhibited a small-to-medium impairment versus healthy controls (SMD=0.34; 95% CI 0.14 to 0.53), with greater impairments observed off dopaminergic medication in within-subjects designs (SMD=0.43, 95% CI 0.29 to 0.57). Within-subjects subcomponent analysis revealed impaired processing off medication on option valuation (SMD=0.57, 95% CI 0.39 to 0.75) and reward response vigour (SMD=0.36, 95% CI 0.13 to 0.59) tasks. However, the opposite applied for reinforcement learning, which relative to healthy controls was impaired on-medication (SMD=0.45, 95% CI 0.25 to 0.65) but not off-medication (SMD=0.28, 95% CI -0.03 to 0.59). ICD was the only neuropsychiatric syndrome with sufficient studies (n=13) for meta-analysis, but no significant impairment was identified compared tonon-ICD patients (SMD=-0.02, 95% CI -0.43 to 0.39).
CONCLUSION: Reward processing disruption in PD differs according to subcomponent and dopamine medication state, and warrants further study as a potential treatment target and mechanism underlying associated neuropsychiatric syndromes. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY. Published by BMJ.

Introduction

Parkinson’s disease (PD) is the fastest growing neurological disorder globally,1 with estimated annual societal costs comparable to those of dementia.2 Traditionally conceptualised as a movement disorder, non-motor symptoms, including disruptions to mood, cognition and motivation, are common and have a greater negative impact on health-related quality of life than motor symptoms.3 Neuropsychiatric syndromes are common in PD (see table 1). One-third of patients experience depression,4 up to one-half experience apathy5 and impulse control disorders (ICDs) associated with dopaminergic medication occur in up to one-quarter.6 Currently, there is a lack of understanding of the mechanisms underlying psychiatric symptoms in PD and this represents a barrier to the development of more effective treatments.7

Table 1

Current understanding of the role of reward processing in neuropsychiatric symptoms & syndromes in PD.

Common PD neuropsychiatric symptoms and syndromes	Prevalence in PD	Relationship with reward processing
Apathy—loss or reduction of motivation compared with an individual’s previous state.	40%s1	Apathy and anhedonia are disorders of motivation. Effort-based decision making for reward, the process of how a potential benefit/reward for performing an activity is evaluated with respect to the cost in effort required to attain it, is believed to be a key reward processing mechanism underlying both symptoms.s3
Anhedonia—consistently diminished interest or pleasure in almost all daily activities.	46%s2
Depression—clinical syndrome with core symptoms of persistent low mood and anhedonia.	20%–30% s4, s5	Disrupted reward processing is understood to be a key cognitive mechanism underlying depressive symptoms. Patients with depression have been shown to have impaired option valuation, reinforcement learning and reward bias versus healthy controls.s6
Anxiety—often co-morbid with depression, symptoms include persistent tension, worry and feelings of apprehension.	25%s7	Individuals with anxiety are less sensitive to rewards depending on certainty, preferring less profitable but more predictable options over riskier more rewarding outcomes.s8
Impulse control disorder (ICD)—development of harmful risk-taking and impulsive behaviours. Can include pathological gambling, hypersexuality and sudden episodes of aggression (intermittent explosive disorder).	25%–30%s3	ICD has been proposed to be secondary to dopamine agonists and Parkinson’s pathology sensitising patients to reward.s9 Increased reward sensitivity is suggested to then lead to immediate reward seeking behaviours and impulsivity.
Dopamine dysregulation syndrome—complication of PD treatment characterised by addictive behaviour and excessive use of dopaminergic medication.	3%–4%s10	The reward deficiency theory of addiction posits that patients have a deficit in recruiting/hypoactivation of striatal reward pathways, leading to compensatory addictive behaviours such as drug seeking. Striatal hypoactivation during reward anticipation has been found in individuals with addiction.s11
Psychosis—used to describe range of hallucinations and delusions.	Visual:s12 22%–38%Auditory:s12 20%Delusions:s12 5%	Abnormal reward processing driven by elevated ventral striatal dopamine levels is hypothesised to underlie psychotic symptoms. Hypoactivation of the ventral striatum during reward anticipation has been reported in psychosis. s13

See online supplement for references.

PD, Parkinson’s disease.

Reward processing describes how reinforcement-related perceptions guide goal-directed behaviours.8 Impaired reward processing is a prominent transdiagnostic feature of several mental health disorders such as depression8 and represents a useful framework for understanding symptoms associated with motivation. The National Institute of Mental Health’s Research Domain Criteria identifies reward processing as one of six major domains underpinning human functioning and psychopathology.9 Dopamine has a well-established role in both reward and motivational pathways.10 Evidence from dopamine depletion studies has not supported the hypothesis that dopamine mediates hedonic responses (‘liking’), but has revealed a crucial role in motivated behaviours toward desired goals (‘wanting’).11

PD is caused by dopaminergic cell death and consequently is a model of striatal and dopamine dysfunction.12 The striatum is reciprocally connected with prefrontal areas as well as other parts of the basal ganglia and midbrain, forming frontostriatal circuits involved in the initiation and control of motor, cognitive and emotional function. These pathways also constitute part of the brain’s reward circuit, responsible for modulating reward-related behaviour and learning.13 Psychiatric syndromes in PD (see table 1) are thought to reflect dysfunction of non-motor frontostriatal circuitry; for example, ICDs are believed to develop through aberrant reward processing, due to an interaction between the disrupted reward processing circuitry underlying PD and dopamine agonist treatment.14

Over the past two decades, studies of reward processing in PD have typically used behavioural tasks assessing three subcomponent processes8: (1) option valuation, the process by which individuals evaluate reward-related options when given explicit information about those options (eg, reward, cost and probability); (2) reward response vigour, which reflects the speed or strength with which an individual executes an action to obtain a reward; (3) reinforcement learning, which describes the process by which an individual uses feedback to change their future behaviour. To date, there has been one meta-analysis of Iowa gambling task performance in PD, which reported significantly impaired reward learning.15 However, the degree and pattern of impairments on other reward processing tasks in PD and any relationship with dopaminergic state and psychiatric symptoms remain unclear.

Here we report the first systematic review and meta-analysis of reward processing behaviour in PD and its relationship with dopamine replacement therapy and associated neuropsychiatric syndromes. Our aims were: (1) to clarify the nature and extent of differences across reward processing subcomponents between PD and healthy groups; (2) to test the role of dopamine state (on or off medication) in reward processing in PD; (3) to investigate any differences in reward processing in patients with PD with and without neuropsychiatric syndromes.

Method

Systematic review

The Ovid MEDLINE/PubMed, Embase, and PsycInfo databases were searched for articles published between 1 January 1946 and 5 November 2020 inclusive, with titles or abstracts containing the terms: Parkins* and (reward* or motivat* or incentiv* or effort* or deci*) and (psychiatric or neuropsychiatric or depress* or psychosis or delus* or impuls* or mood or anxiety or apathy or anhedonia or hallucin*). Inclusion criteria were as follows: (1) case-control design; (2) included a group with PD without dementia or deep brain stimulation (DBS) (studies including participants with dementia or DBS withing the PD group were excluded); (3) participants were at least 18 years old; (4) participants performed a reward-processing task; (5) task rewards were explicit, that is, money, points, water or food (we did not include studies that used outcomes that could be considered purely informational or social feedback, eg, happy/sad faces or variants of correct/incorrect, to ensure specificity); (6) studies reported data on a behavioural measure of reward processing that could be converted to a case-control standardised mean difference (SMD) score. If this was not reported, data were requested from the authors. Articles were independently assessed by HC and AJB, using a rating tool based on the Newcastle-Ottawa scale16 for assessing the quality of non-randomised studies (online supplement). Conflicts in quality assessment rating were resolved through in-person discussion.

Meta-analysis

Behavioural measures from each study were categorised as measuring option valuation, reward response vigour or reinforcement learning, and converted to an SMD score and an associated SE (see online supplemental material for equations).17

Within the option valuation and reward response vigour subcategories, a positive SMD represents a greater or faster response to reward by the control than the PD group, respectively. A positive SMD within the reinforcement learning subcategory represents faster use of feedback to maximise reward by the control group than the PD group.

Meta-analysis was conducted if four or more studies were present within a reward processing subcategory for patients with PD compared with healthy controls, PD with and without a psychiatric symptom, or PD on-medication compared with off-medication (within-subjects designs only).

Meta-analysis was performed using the R statistical programming language and the packages metafor and metaviz, using random effects models. Heterogeneity was analysed using the approximate proportion of total variability (I2).

Funnel plot asymmetry was assessed using visual inspection of a contour-enhanced funnel plot and the Egger test.

Results

We initially identified 2122 studies, excluded 1898 of these by title/abstract and retrieved the remaining 224 full papers (figure 1). Data from 55 studies containing 2578 participants (1638 PD, 940 healthy controls) were analysed (see Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram in figure 1); two studies could not be used in the quantitative analysis due to a lack of reported summary statistics. The median number of patients per study was 24 (IQR 16), median participant age was 63.3 years (IQR 7.5) and median duration of PD was 7.0 years (IQR 4.5).

Figure 1

PRISMA flow diagram of study selection and inclusion. DBS, deep brain stimulation; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Meta-analysis across all reward processing subcomponent categories (see online supplemental table 4) identified a small-to-medium reward processing impairment in patients with PD both on-medication (SMD=0.34; 95% CI 0.14 to 0.53) and off-medication (SMD=0.40; 95% CI 0.19 to 0.62), compared with healthy controls (figure 2A, B). Within-subjects comparison of reward processing between on-medication and off-medication states was possible in 14 studies (see online supplemental table 6), revealing relatively impaired reward processing off-medication, with a medium effect size (SMD=0.43, 95% CI 0.29 to 0.57; figure 3A).

Figure 2

Forest plot of reward processing (RP) in (A) PD ON versus healthy controls (HC), (B) PD OFF versus HC. PD, Parkinson’s disease; SMD, standardised mean difference.

Figure 3

Forest plot of reward processing (RP) in (A) PD ON versus OFF dopamine state, (B) PD with and without impulse control disorder (ICD). PD, Parkinson’s disease; SMD, standardised mean difference.

ICD was the most studied and only neuropsychiatric syndrome with sufficient studies (n=13) for meta-analysis (see online supplemental table 5). No significant impairment (see figure 3B) was identified in reward processing in patients with PD with ICD compared to patients with non-ICD (SMD=−0.02, 95% CI −0.43 to 0.39).

Overall interstudy heterogeneity was substantial (I2=57.48%), and the median power of included studies and R-index was low (online supplemental figure 1), median power=36%; R index=28%). Analysis of funnel plot asymmetry using Egger’s regression line did not meet statistical significance (p=0.32) and was likely a consequence of high heterogeneity and small sample size of included studies (see online supplemental figure 1).

Quality assessment and risk of bias analysis using a modified Newcastle-Ottawa scale (see online supplemental table 7) found the majority of included studies used a validated assessment tool for diagnosis of PD (65.5%), and accounted for PD severity (94.5%) and medication status (90.9%). However, almost half of included studies gave no description of how healthy controls were selected (42.2%) or clearly defined controls as having no past psychopathology (42.2%).

Option valuation

We identified 12 studies containing 347 patients with PD and 278 healthy participants that used option valuation tasks (online supplemental table 1). The mean age of participants was 62.9 (±4.6) years, and mean duration of illness was 7.5 (±2.8) years. Effort-based decision-making tasks (three studies) and the game of dice task (three studies) were most commonly used. Four studies reported psychiatric medication use in participants, three of which included participants taking antidepressant medications.

Meta-analysis of studies comparing option valuation in patients with PD compared with healthy controls showed lower reward weighting in PD, which was moderated by dopamine medication (figure 4A, B). Patients on-medication did not differ significantly from healthy controls (SMD=0.22, 95% CI −0.04 to 0.49), but off-medication there was a medium-to-large impairment (SMD=0.60, 95% CI 0.30 to 0.89). Within-subjects comparison confirmed lower reward weighting off-medication, with a medium-to-large effect (SMD=0.57, 95% CI 0.39 to 0.75; figure 4C).

Figure 4

Forest plot of option valuation in: (A) PD ON versus healthy controls, (B) PD OFF versus healthy controls, (C) PD ON versus OFF dopamine state. HC, healthy controls; PD, Parkinson’s disease; SMD, standardised mean difference.

Four studies compared option valuation in patients with PD with and without neuropsychiatric syndromes. Three of these studies18–20 compared option valuation in patients with PD with and without ICD, with mixed findings. One study19 using an economic choice task reported lower reward weighting in ICD, while the other two studies18 20 using gambling tasks found no difference18 and increased reward weighting,20 respectively.

One study21 investigating the effect of apathy on option valuation reported lower acceptance of offers of reward obtained through physical exertion. This pattern of impairment in apathy was found to be dissociable from the effects of dopamine. Apathy was characterised by rejection of predominantly low reward offers, while dopamine state increased choices of high effort, high reward offers.

In summary, option valuation impairment in PD is dopamine dependent, with lower reward weighting off dopaminergic medication. Too few studies have investigated option valuation in patients with PD with neuropsychiatric syndromes to draw meaningful conclusions.

Reinforcement learning

We identified 37 studies containing 1059 patients with PD and 593 healthy controls that used reinforcement learning tasks (online supplemental table 2). The majority of studies (20/37) used the Iowa gambling task. Ten studies reported psychiatric medication use, of which three included participants taking antidepressant medication.

Reinforcement learning was slowed in patients with PD on-medication versus healthy controls (figure 5A, B) with a medium effect size (SMD=0.45, 95% CI 0.25 to 0.65). Interestingly, there was no significant group difference off-medication (SMD=0.28, 95% CI −0.03 to 0.59). Comparison of reinforcement learning comparing on-medication and off-medication within-subjects (figure 5Cc) was possible in four studies, which did not detect a significant effect (SMD=0.27, 95% CI −0.08 to 0.62); however, we note that this analysis is likely underpowered due to the small number of included studies.

Figure 5

Forest plot of reinforcement learning (RL) in: (A) PD ON versus healthy controls, (B) PD OFF versus healthy controls, (C) PD ON versus OFF dopamine state. HC, healthy controls; PD, Parkinson’s disease; SMD, standardised mean difference.

Sixteen studies investigated reinforcement learning in patients with PD with and without neuropsychiatric symptoms (online supplemental table 2), with the majority (11/16) examining ICD. Meta-analysis of nine studies (online supplemental figure 2) found no significant difference between patients with PD with ICD and non-ICD PD patients (SMD=0.32, 95% CI −0.09 to 0.73).

Two studies22 23 examined reinforcement learning in patients with PD with major depressive disorder. Both22 23 reported impaired reinforcement learning in depressed patients with PD compared with non-depressed patients with PD. One23 also compared reinforcement learning in depressed patients with PD with depressed participants without PD. A similar pattern of impairment in learning from positive feedback was identified in the two groups, suggesting that reinforcement learning impairment may not be specific to depression in PD.9

Two studies24 25 examined the role of apathy in reward learning. Both used the Iowa gambling task but reported conflicting findings: one found significant impairment25 but the other reported better reinforcement learning in patients with PD with apathy,24 compared with those without.

In summary, and in stark contrast to studies of option valuation, reinforcement learning is particularly impaired in PD in the on-medication state. There was no significant impairment in reinforcement learning in patients with PD with ICD compared with those without ICD. Too few studies have investigated reinforcement learning in patients with PD with other neuropsychiatric syndromes to draw meaningful conclusions.

Reward response vigour

We identified seven studies containing 232 patients with PD and 69 healthy controls that investigated reward response vigour in PD (online supplemental table 3). Insufficient studies were identified to allow meta-analysis of reward response vigour in PD compared with healthy controls. Of the three studies26–28 that reported reward response vigour in PD and healthy controls, results were mixed, with studies reporting lower,26 greater27 and no difference28 in patients with PD compared with healthy volunteers.

Meta-analysis of the effect of dopamine state on reward response vigour in four studies (figure 6) identified a small-to-medium increase in reward response vigour on-medication (SMD=0.36, 95% CI 0.13 to 0.59).

Figure 6

Forest plot of reward response vigour (RRV) in PD ON versus OFF dopamine state. PD, Parkinson’s disease; SMD, standardised mean difference.

Six studies investigated reward response vigour in patients with PD with and without neuropsychiatric syndromes (online supplemental table 3). Two studies27 29 examined apathy, one using a rewarded saccadic eye movement task,27 the other a rewarded spatial search task29; both reported no significant group differences. Similarly, no significant difference in reward response vigour was found in two studies comparing patients with ICD and patients with non-ICD,30 31 and two investigating depression in PD.28

In summary, relatively few studies have investigated reward response vigour in PD, and findings are mixed. Reward response vigour in PD was reduced in the off-medication compared with the on-medication state. Too few studies have investigated reward response vigour in patients with PD with neuropsychiatric syndromes to draw meaningful conclusions.

Discussion

This is the first systematic review and meta-analysis of reward processing in PD, associated neuropsychiatric syndromes and the influence of dopaminergic medication. Across all 55 studies, including different subcomponents of reward processing, we found patients with PD to have small-to-medium reward processing impairments relative to healthy participant groups. The degree of impairment in reward processing is similar to that reported in major depressive disorder, a condition where dysfunctional reward processing is a leading aetiological candidate mechanism for ‘interest-activity’ symptoms, such as anhedonia.8 We also identified potentially important differences between reward processing subcomponent categories and the effect of dopamine state.

The option valuation subcategory exhibited the largest impairment in PD which was dopamine dependent, with markedly reduced reward weighting in patients with PD off dopaminergic medication. This finding is supported by animal11 and human experimental studies32 which show impaired valuation following dopamine depletion. Dopamine antagonists such as antipsychotic drugs also reduce preference for high-effort/high-reward options,11 suggesting that dopamine transmission is crucial in cost-benefit decision making. Dopaminergic pathways in the brain reward circuit including the anterior cingulate cortex and basal ganglia are believed to be central in choosing and executing effortful action.5 Option valuation is a component of effort-based decision making and represents a framework for understanding apathy and anhedonia, both common motivational disorders in PD and depression.5 However, no study to date has investigated option valuation in depression in PD, and the only study21 to examine apathy found dissociable effects of dopamine and apathy on decision making, indicating impairment may not only be secondary to dopamine depletion.

In direct contrast to the pattern identified in the option valuation subcategory, reinforcement learning was moderately impaired in PD when patients were on dopamine medication, with no significant difference detected when off medication. This is surprising given decades of evidence that dopaminergic pathways from the midbrain are crucial for reward learning.33 However, recent studies applying cell-type specific monitoring and manipulation of distinct neuronal populations in the striatum have suggested that heterogenous signals in dopaminergic neurons support specific types of learning.34 For example, differentially regulated mechanisms of dopamine release in the basal ganglia underlie distinct functions.35 Reward learning is believed to be facilitated by dopamine cell spiking encoding reward prediction errors, whereas gradual increase in dopamine release mirrors reward expectation.35 Reinforcement learning is therefore believed to be dependent on phasic rather than tonic dopamine signalling. Wave-like spatiotemporal dopamine dynamics in the dorsal striatum have also been implicated in encoding reward prediction errors to facilitate learning.36 It remains unclear what effect exogenous dopamine in PD has on the dynamics of striatal dopamine signalling. Studies of associative learning in healthy subjects have found that dopamine agonists can impair learning by inhibiting phasic dopamine signalling.37 Therefore, one possible interpretation is that dopamine medication may remediate control of reward expectation and motivation within the striatum, but impair the broadcast burst signals required to promote learning.35 However, this requires testing in future studies.

Distinct types of reinforcement learning model used during task performance may also play a crucial role.38 ‘Model-free’ learning describes learning through direct experience rather than through constructing an internal model of the environment in order to develop a complex map of cues and actions which lead to reward.38 Most studies included in our review used model free reinforcement learning tasks. Evidence suggests that these two types of reinforcement learning processes are mechanistically distinct, and differentially dependent on dopamine reward prediction errors.38

The reward response vigour subcategory showed a significant small-to-moderate impairment in the off-medication compared with the on-medication state in patients with PD. However, relatively few studies were identified and reaction times may be vulnerable to attentional confounds. Though several studies reported reaction times during tasks, reward-related speeding (ie, the difference between rewarded and non-reward conditions) was infrequently measured, without which slower reaction times would likely only reflect bradykinesia associated with PD.

Despite PD being a model for dopamine dysfunction, current treatments of common neuropsychiatric syndromes in PD such as depression do not differ from depression in patients with other long-term conditions39 and have limited efficacy.40 Symptoms of anxiety and depression in patients with PD with motor fluctuations can be more common and severe in the off-dopamine state,41 suggesting depression in PD may be related to dopaminergic deficit and have a specific aetiology. Our findings suggest PD is characterised by a specific pattern of impairment in reward processing which is dopamine dependent and potentially could be a causal mechanism underlying neuropsychiatric symptoms such as depression. Although ICDwas not significantly associated with reward processing impairment statistical power was limited, and few studies have investigated reward processing in other PD-associated neuropsychiatric syndromes. Further understanding of how impairment in reward processing is associated with specific neuropsychiatric manifestations of PD is needed to understand the underlying mechanisms of these disabling syndromes and develop more targeted and effective treatments.

Limitations

We categorised reward processing into three subcomponent categories, however there are several ways to measure function in each category which grouped diverse processes. For example, the option valuation subcategory included studies measuring risk taking and decisions to exert effort, resulting in meta-analysis of heterogeneous measures. A minority of studies reported psychiatric medication use in participants. Evidence suggests antidepressant medication may partly exert its effect via modulating reward processing42 which could have confounded results. Though we measured and compared the effect of dopamine medication state on task performance, the medication regime and proportion of patients on dopamine agonist treatment as opposed to levodopa was reported in less than half of included studies (22/55). Different PD medications are disproportionately associated with dopamine-related psychiatric conditions such as ICD,6 and distinct regimes could potentially impact reward processing variably. The majority of studies investigating reward processing in PD-associated neuropsychiatric syndromes used patients with PD without the syndrome as a control group. Only one study23 investigating depression in PD used a control group of patients with depression without PD. In order to establish whether patterns of reward processing impairments are specific to PD-associated neuropsychiatric syndromes and not a common feature of psychiatric symptoms more generally, further studies of this type are needed. Finally, our systematic review and meta-analysis examined the findings of case-control studies which are unable to inform us of the causal relationship between reward processing impairment, PD and its associated neuropsychiatric syndromes. Longitudinal studies are needed to answer these questions and understand how reward processing changes develop as PD advances. Our analyses of the impact of dopamine medication were derived from studies conducted using within-subjects experimental comparisons, and therefore we can be more confident of a causal role. However, the effects of being off-medication in a patient who usually takes dopamine-boosting drugs, including heightened anxiety and physical discomfort, could plausibly affect task performance. A minority of studies (22/55) measured motor symptom severity in both on and off states, and only four studies measured differences in anxiety symptoms in both states.

Conclusions

PD is associated with a small-to-medium level of reward processing impairment overall, with variable degrees of impairment across subcomponent reward processing categories. Reward processing is dependent on dopamine state with greater impairment in option valuation and reward response vigour when patients are off dopaminergic medication, but surprisingly faster reinforcement learning. Other than reinforcement learning in ICD, few studies have investigated the relationship between reward processing and PD associated neuropsychiatric syndromes. Further research, including longitudinal studies are needed to conclude whether specific patterns of impairment in reward processing have a causal relationship with neuropsychiatric syndromes in PD.