Exploring the effects of depression and treatment of depression in reinforcement learning.

In this research topic, Herzallah et al. (2013) have contributed original work, exploring the effects of Major Depressive Disorder (MDD) and of treatment with a selective serotonin reuptake inhibitor (SSRI) on responses in a reinforcement-learning task. The task developed by the authors allows for dissociation between positive (reward) and negative (punishment) reinforcement learning and was used to compare, in a cross-sectional design, healthy controls (HC), patients with MDD prior to initiating treatment (MDD), and patients recovered from MDD under treatment with paroxetine (a SSRI; MDD-T). The main finding of their work was that, when compared to HC, patients with untreated MDD were impaired in reward learning, while MDD patients under treatment with paroxetine were impaired in both reward and punishment learning. As a result of these response profiles, the relative learning from reward and punishment was similar between HC and MDD-T (due to blunted responses to both valences in MDD-T) while, in MDD patients, learning from negative feedback was exaggerated when compared to learning from positive reinforcement.

The findings reported in this paper (Herzallah et al., 2013) contribute novel insights to the body of literature exploring the relationship between depression, reinforcement and serotonin (5-HT). The interest on understanding reinforcement processes in the context of depression is sustained on the fact that anhedonia, i.e., the loss of interest or pleasure in all or almost all activities, is a key-symptom of MDD (APA Task Force on DSM-IV, 1994). Preclinical analogs to anhedonia, commonly used to assess depression-like behavior in rodents, are the sucrose intake and preference tests, whereby decreased intake of, or preference for, a sweet sucrose solution (relative to water) are argued to reflect an anhedonic state (Monleon et al., 1995). The relationship between serotonergic processing and depression has been suggested mostly from the anti-depressant effects of drugs modulating serotonergic neurotransmission (Shopsin et al., 1976; Delgado et al., 1990). Recently, several hypotheses have been put forward to suggest that the link between MDD and 5HT might be indirect and mediated by associative learning (Robinson and Sahakian, 2008) and/or disinhibition of negative thoughts (Dayan and Huys, 2008). Elliott et al. (1996), for example, found that during application of a neuropsychological assessment battery, after having solved one problem incorrectly, MDD patients were more likely than controls to fail the subsequent problem. This finding correlated with the severity of the depression and has been interpreted as oversensitivity to negative feedback.

Several paradigms have been used to explore the processing capabilities of depressed patients in contexts of reward and/or punishment. Findings have not been entirely consistent, possibly due to heterogeneity of patient samples and differences in testing protocols, with different authors suggesting that patients with MDD have more significant deficits in reward learning (Henriques and Davidson, 2000; Pizzagalli et al., 2008; McFarland and Klein, 2009; Robinson et al., 2012) punishment learning (Murphy et al., 2003; Santesso et al., 2008) or both (Must et al., 2006). Henriques and Davidson (2000) compared MDD patients to a group of non-depressed control subjects on a verbal memory task under three monetary payoff conditions: neutral, reward, and punishment. While control subjects maximized their earnings by modulating their pattern of response in both reward and punishment conditions, relative to the neutral condition, depressed subjects did not do so during reward. Others (McFarland and Klein, 2009) have reported similar findings, with reactivity in response to anticipated reward being significantly diminished in currently depressed compared with never depressed participants, and marginally diminished in comparison with previously depressed participants. There is also evidence to suggest exaggerated responses to punishment in MDD, rather than diminished responses to reward. In a study based on a probability reversal task (Murphy et al., 2003), when given misleading negative feedback, MDD patients were impaired in the ability to maintain the response set, as shown by their increased tendency to switch responding to the 'incorrect' stimulus following negative reinforcement, relative to controls.

Others have compared neural responses to stimuli with positive and negative valence between depressed and non-depressed volunteers. McCabe et al. (2009) compared unmedicated patients with a history of major depression to age and gender matched HC. Despite no differences in stimulus ratings, recovered patients had decreased neural responses to the pleasant stimulus in the ventral striatum and increased responses in the caudate nucleus to the aversive stimulus. The same authors (McCabe et al., 2010) later found that citalopram (a SSRI antidepressant), but not reboxetine (another antidepresssant that acts as a noradrenaline reuptake inhibitor), reduced activation from rewarding stimuli in the ventral striatum and orbitofrontal cortex. Citalopram also decreased neural responses to the aversive stimuli conditions in areas such as the lateral orbitofrontal cortex, while reboxetine produced a similar, although weaker, effect. Others have found that another antidepressant (duloxetine, a dual serotonin and noradrenaline reuptake inhibitor), rather than diminishing the neural processing of both rewarding and aversive stimuli, can increase ventral striatal activity in humans (Ossewaarde et al., 2011).

The findings now reported by Herzallah et al. (2013) highlight the importance of considering treatment effects when testing the relationship between depression and reinforcement learning. Furthermore, the authors provide behavioral support for the neural findings previously described by McCabe et al. (McCabe et al., 2010): in both cases responses to reward and punishment were reduced by treatment with a SSRI, possibly accounting for the experience of emotional blunting described by some patients during SSRI treatment (Price et al., 2009). However, as is acknowledged by the authors (Herzallah et al., 2013), these findings should be interpreted in the context of their cross-sectional study design. For example, it is unclear if, before starting treatment, disease severity in MDD-T patients was similar to that found in MDD patients, and also if the latter group will respond to treatment similarly to the MDD-T group. Importantly, because MDD-T patients are responders to treatment, the experimental design also does not distinguish entirely between the effects of paroxetine per se and the effects of recovery from depression. Future work, with a longitudinal study design and/or including groups at different phases of treatment or treated with non-SSRI drugs or non-pharmacological alternatives, should address these questions.