Intrinsically regulated learning is modulated by synaptic dopamine signaling.

We recently provided evidence that an intrinsic reward-related signal-triggered by successful learning in absence of any external feedback-modulated the entrance of new information into long-term memory via the activation of the dopaminergic midbrain, hippocampus, and ventral striatum (the SN/VTA-Hippocampal loop; Ripollés et al., 2016). Here, we used a double-blind, within-subject randomized pharmacological intervention to test whether this learning process is indeed dopamine-dependent. A group of healthy individuals completed three behavioral sessions of a language-learning task after the intake of different pharmacological treatments: a dopaminergic precursor, a dopamine receptor antagonist or a placebo. Results show that the pharmacological intervention modulated behavioral measures of both learning and pleasantness, inducing memory benefits after 24 hr only for those participants with a high sensitivity to reward. These results provide causal evidence for a dopamine-dependent mechanism instrumental in intrinsically regulated learning and further suggest that subject-specific reward sensitivity drastically alters learning success.

RCT Entities:   Population Interventions Outcomes

Introduction

Growing evidence both from animal and human studies support the notion that midbrain dopaminergic neurons of the substantia nigra/ventral tegmental area complex (SN/VTA), along with the ventral striatum (VS) and the hippocampus (HP), form a functional loop (the SN/VTA-HP loop) in the service of learning and memory (Lisman and Grace, 2005; Goto and Grace, 2005; Lisman et al., 2011; Shohamy and Adcock, 2010; Kamiński et al., 2018). In the downward arm of the circuit, signals are sent from the HP to the SN/VTA through the VS, which is thought to integrate affective, motivational, and goal-directed information into the loop (Lisman and Grace, 2005; Goto and Grace, 2005). In the upward arm of the loop, dopamine is released from the SN/VTA back into the HP, which in turn enhances memory formation and learning through long-term potentiation (LTP) processes (Lisman et al., 2011; Lisman and Grace, 2005; Shohamy and Adcock, 2010). Within this loop, dopamine plays a critical role, as its release promotes the creation of stable memories by allowing LTP to persist over time (Bethus et al., 2010; Frey et al., 1990; Hansen and Manahan-Vaughan, 2014; Huang and Kandel, 1995; McNamara et al., 2014; Rossato et al., 2009).

In this vein, fMRI research in humans has consistently shown that both explicit (Adcock et al., 2006; Wittmann et al., 2005; Wolosin et al., 2012; Callan and Schweighofer, 2008) and implicit reward (Ripollés et al., 2016), and even intrinsic motivational states (i.e., curiosity; Gruber et al., 2014), can promote the storage of new information into long-term memory through the activation of the SN/VTA-HP loop (see Figure 8 in, Ripollés et al., 2016). However, although fMRI activity within the SN/VTA is usually associated with the release of dopamine (Düzel et al., 2009 ; Ferenczi et al., 2016; Knutson and Gibbs, 2007; Salimpoor et al., 2011; Schott et al., 2008), neuroimaging studies can only provide indirect evidence of the actual involvement of the dopaminergic mesolimbic system. In order to prove that a dopamine-dependent mechanism plays a critical role in learning and memory processes, one avenue to pursue is to directly manipulate dopaminergic neurotransmission in the human brain through pharmacological interventions. In this vein, several studies have shown that administration of dexamphetamine and methylphenidate (which increase dopamine concentrations in the synapsis by blocking its reuptake; Breitenstein et al., 2004; Whiting et al., 2007; Whiting et al., 2008; Linssen et al., 2014) and specially, levodopa (the immediate precursor of dopamine) can enhance memory and learning in both healthy (Shellshear et al., 2015; Bunzeck et al., 2014; Chowdhury et al., 2012; Knecht et al., 2004) and clinical populations (Berthier et al., 2011).

We recently provided behavioral, functional and physiological evidence by means of fMRI and skin conductance response, showing that an intrinsic reward-related signal—triggered by successful learning in absence of any external feedback or explicit reward—modulated the entrance of new information into long-term memory via the activation of the SN/VTA-HP loop (Ripollés et al., 2016). Here, we used a double-blind, within-subject randomized pharmacological intervention to directly assess the hypothesis that synaptic dopamine availability plays a causal role in this learning process. A group of 29 individuals were asked to perform a language-learning task (that mimics our capacity to learn the meaning of new-words presented in verbal contexts; Ripollés et al., 2014; Ripollés et al., 2016; Ripollés et al., 2017; Mestres-MisseMissé et al., 2007) after the intake of three different pharmacological treatments: a dopaminergic precursor (levodopa, 100 mg +carbidopa, 25 mg), a dopamine antagonist (risperidone, 2 mg), or a placebo (lactose). Levodopa is rapidly taken up by dopaminergic neurons, transformed into dopamine and stored in vesicles from which it will be released into the synaptic cleft each time the neuron fires. Thus, levodopa leads to a general increase in dopamine available for release in brain areas innervated by dopaminergic afferents. On the other hand, risperidone—a dopamine antagonist—interferes with dopaminergic neurotransmission by binding with a group of receptors known as D2 or D2-like (Burstein et al., 2005). Therefore, in the presence of risperidone, the transmission of dopamine-mediated signals to post-synaptic neurons will be reduced due to the blockade of the D2 receptor family.

We aimed at assessing the influence of dopamine signaling on learning and reward using the pharmacological approach described above. Each of the two experimental sessions involving active drugs were intended to shift dopaminergic neurotransmission away from each individual’s physiological status, as measured in a placebo session, and in opposite directions: levodopa to enhance the dopamine availability for release into the synapse, and risperidone to reduce synaptic transmission of the dopamine-associated signal by hindering dopamine-receptor interactions (for the use of levodopa or risperidone during cognitive tasks, see e.g. Rabella et al., 2016; Wittmann and D'Esposito, 2015; de Vries et al., 2010; Knecht et al., 2004) . Accordingly, we predicted that behavioral measures of both learning and reward should increase and decrease under levodopa and risperidone, thus modulating—with opposite effects—the memory benefits for the learned words after a consolidation period (24 hr).

Results

Twenty-nine healthy participants completed a behavioral version of our word-learning task (see Materials and methods), in which the meaning of a new-word could be learned from the context provided by two sentences built with an increasing degree of contextual constraint (Mestres-Missé et al., 2010). Only half of the pairs of sentences disambiguated multiple meanings, allowing the encoding of a congruent meaning of the new-word during its second presentation (M+ condition). For the other pairs, the new-word was not associated with a congruent meaning across the sentences and could not be learned (M- condition). This condition, as in our previous study (Ripollés et al., 2016), was included to control for possible confounds related to novelty, attention and task difficulty (Guitart-Masip et al., 2010; Bunzeck and Düzel, 2006; Boehler et al., 2011). At the end of each learning trial (i.e. after the second sentence for a particular new-word appeared), participants first provided a confidence rating (a subjective evaluation of their performance) and then rated their emotions with respect to arousal and pleasantness. After approximately 24 hr (no drug intake occurred during the second day of testing), participants completed a recognition test to assess their learning (chance level was 25%; see Materials and methods). Three participants were excluded from the analyses (see Materials and methods) and thus the final sample was reduced to 26 individuals (17 women, mean age = 22.27 ± 3.69).

We first assessed whether our participants’ performance under the placebo condition replicated our previous results. Participants ascribed correct meaning to 60 ± 10% of new-words from the M+ condition during the encoding phase. In 61 ± 15% of the M- trials, participants correctly indicated an absence of coherent meaning. After 24 hr, participants still recognized the correct meaning of 65 ± 17% of learned new-words during the encoding phase [significantly above 25% chance level, t(25)=12.28, p<0.001, d = 2.33; Bayes Factor-BF10- equal to 1.9e + 9] and correctly indicated that 41 ± 22% of M- new-words identified during the encoding phase had no meaning ascribed [significantly above 25% chance level, t(25)=3.70, p<0.001, d = 0.70; BF10 = 35.38].

In order to compare this performance with our previous results (24 participants from Exp. three in Ripollés et al., 2016), we submitted the learning scores to a mixed repeated measures ANOVA with Condition (M+,M-) as a within-subjects variable and Group (Pharmacological Group, Exp. 3 in Ripollés et al., 2016) as a between subjects variable. No significant effect of Group [Learning Day 1: F(1,48)=0.246, p=0.622, partial η2 = 0.005, BFInclusion = 0.297; Recognition Day 2: F(1,48)=3.56, p=0.065, partial η2 = 0.069, BFInclusion = 1.05] or Group × Condition interaction [Learning Day 1: F(1,48)=0.749, p=0.391, partial η2 = 0.015, BFInclusion = 0.381; Recognition Day 2: F(1,48)=0.222, p=0.639, partial η2 = 0.005, BFInclusion = 0.313] was found for the learning scores of Day 1 or the recognition rate after 24 hr. This shows that the new group of participants, during the placebo session, learned and remembered words from the M+ condition and correctly identified M- words (i.e. no meaning ascribed) at the same rate as in our previous experiment.

We then focused our analyses on learned (on Day 1) and still remembered (on Day 2) M+ new words. In our previous work (Ripollés et al., 2016), this was the condition associated to the largest fMRI activity within the SN/VTA-HP loop, the largest physiological response and the highest subjective pleasantness ratings, even when compared with learned words that were forgotten after 24 hr (as a control, we used M- new-words correctly identified during the encoding phase and after 24 hr). Accordingly, in the present study subjective pleasantness and confidence ratings on Day 1 were higher for remembered than for forgotten M+ new words in the 24 hr recognition test [pleasantness, t(25)=2.75, p=0.011, d = 0.42, BF10 = 4.39; confidence, t(25)=4.56, p<0.001, d = 0.68, BF10 = 232.82], while no difference in arousal ratings was encountered [t(25)=0.20, p=0.835, d = 0.025, BF10 = 0.21]. In replicating our previous results (Ripollés et al., 2016), these findings confirm that intrinsic reward (i.e. derived from an internal monitoring of learning success) had a modulatory effect on long-term memory. Regarding the M- control condition, as expected, there was no difference in subjective pleasantness [t(25)=1.40, p=0.172, d = 0.26, BF10 = 0.49], arousal [t(25)=1.28, p=0.212, d = 0.20, BF10 = 0.43] or confidence ratings [t(25)=1.18, p=0.247, d = 0.18, BF10 = 0.38] for M- new-words which were correctly identified during the encoding phase and still correctly rejected in the 24 hr test and those which were not. When submitting these ratings to a mixed repeated measures ANOVA with Condition (M+,M-) and Group (Pharmacological, Previous Data) as factors (in our previous analyses we excluded four participants from the rating analyses, see Ripollés et al., 2016; thus in this ANOVA we compare 20 participants from the previous dataset against 26 for the placebo session), no significant effect of Group [Pleasantness: F(1,44)=0.143, p=0.707, partial η2 = 0.003, BFInclusion = 0.349; Arousal: F(1,44)=1.66, p=0.204, partial η2 = 0.036, BFInclusion = 0.80; Confidence: F(1,44)=3.49, p=0.068, partial η2 = 0.073, BFInclusion = 1.17] or Group × Condition interaction [Pleasantness: F(1,44)=0.239, p=0.627, partial η2 = 0.005, BFInclusion = 0.321; Arousal: F(1,44)=0.216, p=0.645, partial η2 = 0.005, BFInclusion = 0.304; Confidence: F(1,44)=0.028, p=0.868, partial η2 = 0.001, BFInclusion = 0.280] was found. This shows that participants’ ratings were also in line with those of our previous experiment (Ripollés et al., 2016).

Hence, we calculated the drug effect on the behavioral data. Specifically, for the levodopa and risperidone interventions and for each subject, we calculated the percentages of change in learning scores and behavioral ratings with respect to the placebo session (see Materials and methods). Notably, for the M+ condition, our findings show a pharmacological modulation of learning performance and behavioral reward ratings. The percentage of learned words during the encoding phase was higher under levodopa than under risperidone [as compared to placebo; t(25)=2.72, p=0.012, d = 0.56, BF+0=8.26]. Importantly, this effect was still present at 24 hr for the total number of remembered new-words [t(25)=2.10, p=0.046, d = 0.45, BF+0=2.62; see Figure 1A]. In addition, while no significant changes were found for the arousal ratings [t(25)=0.31, p=0.757, d = 0.049, BF+0=0.26], the drug effect approached significance for confidence ratings [t(25)=2.05, p=0.051, d = 0.51, BF+0=2.40] and was significant for pleasantness ratings [t(25)=2.70, p=0.012, d = 0.64, BF+0=7.93], where scores for remembered words at 24 hr where higher under levodopa than under risperidone (as compared to placebo; see Figure 1B). There was not, however, a significant effect of drug on the recognition rate [i.e. the percentage of remembered words in the recognition test of Day 2 compared to those that were learned on Day 1; t(25)=-0.013, p=0.989, d = 0.003, BF+0=0.20]. This suggests that the pharmacological intervention was able to modulate measures of reward, memory and online learning selectively during the main M+ condition (see the Individual Variability of the Drug Effect section of Appendix I, for a more in depth description of the individual differences found for the drug effect for each measure). Additional analyses using the values for the three sessions separately (instead of the percentage of change from placebo) further confirmed this pattern of results (see the Supplemental Behavioral Analyses section of Appendix 1 and Figure 1—figure supplementary 1A–C).

Figure 1.

Effects of the pharmacological intervention (mean ± SEM) in (a) Learning and memory scores and (b) subjective ratings.

Note that subjective ratings were only measured during the learning phase of Day 1. Effects are calculated as % of change with respect to the placebo session. *p<0.05, **p<0.01.

Figure 1—figure supplement 1.

Mean plus standard error of the mean M+ scores for risperidone (black), placebo (white) and levodopa (grey) interventions separately for (A) learning and memory measures, (B) learning and memory measures separated for high (H+; black line) and low (H-; grey line) hedonic participants and (C) subjective ratings.

As expected, for the control M- condition no significant differences between the risperidone and levodopa interventions as compared to placebo were found for the online learning scores on Day 1 [t(25)=1.53, p=0.137, d = 0.28, BF+0=1.08], the total number of correctly rejected M- words at 24 hr [t(25)=0.62, p=0.538, d = 0.15, BF+0=0.35], the recognition rate [percentage of words correctly rejected on Day 2, in respect to those correctly rejected on Day1; t(25)=-0.04, p=0.968, d = 0.011, BF+0=0.20; see Figure 1A], or the subjective ratings of arousal [t(23)=1.72, p=0.097, d = 0.45, BF+0=1.46] and confidence [t(23)=0.36, p=0.720, d = 0.10, BF+0=0.28; see Figure 1B; two participants were excluded from the rating analyses after not correctly rejecting any M- word at 24 hr from those correctly rejected during encoding in the levodopa intervention]. For the pleasantness ratings, however, the difference was close to significance [t(23)=2.02, p=0.055, d = 0.40, BF+0=2.32]. However, it is important to note that the pleasantness ratings for M- trials remembered at 24 hr were not different from 0 at any session [Risperidone mean rating = −0.23, t(23)=-1.03, p=0.309, d = 0.20, BF10 = 0.34; Placebo mean rating = 0.24, t(23)=1.01, p=0.319, d = 0.20, BF10 = 0.34; Levodopa mean rating = 0.30, t(23)=1.20, p=0.240, d = 0.23, BF10 = 0.40], implying that participants did not find this learning condition particularly rewarding even if the pharmacological intervention slightly modified their subjective ratings.

Given that our learning task modulates activity within the reward network and is associated with increased subjective reports of pleasure (Ripollés et al., 2014; Ripollés et al., 2016), we further tested whether individual differences in sensitivity to reward interacted with the drug intervention to modulate memory benefits (Ferreri and Rodriguez-Fornells, 2017; de Vries et al., 2010; Apitz and Bunzeck, 2013). Twenty-four out of the 26 participants completed the Physical Anhedonia Scale (PAS; Chapman et al., 1976); mean score = 11.62 ± 5.47). The PAS is a well-validated scale that evaluates difficulty in feeling physical and aesthetic pleasure in response to typical pleasurable stimuli (Padrão et al., 2013; Mas-Herrero et al., 2014). We correlated (using Spearman´s rho) participants’ individual scores with the drug effect for each learning condition (the drug effect was calculated as the subtraction of the percentage of change from placebo of the levodopa intervention minus the percentage of change from placebo of the risperidone intervention, see Materials and methods). As a control and in order to take into account previous results (Chowdhury et al., 2012),we also assessed the relationship of the learning scores with the weight-dependent measure of drug dose (calculated in mg of levodopa/risperidone administered per kilogram, mean value = 1.66 ± 0.23). As expected, no significant correlations were found between the M- learning scores and the PAS [Learning Day 1 rs = −0.19, p=0.372; number of correctly rejected words during Day 2, rs = −0.34, p=0.097; recognition rate, rs = −0.19, p=0.372]. In addition, no significant linear correlation or inverted U-shape relationship (Chowdhury et al., 2012) was found for any learning score (M+ or M-) and the weight dependent drug dosage (all ps > 0.13). However, for the drug effect for M+ trials, the number of learned words during encoding (rs = −0.45, p=0.025), the total number of remembered words during Day 2 (rs = −0.67, p<0.001) and, strikingly, the recognition rate (rs = −0.49, p=0.017), showed a significant correlation with the PAS (all correlations were FDR-corrected at a p<0.05 threshold, see Figure 2A). One participant was excluded from the correlations with the learning scores on Day 2 after being identified as a bivariate outlier (note that if included, the correlations become more significant: number of remembered words during Day 2, rs = −0.71, p<0.001; recognition rate, rs = −0.55, p=0.005). Importantly, this participant obtained the highest (more anhedonic) score on the PAS (score of 24, more than two standard deviations above the mean score of 11.62 of the group). Additional correlational analyses with the results of each intervention separately (risperidone, placebo, levodopa) instead of the drug effect only, further confirm a relationship between learning and memory scores and the PAS, with Spearman’s rhos for this relationship being consistently lower than placebo for risperidone and higher than placebo for levodopa (see the Supplemental Correlational Analyses section of Appendix 1 and Figure 2—figure supplement 1). Although the correlation between the drug effect (calculated as the subtraction of the percentage of change from placebo of the levodopa intervention minus the percentage of change from placebo of the risperidone intervention) for pleasantness ratings and the PAS was not significant for the main condition of interest (M+, rs = −0.274, p=0.196), raw pleasantness ratings during placebo and levodopa interventions separately were indeed correlated with participants’ individual differences in sensitivity to reward (see the Supplemental Correlational Analyses section of Appendix 1 and Figure 2—figure supplement 2 ).

Figure 2.

Relation between the effect of the pharmacological intervention for the M+ condition and subjective sensitivity to reward for the learning scores (i.e., online Learning on Day 1, Words Remembered on Day 2; Recognition Rate on Day 2) obtained by (A) correlating drug effect and PAS scores (the lower the PAS values are, the higher the general hedonia); (B) computing the drug effect (mean ± SEM) according to high (Hedonic +) and low (Hedonic -) hedonic subjects (median split using the PAS).

*p<0.05, **p<0.001.

Significant correlations (FDR-corrected for multiple comparisons) are marked in bold letters and with a red line.

Significant correlations (FDR-corrected for multiple comparisons) are marked in bold letters and with a red line.

Figure 2—figure supplement 1.

Correlations between the PAS and the different measures of M+ learning and memory during risperidone, placebo or levodopa interventions.

Significant correlations (FDR-corrected for multiple comparisons) are marked in bold letters and with a red line.

Figure 2—figure supplement 2.

Correlations between the PAS and the M+ subjective ratings during risperidone, placebo or levodopa interventions.

Significant correlations (FDR-corrected for multiple comparisons) are marked in bold letters and with a red line.

Altogether, this pattern of results suggests that the dopaminergic pharmacological intervention induced greater memory and accuracy benefits/deficits in those participants with high sensitivity to reward. Note that the drug effect for the recognition rate, which showed no differences in memory performance when pooling all participants together, becomes significant if we divide our participants into high and low sensitivity to reward (i.e. hedonic) groups [U = 35, p=0.033, η2 = 0.19, BF+0=2.81; groups divided according to the median split of their PAS scores; due to the reduced N of the groups we used a Mann-Whitney non-parametric statistical tests, see Materials and methods, Figure 2B, Appendix 1 and Figure 1-Figure supplementary 1B]. This difference was also present when comparing high/low groups for the number of words learned during encoding [U = 33, p=0.024, η2 = 0.21, BF+0=4.34] and the total number of words remembered on Day 2 [U = 12.5, p<0.001, η2 = 0.49, BF+0=78.74].

All in all, these results show that the dopaminergic pharmacological intervention did have an effect in terms of both online learning and subjective pleasantness in our learning task, inducing greater memory benefits in those participants more sensitive to reward.

Discussion

By using a double-blind, within-subject randomized pharmacological intervention during a learning task—guided by an intrinsically regulated reward process—known to activate the SN/VTA-HP loop (Ripollés et al., 2016), we showed that dopamine can modulate the entrance of new information into long-term memory. In particular, the administration of a dopaminergic precursor (levodopa) and a dopaminergic antagonist (risperidone), increased and decreased, respectively, both the learning rate and the level of pleasantness experienced by the participants during encoding, as well as the number of words remembered after a consolidation period (24 hr; see Figure 1B). Strikingly, the memory effects induced by the dopaminergic pharmacological intervention were stronger in participants with a higher sensitivity to reward (i.e. more hedonic; see Figure 2).

In a previous study using the same task (Ripollés et al., 2016), we showed that successful learning was in itself (i.e. in the absence of external feedback) associated with increased reward processing and heightened activity within the SN/VTA and the VS. We suggested that this intrinsic reward-related signal induced a higher release of dopamine in the HP, which ultimately resulted in enhanced memory formation due to the well-known role of dopamine in mediating LTP processes. The memory effects reported here (after a 24 hr consolidation period) are also in agreement with the prediction that the SN/VTA-HP loop should specially enhance delayed memory (Lisman and Grace, 2005; Lisman et al., 2011; Adcock et al., 2006; Wittmann et al., 2005; Wolosin et al., 2012; Murayama and Kitagami, 2014). In addition to the delayed memory effects reported, here we show that dopamine had an additional role during online learning on Day 1: participants not only learned more words (i.e. they performed better) under levodopa than under risperidone, but also found the learning experience more rewarding when potentiating, rather than hindering, dopaminergic synaptic signaling. This result is in accord with previous work demonstrating that dopamine improves feedback-based learning in humans (de Vries et al., 2010) and also with research showing that internally generated signals of self-performance—driven by mesolimbic areas and in absence of external feedback—can guide and improve perceptual learning in humans (Daniel and Pollmann, 2012; Daniel and Pollmann, 2014; Guggenmos et al., 2016) and song learning (i.e. motor performance) in songbirds (Mandelblat-Cerf et al., 2014). Moreover, these results also converge with proposals highlighting the role of internally triggered and regulated reward signals in sustaining intrinsic motivation to perform activities that allow acquiring and storing new information (Gottlieb et al., 2013; Berlyne, 1960). Importantly, the ability to glue information acquisition activities to internal reward mechanisms might have been crucial to the evolutionary success of humans (e.g. increasing environmental control, acquiring language; Barto, 2013; Ripollés et al., 2014).

An interesting interpretation of our results is therefore that the level of dopamine directly affected the reward value or the salience (Knecht et al., 2004) of the learning outcome in our task (i.e. learning was more enjoyable), prompting participants to be more motivated (Murty and Adcock, 2014) and to perform better. The VS, through its connections to the prefrontal cortex (PFC; Lehéricy et al., 2004; Cummings, 1993; Alexander et al., 1986), is located in a perfect anatomical position to add information about the relevance, salience and motivational value (Berridge and Kringelbach, 2008) of the stimuli to be learned into the SN/VTA-HP loop. As both the VS and the PFC are known to contain and receive dopaminergic receptors and projections (Haber and Knutson, 2010), dopamine might be able to alter this input, thus modulating the perceived reward underpinning the learning processes. This, together with the significant correlation between the PAS and the memory and learning scores (see Figure 2), is also in agreement with previous studies showing that anhedonia is associated with both reduced activity and connectivity between regions within the mesolimbic reward pathway (especially between the VS and the SN/VTA; Keller et al., 2013). An alternative explanation, which cannot be fully ruled out, is that the benefit in performance was driven by the suggested role of dopamine in working memory and attention (Surmeier, 2007; Brozoski et al., 1979; Linssen et al., 2014; Drijgers et al., 2012; Mehta and Riedel, 2006). However, the fact that no significant learning or memory benefits were induced in the control M- condition and, especially, the relationship between the learning improvements during the encoding phase and the participants’ sensitivity to reward (for a similar effect, see Ferreri and Rodriguez-Fornells, 2017), suggest that the learning enhancement was partially driven by reward-related and dopamine-dependent processes (Diehl and Gershon, 1992; Nieoullon and Coquerel, 2003; de Vries et al., 2010).

The dopaminergic-dependent memory effects reported in this work are also in line with previous studies using risperidone and levodopa, although note that there is a lack of data on the memory effects of risperidone in healthy humans. The cognitive effects caused by the blockade of D2 receptors have usually been studied in the treatment of schizophrenia (Rabella et al., 2016; Désaméricq et al., 2014), with antipsychotic drugs (including risperidone) sometimes improving cognitive function (Keefe et al., 1999), but also leading to impairments in several cognitive domains (Sakurai et al., 2013), including executive functions and memory (Uchida et al., 2009; Hori et al., 2006). On the other hand, levodopa intake has been related to improvements in feedback-based grammar learning (de Vries et al., 2010), semantic activation and priming (Angwin et al., 2004 and Angwin et al., 2009; Copland et al., 2009) and, most importantly, memory and learning (Shellshear et al., 2015; Knecht et al., 2004; Chowdhury et al., 2012). In this vein, the studies in which levodopa intake was related to long-term memory benefits, used associative learning tasks and suggested that the memory enhancements occurred possibly due to the increase of the levels of dopamine in the HP (Shellshear et al., 2015; Knecht et al., 2004; Chowdhury et al., 2012). However, our findings draw a more complex and perhaps more informative picture: the lack of a clear and significant memory enhancement for the control M- condition and the fact that more hedonic participants benefitted the most from the dopaminergic intervention only in the learning condition related to reward (M+), suggest that when using a reward-based learning task (Apitz and Bunzeck, 2013; Patil et al., 2017; Oyarzún et al., 2016; Kizilirmak et al., 2016; de Vries et al., 2010), the level of memory enhancement depends on dopamine synaptic signaling, but also on individual differences in sensitivity to reward (Ferreri and Rodriguez-Fornells, 2017; Mas-Herrero et al., 2014; Camara et al., 2010; Marco-Pallarés et al., 2009; Padrão et al., 2013). This discovery can be crucial for dopamine-related pharmacological interventions in, for example, clinical populations with language deficits (Berthier et al., 2011). Indeed, studies with levodopa in aphasia recovery, have resulted in both positive (Seniów et al., 2009) and negative (Breitenstein et al., 2015; Leemann et al., 2011) effects. In this type of therapy, in which patients try to learn of re-learn words that are no longer accessible (Brady et al., 2012), the intensity of the language training is usually related with recovery (Bhogal et al., 2003) and it has been suggested that high-training intensity may cause a ceiling effect that prevents levodopa from providing additional memory benefits (Breitenstein et al., 2015; Leemann et al., 2011). A reward-based learning task such as the one used here, along with a better understanding of the interaction between the dopaminergic precursor and the patient’s hedonic state could aid to achieve a more personalized and efficient rehabilitation success, without the need for high intensive training.

In conclusion, here we show that a dopaminergic pharmacological intervention is able to modulate behavioral measures of pleasantness, task-performance and long-term memory according to inter-individual differences in reward sensitivity. These findings further advance the idea that learning—even when achieved using a task guided by intrinsic reward—is a dopamine-dependent process, and shed new light on possible reward-based interventions for learning stimulation and/or rehabilitation.

Materials and methods

Participants

Around 150 individuals responded to advertisements and were contacted for a first phone pre-screening. Of those, 45 confirmed their availability and, after giving informed consent, were admitted at the hospital for further screening, medical examination and laboratory exams (blood and urine analysis). The study was approved by the Ethics Committee of Hospital de la Santa Creu i Sant Pau and the Spanish Medicines and Medical Devices Agency (EudraCT 2016-000801-35). The study was carried out in accordance with the Declaration of Helsinki and the ICH Good Clinical Practice Guidelines. All volunteers gave their written informed consent to participation prior to any procedure.

Subjects were judged healthy at screening 3 weeks before the first dose based on medical history, physical examination, vital signs, electrocardiogram, laboratory assessments, negative urine drug screens, and negative hepatitis B and C, and HIV serologies. The volunteers were excluded if they had used any prescription or over-the-counter medications in the 14 days before screening, if they had a medical history of alcohol and/or drug abuse, a consumption of more than 24 or 40 grams of alcohol per day for female and male, respectively, or if they smoked more than 10 cigarettes/day. Women with a positive pregnancy test or not using efficient contraception methods and subjects with musical training or those unable to understand the nature and consequences of the trial or the testing procedures involved were also excluded. Additionally, volunteers were requested to abstain from alcohol, tobacco and caffeinated drinks at least during the 24 hr prior to each experimental period.

Twenty nine volunteers were randomized and completed the study (19 females, mean age = 22.83 ± 4.39) in exchange of a monetary compensation according to the Spanish Legislation. The original sample size was chosen to be 30 participants, but one participant dropped out early in the study and only 29 finalized it. This sample size was selected based on several criteria, including the recommendation that, in order to achieve 80% of power, at least 30 participants should be included in an experiment in which the expected effect size is medium to large (Cohen, 1988). In addition, we took into account the sample sizes of previous studies using levodopa to modulate memory (range: between 10 and 30 participants; Apitz and Bunzeck, 2013; Copland et al., 2009, de Vries et al., 2010 Knecht et al., 2004; Chowdhury et al., 2012; Shellshear et al., 2015) and our previous behavioral studies using the same learning task (24 participants; Ripollés et al., 2016). We also computed a sample size analysis using the G*Power program, which showed that a sample size of 28 was required to ensure 80% of power to detect a significant effect (0.25) in a repeated-measures ANOVA with three sessions at the 5% significance level. We excluded three participants from the analyses after showing very poor memory performance on the word learning task during the placebo session (on Day 2, they remembered less than four of the M+ words learned during the encoding session). The final sample analyzed for this learning paradigm consisted of 26 participants (17 women, mean age = 22.27 ± 3.69).

Study design and procedure

This double-blind, crossover, treatment sequence-randomized study was performed at the Neuropsychopharmacology Unit and Center for Drug Research (CIM) of the Santa Creu i Sant Pau Hospital of Barcelona (Spain). Experimental testing took place over three sessions. For each session, participants arrived at the hospital under fasting conditions and were given a light breakfast. Subsequently, they received in a double-blind masked fashion a capsule containing the treatment (see Appendix 2 for details about counter-balancing across drug and placebo sessions): a dopaminergic precursor with an inhibitor of peripheral dopamine metabolism (levodopa, 100 mg + carbidopa, 25 mg), a dopamine receptor antagonist (risperidone, 2 mg), or placebo (lactose). The dopaminergic system has a physiological or intrinsic state whose effects are most likely reflected by the values of the dependent variables measured during the placebo session. In this study, we intended to lower and raise this baseline dopaminergic state by means of two independent pharmacological interventions involving low-to-moderate doses of levodopa and risperidone. Drug doses were carefully chosen to be low enough to induce the desired modulation but not too large to allow collateral effects to become a confounding factor. In particular, the levodopa dose was kept in line with previous studies in healthy participants and within the dose range administered in clinical practice for the treatment of Parkinson's disease. While a higher dose could have been administered, increasing the likelihood of the dependent variables showing statistically significant differences when compared with placebo, this would have led to an unacceptable higher risk of adverse events (e.g. dangerous decreases in blood pressure, intense nausea and vomiting, prominent general discomfort). In addition, increasing the levodopa dose could also induce negative effects on cognition due to the inverted U-shaped effects of dopamine (Chowdhury et al., 2012). Regarding risperidone, a higher dose could have also confounded the experiment due to the well-known sedative effects risperidone can induce in healthy volunteers when administered at higher doses. Thus, drug doses use were decided upon these ethical concerns and the binding request on the part of our local Institute Review Board.

After 1 hr of completing several behavioral tasks not described in the current manuscript, the participants completed our word learning task which lasted 45 min approximately. Next, participants spent their time in a resting room and were allowed to leave the hospital after 6 hr from the treatment administration. For each session, each participant came back 24 hr after for a behavioral retesting (without any pharmacological intervention), which lasted about 15 min. At least 1 week passed between one session and the other.

Experimental word learning task

The task was virtually identical to that of our previous work (Ripollés et al., 2014; Ripollés et al., 2016; Ripollés et al., 2017). Stimuli were presented using the Psychophysics Toolbox 3.09 (Brainard, 1997) and Matlab version R2012b. Stimuli consisted of 168 pairs of eight word-long Spanish sentences ending in a new-word, built with an increasing degree of contextual constraint (Mestres-MisseMissé et al., 2007; Mestres-Missé et al., 2014). Mean cloze probability (the proportion of people who complete a particular sentence fragment with a particular word) was 29.16 ± 18.95% for the first sentence (low constraint), and 81.67 ± 11.80% for the second (high constraint). The new-words respected the phonotactic rules of Spanish, were built by changing one or two letters of an existing word (mean number of letters = 6.02 ± 0.99) and always stood for a noun (mean frequency 43.26 ± 78.94 per million).

For each of the three different sessions, only half of the pairs of sentences disambiguated multiple meanings, thus enabling the extraction of a correct meaning for the new-word (M+ condition; e.g., 1. 'Every Sunday the grandmother went to the jedin'2. 'The man was buried in the jedin'; jedin means graveyard and is congruent with both the first and second sentences). For the other pairs, second sentences were scrambled so that they no longer matched their original first sentence. In this case, the new-word was not associated with a congruent meaning across the sentences (M- condition; e.g., 1. 'Every night the astronomer watched the heutil'. Moon is one possible meaning of heutil. 2. 'In the morning break co-workers drink heutil.' Coffee is now one of the possible meanings of heutil, which is not congruent with the first sentence). These constituted the M- condition in which congruent meaning extraction was not possible. To ensure that both stimulus types were equally comparable, participants were told that it was just as crucial to learn the words of the M+ condition as it was to correctly reject the new-words from the M- condition.

Given that the pharmacological intervention included three sessions, we created three versions of our task that only differed in the stimuli being presented. Thus, the 168 pairs of sentences were divided into six lists of 28 pairs (as aforementioned, two conditions, M+ and M-, were presented in each of the three sessions). The six lists were created so that there were no differences (one-way ANOVA) in the cloze probability of the sentences [first sentences: F(5,162)=0.688, p=0.633, η2 = 0.021, BF10 = 0.044; second sentences: F(5,162)=0.419, p=0.835, η2 = 0.013, BF10 = 0.03], the frequency of the meanings of the real words to be learned [F(5,162)=1.324, p=0.256, η2 = 0.039, BF10 = 0.13] or the total number of letters of the new-words [F(5,162)=1.10, p=0.360, η2 = 0.033, BF10 = 0.09]. The six lists of sentences were randomly assigned in pairs to the three different sessions. Presentation of the lists was counterbalanced across the experiments so that half of the times one list was used for the M+ condition and the other half for the M-. For each participant, new-words were randomly assigned to each pair of sentences.

During each session, four pairs of M+ and four pairs of M- sentences were presented per learning block (7 blocks in total). Therefore, a total of 28 new-words from the M+ and 28 from the M- conditions were presented during each of the three sessions. In order to achieve an ecologically valid paradigm, presentation of the first and second sentences with the same new-word at the end were separated in time. The four first sentences of each of the M+ and M- conditions (a total of eight new-words) were presented in a pseudo-randomized order (e.g., M + 1A, M-1A, M-1B, M-1C, M + 1B, M + 1C, M + 1D, M-1D). Then, the second ‘pair’ sentences+ and M- conditions were presented (i.e. second presentation of the identical eight new-words), again in a pseudo-randomized order (e.g. M-2C, M-2B, M + 2B, M + 2D, M-2D, M + 2C, M + 2A, M-2A). The temporal order of the different new-words during first sentence presentation was not related in any systematic way to the order of presentation of the same new-words for their second sentence. Participants were instructed to produce a verbal answer 8 s after the new-word of a second sentence appeared. If participants thought that the new-word had a congruent meaning, they had to provide its meaning in Spanish (e.g. graveyard). If the new-word had no consistent meaning, they had to say the word incongruent. If they did not know whether the new-word had a consistent meaning or not, they had to remain silent. Vocal answers were recorded and later corrected (for the M+ condition, incorrect answers included misses, providing the wrong meaning or saying incongruent; for the M- condition, incorrect answers included misses or providing any meaning at all). After giving a verbal answer, participants first provided a confidence rating that allowed for the assessment of the subjective evaluation of their performance. Specifically, subjects were requested to enter, using the keyboard, a value between −4 and 4 (9 point scale with 0 as the neutral value). Then, participants had to rate their emotions with respect to arousal and pleasantness using the 9-point (as with confidence ratings, from −4 to 4) visual Self-Assessment Manikin scale (SAM). For valence/pleasantness, the SAM ranges from a sad, frowning figure (i.e. very negative) to a happy, smiling figure (i.e. very positive). For arousal, the SAM ranges from a relaxed figure (i.e. very calm) to an excited figure (i.e. very aroused). All participants completed a training block to familiarize them with the task.

Each trial started with a fixation cross lasting 1000 ms, continued with the seven first Spanish words of the sentence presented for 2 s, and was followed by a 1-s duration dark screen. The new-word was presented for 1000 ms and was followed by 7 s of a small fixation point presented in the middle of the screen. For first sentences, a new trial was presented after 3 s of dark screen. For second sentences, after this period, a screen with the word Answer appeared and subjects had 3 s to produce a verbal answer. Then, the confidence and SAM scales for pleasantness and arousal were sequentially presented (the experiment did not continue until participants provided a rating). Finally, a new second sentence trial started after 3 s of dark screen. All words were placed in the middle of a black screen with a font size of 22 and in white color.

To avoid biasing our results, participants were not told at any point prior to the start of the experiment that the goal of the study was to assess whether the learning of a new-word and its meaning was intrinsically rewarding. Instead, they were told that the objective of the study was to assess how reading load affects mood and that, in order to ensure that there was a real reading load, they had to learn the words of the M+ condition and to detect the incongruence of the new-words from the M-. Finally, participants were told that they had to give pleasantness and arousal ratings when the second sentences appeared because that moment signaled that reading load had already occurred (i.e. half of the encoding block had already elapsed). After the experiment, participants were first questioned about the objective of the study. None of them answered that it was to assess whether word-learning was rewarding.

Approximately 24 hr after the learning lesson ended, participants returned to the lab to complete a recognition test (note that no drugs were administered to subjects on Day 2). In this test, participants were presented, in a pseudo-randomized order, with all the 28 M+ and 28 M- new-words used during the encoding session. This test was devised in order to assess which of the learned words during encoding were still remembered and which of them had been forgotten after a 24 hr retention period. Participants were aware that they would complete this test before completing the encoding session. It was made explicit that they would assess both M+ and M- new-words during the test phase. In the test, participants were presented with a new-word at the centre of the screen with two possible meanings below: one on the left and one on the right. If the new-word tested did not have a congruent meaning associated between the first and the second sentence, and thus correct meaning extraction was not possible (M- condition), participants had to press a button located in their left hand. In this case, the two possible meanings presented served as foils: one was the meaning evoked by the second sentence of the M- new-word being tested; the other word shown was the meaning evoked by another second sentence presented in the same run as the new-word being tested. Instead, if the new-word tested had a consistent meaning through the first and second sentence, and thus correct meaning extraction was possible (M+ condition), participants had to select the correct meaning among the two presented. In this case, one of the two possible meanings was correct and the other, which served as a foil, was the meaning of another new-word presented in the same run. In addition, participants could also press a fourth button if they did not know the answer. Thus, chance level was at 25% (no consistent meaning, consistent meaning on the left, consistent meaning on the right, not remembered).

Statistical analyses for confidence, pleasantness and arousal subjective scales and learning scores for encoding and retrieval

We first assessed whether the results of the placebo session replicated our previous behavioral data (Experiment 3 in Ripollés et al., 2016). Besides the three subjective ratings, for these first comparisons, we used two additional scores: the percentage of words learned on Day 1 (total number of words learned divided by the total number of words presented) and the recognition rate (total number of words learned during encoding and remembered on Day 2 divided by the number of words learned during Day 1). For the analyses regarding the subjective scales, we divided our M+ trials into those in which subjects learned the new-word during the learning session and still remembered it in the test after the recognition test (remembered condition) and those in which the new-word was not correctly identified in the 24 hr test (forgotten condition). We used the same approach to divide the M- trials into those in which a word was correctly marked as incongruent during encoding and still correctly rejected after 24 hr and those in which the new-word was not correctly rejected in the follow-up test. To replicate our previous results, we first used paired t-tests to compare whether ratings for confidence, arousal and pleasantness were greater for remembered than for forgotten M+ and M- new-words. We then submitted both the ratings and the learning scores to a mixed repeated measures ANOVA with Condition (M+,M-) as a within-subjects variable and Group (Pharmacological Group, Exp. 3 in Ripollés et al., 2016) as a between subjects variable.

Given that current behavioral results replicate our previous work (see results) and that in our previous study (Ripollés et al., 2016) remembered M+ words were the trials showing the highest fMRI activity within the SN/VTA-HP loop, the largest physiological response and the highest subjective pleasantness ratings, we focused all the analyses regarding the effect of the pharmacological intervention in the trials in which a word was learned during encoding and still remembered during the recognition test at 24 hr (M+ condition). For the control condition, we used those M- trials in which a word was correctly rejected during both encoding and the follow-up test. As measures for memory effects, we used the total number of words learned during encoding and remembered on the follow-up test and the percentage remembered words in the recognition test compared to the number of learned words during the learning phase (i.e. the recognition rate). Note that these two measures are delayed memory scores, while the percentage of learned words during Day 1 is an online learning rather than a pure delayed memory measure (participants try to learn the meaning of each new word from the two sentences presented and then provide a verbal answer when the second sentence is presented).

To ensure that no online learning or memory effects were present across sessions independent of the treatment (i.e. participants could learn more on the third session just because they completed the task twice before), we performed a control analysis for the percentage of learned words during Day 1, total number of words remembered on Day 2 and the recognition test. For each score, we computed a repeated measures ANOVA with Session (First, Second, Third; regardless of the treatment) as the main factor. There was not a significant effect of session order regardless of the treatment for the main M+ learning condition [percentage of learned words during Day 1: F(2,50)=0.957, p=0.391, η2 = 0.037, BF10 = 0.236; total number of words remembered on Day 2: F(2,50)=0.489, p=0.616, η2 = 0.019, BF10 = 0.161; recognition rate on Day 2: F(2,50)=0.263, p=0.770, η2 = 0.010, BF10 = 0.133] or M- control condition [percentage of correctly rejected words during Day 1: F(2,50)=0.964, p=0.388, η2 = 0.037, BF10 = 0.234; total number of correctly rejected words on Day 2: F(2,50)=1.250, p=0.295 η2 = 0.048, BF10 = 0.284; recognition rate on Day 2: F(2,50)=0.604, p=0.550, η2 = 0.024, BF10 = 0.176].

The purpose of this study was to elucidate whether modulation of the dopaminergic system influenced the variables under study (i.e. memory, learning and reward), rather than to assess the capacity of the drugs themselves to block or enhance the natural physiological responses influenced by dopamine. Levodopa and risperidone were chosen to ‘displace’ the baseline physiological system in opposite directions: risperidone to lower the effects of physiological dopamine release and levodopa to enhance dopaminergic neurotransmission. Thus, as the objective was to bring the dopaminergic system away from its intrinsic state (i.e. the placebo session) and in opposite directions, our analyses focused in directly comparing the risperidone and levodopa data against each other by using the placebo session as a baseline (but see Appendix 1 for analyses taking into account the three pharmacological sessions separately). For each learning score and subjective scale, we calculated the percentage of change from the placebo session [e.g. (levodopa score - placebo score)/(placebo score)]. Therefore, for each participant, learning score and subjective scale, we obtained the percentage of change from placebo of the risperidone and levodopa interventions. We used paired t-tests to calculate whether the difference between the changes induced by the risperidone and levodopa interventions were significant.

For the correlations between the learning scores and the PAS, we used Spearman's rho with a p<0.05 FDR correction to account for the three different correlations calculated per condition. The PAS is one of the most easy to administer and well-validated tests for measuring reward sensitivity and anhedonia. In addition, it shows convergent results with more modern tests measuring trait anhedonia, as the Mood and Anxiety Symptom Questionnaire Short Form (Keller et al., 2013) or the Temporal Experience of Pleasure Scale (Favrod et al., 2009). Some other psychometric measures of reward sensitivity and anhedonia that are translated and validated into Spanish, such as the Sensitivity to Punishment/Sensitivity to Reward Questionnaire (Torrubia et al., 2001) or the BIS/BAS (Caseras et al., 2003) show poorer alpha coefficients—less than 0.78 and 0.73, respectively—than the PAS (0.92; Fonseca-Pedrero et al., 2009). Indeed, we have previously used the PAS in previous studies exploring individual differences in anhedonia trait (Padrão et al., 2013; Mas-Herrero et al., 2014; Martínez-Molina et al., 2016; Mas-Herrero et al., 2018). Here, the PAS was used as a proxy to reflect the degree of pleasure taken by individuals when engaging in rewarding behavior (Der-Avakian and Markou, 2012). Note that two participants were excluded from this analysis as they did not complete the PAS. We also correlated the learning scores with a weight-dependent measure of drug dose, calculated in mg of levodopa/risperidone per kilogram. Finally, we used the median PAS value to split our final sample of 24 participants into high and low hedonic groups. For the learning scores, we first calculated the drug effect as a subtraction of the percentage of change from placebo induced by the levodopa intervention minus that induced by the risperidone intervention. We then assessed were the total drug effect for the learning scores was different for high vs. low hedonic groups by using a non-parametric test Mann-Whitney (to better account for the reduced number of participants in each group).

For significant interactions of mixed between-within ANOVA models, partial eta squares (η2) is provided as a measure of effect size. For significant differences in between group one-way ANOVAs, eta squares (η2) is provided (calculated by dividing the between groups sum of squares by the total sum of squares). For significant differences measured with t-tests, Cohen's d is provided after applying Hedges’ correction (the average of the standard deviation of the variables being compared was used as a standardizer; Cumming, 2012). For significant differences measured with the Mann-Whitney test, eta squares (η2) is provided (calculated as Z2/N)

In addition, confirmatory Bayesian statistical analyses were computed with the software JASP using default priors (JASP Team, 2018; Morey et al., 2015; Rouder and Morey, 2012; Wagenmakers et al., 2018b; Wagenmakers et al., 2018a). We reported Bayes factors (BF10), which reflect how likely data is to arise from one model, compared, in our case, to the null model (i.e. the probability of the data given H1 relative to H0). For comparisons with a strong a priori, the alternative hypothesis was specified so that one group/condition was greater than the other (BF+0). We did this, specifically, for the drug effects comparisons in which we expected levodopa and risperidone to facilitate and disrupt learning/ratings, respectively; and for the group comparisons in which we expected more hedonic participants to remember more words than less hedonic participants. For mixed within-between models we used the Bayes Inclusion factor based on matched models, representing the evidence for all models containing a particular effect to equivalent models stripped of that effect (BFInlcusion, also called Baws factor).

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Intrinsically regulated learning is modulated by synaptic dopamine availability" for consideration by eLife. Your article has been reviewed by three peer reviewers, including Vishnu Murty as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Sabine Kastner as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Bart Larsen (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

Summary:

Rippolles and colleagues present an interesting pharmacological study following-up on prior work showing engagement of VTA-HPC loops on intrinsically regulated learning. The authors use a pharmacological challenge to both up-regulate (l-Dopa) and down-regulate (risperidone) dopamine signaling during learning, and characterize the effects of putative "intrinsic rewards" on encoding, delayed memory, and affective ratings. Myself and the two reviewers all found merit in the manuscript, and were excited about the contribution of this pharmacological challenge on a growing field of reward and episodic memory.

Essential revisions:

1) The Introduction/Discussion warrants more theoretical and empirical background on l-Dopa and Risperidone. In particular, the authors should address the differences across these pharmacological challenges as they relate to dopamine physiology (i.e., non-selective increases vs. D2-specific antagonism). Further, the authors should provide more details on how their findings fit in with prior work using these drug manipulations during learning and memory.

2) All three reviewers agreed that the authors should include further analysis comparing each drug group against placebo separately. This could either be done as post-hoc t-tests or including all three factors in an ANOVA. This approach would also be relevant for the correlations with PAS (both analysis and figures), as it is unclear whether differences are driven by l-DOPA increases or risperidone decreases.

3) More details are needed on the PAS scores, including empirical and theoretical evidence for using these measures in the Introduction/Discussion. Second, can the authors comment if any individuals classify as being clinically anhedonic, and if so this should be addressed in the discussion. Third, do PAS scores relate to learning/memory in the placebo condition? Fourth, can the authors provide more details of how they believe anhedonia relates to their learning mechanisms in the discussion.

4) Did some individuals perform worse on levodopa than risperidone. If so are these the same individuals across measures of learning, delayed recognition, and affective responses. Addressing this component, in light of the complexity of DA systems, may help connect these findings to the broader literature (i.e., U-shaped DA curves).

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Intrinsically regulated learning is modulated by synaptic dopamine signaling" for further consideration at eLife. Your article has been reviewed by three peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Joshua Gold as the Senior Editor.

The manuscript has been significantly improved but there are a few remaining issues that need to be addressed before acceptance, as outlined below:

As you can see below, there are just a few minor, easily addressable points. These are centered on the following items. First, the authors should qualify the discussion of the results, in particular describing them as fully bidirectional, as a few of the results did not show significant differences across baseline for both drugs. Second, the authors should revise their supplemental across-subject analysis to use each drug separately compared against baseline, as opposed to each correlation on its own. Third, the authors should include the findings (which were detailed in the revision) in the supplement of the number of participants in the L-dopa condition performing worse. Finally, the authors should include additional details of why PAS/anhedonia was used as the measure of reward sensitivity as opposed to other measures.

Below you can see the point-by-point responses by each reviewer.

Reviewer #1:

The authors did a commendable job responding to the revisions in the previous manuscript. I only had a few remaining concerns that should be easily addressable.

Firstly, the authors throughout the discussion refer to the bi-directionality of the modulation of dopaminergic neurotransmission. Given that some of the findings from the comparisons against placebo were not significant, these claims should be qualified.

Further, I believe the request for separate across-subject analyses were mis-interpreted in the first round of reviews. Rather, then looking at each correlation in each group separately, I was hoping to see the correlations with scores calculated as Risperidone>Baseline and then separately L-Dopa>Baseline. I believe these will be more compelling correlations as they remove any subject-specific variance.

Reviewer #2:

1) I think the authors could have done a better job addressing rationale for using the PAS and could include more in the discussion about this. There are many measures of reward sensitivity and anhedonia and there is not much discussion of why this measure was included.

2) Please include the findings about some people performing worse in L-dopa in the supplement.

3) I apologize for not noticing this before, so this is something for the researchers to consider in the future. There are findings that suggest that individuals who smoke cigarettes who have abstained from smoking have blunted reward anticipation to money (Sweitzer et al., Biological Psychiatry, 2014). In the future I would encourage the authors to perhaps recruit only non-smokers or to include smoking status as a variable in their analyses. I would be curious to know how many participants in their pool were smokers and if their findings differed based on smoking status. I do not think it is necessary to include in the present manuscript.

Reviewer #3:

The changes made by the authors have sufficiently addressed my concerns. In particular, I feel the added analyses and figures provide a richer presentation of the data (even though many are supplemental) substantially improve the manuscript.

Essential revisions:

1) The Introduction/Discussion warrants more theoretical and empirical background on l-Dopa and Risperidone. In particular, the authors should address the differences across these pharmacological challenges as they relate to dopamine physiology (i.e., non-selective increases vs. D2-specific antagonism). Further, the authors should provide more details on how their findings fit in with prior work using these drug manipulations during learning and memory.

With hindsight, we agree with the reviewer that more information regarding how levodopa and risperidone affect dopamine physiology should be included in the manuscript. The Introduction section has been updated accordingly:

“Levodopa is rapidly taken up by dopaminergic neurons, transformed into dopamine and stored in vesicles from which it will be released into the synaptic cleft each time the neuron fires. […] Each of the two experimental sessions involving active drugs were intended to shift dopaminergic neurotransmission away from each individual’s physiological status, as measured in a placebo session, and in opposite directions: levodopa to enhance the dopamine availability for release into the synapse, and risperidone to reduce synaptic transmission of the dopamine-associated signal by hindering dopamine-receptor interactions (see e.g. Rabella et al., 2016; Wittman and D’Esposito, 2015; De Vries et al., 2010; Knecht et al., 2004 for the use of levodopa or risperidone during cognitive tasks).”

Regarding how our findings are related to prior work using these drug manipulations, the Discussion section has been extended to include a more detailed account of previous and current work:

Discussion section:

“The dopaminergic-dependent memory effects reported in this work are in line with previous studies using risperidone and levodopa, though note that there is a lack of data on the memory effects of risperidone in healthy humans. […] In particular, the lack of a clear and significant memory enhancement for the control M- condition and the fact that more hedonic participants benefitted the most from the dopaminergic intervention only in the learning condition related to reward (M+), suggest that when using a reward-based learning task (Apitz and Bunzeck, 2013; Patil et al., 2016; Oyarzun et al., 2016; Kizilirmak et al., 2015; de Vries et al., 2010), the level of memory enhancement depends on dopamine synaptic signaling, but also on the individual differences in sensitivity to reward (Ferreri et al., 2017; Mas-Herrero et al., 2014; Camara et al., 2010; Marco-Pallares et al., 2009; Padrao et al., 2013).”

We have also updated the title of the manuscript to reflect a more accurate account of the effects induced by risperidone and levodopa:

Intrinsically regulated learning is modulated by synaptic dopamine signaling

2) All three reviewers agreed that the authors should include further analysis comparing each drug group against placebo separately. This could either be done as post-hoc t-tests or including all three factors in an ANOVA. This approach would also be relevant for the correlations with PAS (both analysis and figures), as it is unclear whether differences are driven by l-DOPA increases or risperidone decreases.

The reviewers raise a crucial point that we now highlight in the revised version of our manuscript. While previous studies have mainly focused on the role of levodopa in enhancing memory, here we decided to use a two-way approach to assess whether our intrinsic-reward learning task was modulated bidirectionally to link dopamine-related modulation even more tightly to the behavioral read-outs and the critical receptor types.

The dopaminergic system has a physiological or intrinsic state whose effects are reflected by the values of the dependent variables measured in the course of the placebo session. What was intended and attained in our study was to lower and raise this baseline dopaminergic state by means of two independent pharmacological interventions involving low-to-moderate doses of two different drugs. These drugs were chosen to "displace" the baseline physiological system in opposite directions: risperidone to lower the effects of physiological dopamine release and levodopa to enhance dopaminergic neurotransmission. Thus, by bringing the dopaminergic system away from its intrinsic state and in opposite directions, our analyses were focused in directly comparing the risperidone and levodopa data against each other. In other words, the purpose of this study was not to assess the capacity of the drugs themselves to block or enhance the natural physiological responses influenced by dopamine, but rather to elucidate whether modulation of the dopaminergic system influenced the variables under study (i.e., memory, learning and reward).

While we hypothesize that larger doses of each treatment would have increased the reported effects, the doses of both levodopa and risperidone were carefully chosen to be low enough to induce the desired modulation but not too large to allow collateral effects to become a confounding factor. In particular, the levodopa dose was kept in line with previous studies in healthy participants and within the dose range administered in clinical practice for the treatment of Parkinson's disease (PD). While a higher dose could have been administered, again increasing the likelihood of the dependent variables showing statistically-significant differences when compared with placebo,our local Institutional Review Board (IRB) strongly insisted against the use of larger doses of levodopa. They argued that participants were healthy volunteers, not patients suffering from dopamine deficiency (i.e., not PD patients), and were seriously concerned that larger doses would lead to an unacceptable higher risk of adverse events. Amongst the untoward effects cited by the IRB where: dangerous decreases in blood pressure, intense nausea and vomiting, and prominent general discomfort. In addition, increasing the levodopa dose could also induce negative effects on cognition due to the inverted U-shaped effects of dopamine (Chowdury et al., 2012). Thus, the final levodopa dose use was decided upon these ethical concerns and the binding request on the part of the IRB.

Regarding risperidone, we also hypothesize that a higher dose would have probably led to significant results vs. placebo. However, these effects would have been confounded by the well-known sedative effects risperidone can induce in healthy volunteers when administered at higher doses.

We have extended the Materials and methods section to better explain the rationale behind our analyses:

“The dopaminergic system has a physiological or intrinsic state whose effects are most likely reflected by the values of the dependent variables measured during the placebo session. […] Thus, drug doses use were decided upon these ethical concerns and the binding request on the part of our local Institute Review Board.”

“The purpose of this study was to elucidate whether modulation of the dopaminergic system influenced the variables under study (i.e., memory, learning and reward), rather than to assess the capacity of the drugs themselves to block or enhance the natural physiological responses influenced by dopamine. Levodopa and risperidone were chosen to "displace" the baseline physiological system in opposite directions: risperidone to lower the effects of physiological dopamine release and levodopa to enhance dopaminergic neurotransmission. Thus, as the objective was to bring the dopaminergic system away from its intrinsic state (i.e., the placebo session) and in opposite directions, our analyses focused in directly comparing the risperidone and levodopa data against each other by using the placebo session as a baseline (but see Appendix 1 for analyses taking into account the three pharmacological sessions separately).”

To comply with the reviewers’ suggestions, we also performed the following analyses, which were again in line with our interpretation. We have used repeated measures ANOVAs with three levels (Risperidone, Placebo and Levodopa) and paired post-hoc t-tests to re-analyze the data for the learning and memory scores and the subjective ratings. For the learning measures, in addition and taking into account the relationship between the PAS and the percentage of learned and remembered words, we have also calculated 3 x 2 mixed repeated measures ANOVAs with Session (Risperidone, Placebo and Levodopa) as the within and Group (H+, H-) as the between group factor. The results of these analyses are in agreement with our previous report: the pharmacological intervention succeeds in modulating (i.e., displacing the dopaminergic baseline physiological system) both reward and memory/learning scores (i.e., a significant effect of Session is found).

Specifically, for the main M+ learning condition, we found a main effect of session for the percentage of words learned on Day 1 [F(2,50)=5.041, p=0.01, partial η2=0.168] and the total number of remembered words on Day 2 [F(2,50)=4.793, p=0.012, partial η2=0.161]. Post-hoc paired t-tests with Holm-Bonferroni correction showed, as expected, a significant difference between the Levodopa and the Risperidone sessions for both learning on Day 1 [t(25)=2.95, p_corr=0.02, d=0.580] and the total number of words remembered on Day 2 [t(25)=2.44, p_corr=0.044, d=0.479]. Comparisons of levodopa against placebo were not significant after the correction for multiple comparisons was applied [Learning on Day 1: t(25)=1.629, p_corr=0.214, d=0.319; Total Number of Remembered Words on Day 2: t(25)=-0.240, p_corr=0.812, d=0.047]. The risperidone vs. placebo comparison reached significance for the total number of remembered words on Day 2 [t(25)=-2.90, p_corr=0.023, d=0.569], while was still not significant for the percentage of learned words on Day 1 [t(25)=-1.63, p_corr=0.214, d=0.328]. As expected, the recognition rate on Day 2 yielded no significant main effect of Session [F(2,50)=1.661, p=0.20, partial η2=0.062].

When dividing the participants into two groups according to their PAS scores, the ANOVA results were also in agreement with our main analyses. A significant Group x Session interaction was found for the total number of remembered words on Day 2 [F(2,50)=7.825, p<0.001, partial η2=0.262], while for the percentage of words learned on Day 1 [F(2,50)=2.786, p=0.073, partial η2=0.112] and the recognition rate on Day 2 [F(2,50)=2.615, p=0.084, partial η2=0.106] the interaction was marginally significant.

Regarding the subjective ratings, as expected, there was a significant main effect of Session for Pleasantness [F(2,50)=2.710, p=0.009, partial η2=0.174] and Confidence [F(2,50)=3.70, p=0.032, partial η2=0.129], but not for Arousal [F(2,50)=0.319, p=0.728, partial η2=0.013]. Post-hoc paired t-tests with Holm-Bonferroni correction showed that Pleasantness ratings were higher under Levodopa than under Risperidone [t(25)=2.85, p_corr=0.026, d=0.560], with no significant differences between the levodopa [t(25)=1.584, p_corr=0.137, d=0.311] or risperidone [t(25)=-1.903, p_corr=0.137, d=0.373] against the placebo session. For Confidence ratings no significant differences were found between any pair of sessions after applying the correction for multiple comparisons [Risperidone vs. Placebo: t(25)=-2.275 p_corr=0.095, d=0.446; Risperidone vs. Levodopa: t(25)=2.231, p_corr=0.095, d=0.438; Levodopa vs. Placebo: t(25)=0.521, p_corr=0.607, d=0.102].

We have added a new figure to show the results for the three sessions separately as a figure supplement to Figure 1. See Figure 1—figure supplement 1

Following the reviewers’ advice we have also calculated correlations (using Spearman´s rho) between the PAS and all measures of learning and memory and also all of the subjective ratings at each session (a total of 18 correlations; we used a p<0.05 FDR corrected threshold to control for the multiple correlations calculated). We have created two figures supplemental to Figure 2 to depict these analyses, see Figure 2—figure supplements 1 and 2.

These correlational analyses show that the relationship between the PAS and the different measures of memory and learning and the pleasantness ratings are also modulated by the drug intervention: r scores are consistently lower for risperidone and higher for levodopa sessions.

We think that these additional analyses further show that the pharmacological intervention modulated the reward and memory related measures collected and further provide causal evidence for a dopamine-dependent mechanism instrumental in intrinsically regulated learning.

We have added these three figures as Supplements to Figures 1 and 2, added an Appendix with the complementary analyses of the three sessions separately and updated the Results section accordingly:

“This suggests that the pharmacological intervention was able to modulate measures of reward, memory and online learning selectively during the main M+ condition. Additional analyses using the values for the three sessions separately (instead of the percentage of change from placebo) further confirmed this pattern of results (see the Supplemental Behavioral Analyses section of Appendix 1 and Figure 1—figure supplements 1A and 1C).”

“Additional correlational analyses with the results of each session separately (risperidone, placebo, levodopa) instead of the drug effect only, further confirm a relationship between learning and memory scores and the PAS, with Spearman’s rhos for this relationship being consistently lower than placebo for risperidone and higher than placebo for levodopa (see the Supplemental Correlational Analyses section of Appendix 1 and Figure 2—figure supplement 1). Although the correlation between the drug effect (calculated as the subtraction of the percentage of change from placebo of the levodopa session minus the percentage of change from placebo of the risperidone session) for pleasantness ratings and the PAS was not significant for the main condition of interest (M+, rs=-0.274, p=0.196), raw pleasantness ratings during placebo and levodopa sessions separately were indeed correlated with participants’ individual differences in sensitivity to reward (see the Supplemental Correlational Analyses section of Appendix 1 and Figure 2—figure supplement 2).”

Appendix 1:

“Supplemental Behavioural Analyses

While the purpose of this work was to elucidate whether modulation of the dopaminergic system influenced the variables under study (i.e., intrinsically regulated learning), rather than to assess the capacity of the drugs themselves to significantly block or enhance the natural physiological responses influenced by dopamine, here we provide separated analyses for the three different sessions (risperidone, placebo, levodopa) and the main condition of interest (M+). […] The significant main effects and interactions for both memory and reward measures further demonstrated that: i) the pharmacological intervention attained its main objective, which was to bring the dopaminergic system away from its intrinsic state (i.e., the placebo session) and in opposite directions (see, for example the Pleasantness ratings in Figure 1—figure supplement 1C); ii) that intrinsically regulated learning is a dopamine-dependent mechanism; and iii) that subject-specific reward/hedonic sensitivity drastically alters learning success (modulations for memory and learning scores only happened in highly hedonic individuals, see Figure 1—figure supplement 1B).”

“Supplemental Correlational Analyses

Following the same rationale as with the supplemental behavioral analyses above and taking into account the significant correlations found between the drug effect (calculated as the subtraction of the percentage of change from placebo of the levodopa session minus the percentage of change from placebo of the risperidone session) and the PAS (see Figure 2 in the main text), here we also calculated correlations (using Spearman´s rho) between the PAS and all measures of learning, memory and the subjective ratings at each session separately (a total of 18 correlations; we used a p<0.05 FDR corrected threshold to control for the multiple correlations calculated). […] These results further suggest that: i) the pharmacological intervention was able to "displace" the baseline physiological system in opposite directions, making the learning experience more (levodopa) and less (risperidone) rewarding; ii) that this led to higher and lower learning and retention rates; and iii) more hedonic participants were able to better capitalize in the increase of the “rewarding experience” (see, for example, the high correlation between the PAS and the retention rate on Day 2 in the levodopa session in Figure 2—figure supplement 2).”

3) More details are needed on the PAS scores, including empirical and theoretical evidence for using these measures in the Introduction/Discussion. Second, can the authors comment if any individuals classify as being clinically anhedonic, and if so this should be addressed in the discussion. Third, do PAS scores relate to learning/memory in the placebo condition? Fourth, can the authors provide more details of how they believe anhedonia relates to their learning mechanisms in the discussion.

We thank the reviewers for these suggestions. We have added new information in different parts of the manuscript regarding the rationale and the implications of using the PAS:

Results section:

“Given that our learning task modulates activity within the reward network and is associated with increased subjective reports of pleasure (Ripollés et al., 2014 and 2016), we further tested whether individual differences in sensitivity to reward interacted with the drug intervention to modulate memory benefits (Ferreri et al., 2017; de Vries et al., 2010; Apitz and Bunzeck, 2013). Twenty-four out of the twenty-six participants completed the Physical Anhedonia Scale (PAS, Chapman et al. 1976; mean score = 11.62 ± 5.47). The PAS is a well-validated scale that evaluates difficulty in feeling physical and aesthetic pleasure in response to typical pleasurable stimuli (Padrao et al., 2013; Mas-Herrero et al., 2014).”

Discussion section:

“This, together with the significant correlation between the PAS and the memory and learning scores (see Figure 2), is also in agreement with previous studies showing that anhedonia is associated with both reduced activity and connectivity between regions within the mesolimbic reward pathway (especially between the VS and the SN/VTA; Keller et al., 2013).”

Regarding whether any of the participants were clinically anhedonic there was one participant with a score of 24 that was more than 2 standard deviations above the mean PAS for the whole group. This is the participant that was a bivariate outlier in the correlations between the PAS and the drug effect (levodopa minus risperidone) for the total number of words remembered and also the recognition rate on Day 2 (this was all described in the Results section of the previous version of the manuscript). This participant, being a bivariate outlier, was excluded from those two correlations (note that the correlations were stronger when including the participant) and therefore is not shown in Figure 2. For the sake of clarity in Author response image 1 we depict the correlations with the anhedonic participant included.

Author response image 1.

Correlations between the PAS and the drug effect (calculated as the difference in percentage of change from the placebo session between levodopa and risperidone) with the anhedonic participant (marked with a red circle) included.

This participant has been included in the correlations between the PAS and each learning/memory score on each session shown in Figure 2—figure supplement 1 (see the previous answer 2 of this manuscript) as in those correlations the subject was not a bivariate outlier. This participant exhibits low percentages of learning on Day 1 and especially poor memory measures (only 5 words remembered) on Day 2. We have added more information about this participant in the Results section:

“Importantly, this participant obtained the highest (more anhedonic) score on the PAS (score of 24, more than 2 standard deviations above the mean score of 11.62 of the group).”

The PAS showed a trend for a correlation with the learning and memory scores during the placebo session as shown in the previous answer to point 2 and in Figure 2—figure supplement 1. As stated before, those correlations are now part of Appendix 1 of the new version of the manuscript.

4) Did some individuals perform worse on levodopa than risperidone. If so are these the same individuals across measures of learning, delayed recognition, and affective responses. Addressing this component, in light of the complexity of DA systems, may help connect these findings to the broader literature (i.e., U-shaped DA curves).

The reviewers raise an interesting point here. As can be seen in Figure 2A, there were some participants who performed worse in terms of learning and memory on levodopa than risperidone for the main learning condition (i.e., showed a negative drug effect, calculated as the subtraction of the percentage of change from placebo of the levodopa session minus the percentage of change from placebo of the risperidone session). In particular, the number of participants who performed worse in levodopa than in the risperidone session was 6 for the percentage of learned words on Day 1, 6 for the total number of words remembered on Day 2, 9 for the recognition rate on Day 2 and 9 for the pleasantness ratings. However, there was not a single participant who showed a negative drug effect in all three learning/memory scores and the pleasantness ratings. In contrast, 9 participants showed positive values in all memory and learning scores and in the pleasantness ratings. In this vein, the drug effect for pleasantness did not significantly correlate with any of the drug effects for learning and memory scores (learning on Day 1: r=-0.04, p=0.829; total number of remembered words on Day 2: r=0.108, p=0.608; recognition rate on Day 2: r=0.153, p=0.463).

If we assess the overlap for the learning and memory scores, there was only 1 participant showing a negative drug effect (levodopa minus risperidone) in the percentage of learned words on Day 1, the total number of words remembered on Day 2 and the recognition rate on Day 2 at the same time (while, in contrast, 12 showed positive values on the three measures). Indeed, while the drug effect for the percentage of learned words on Day 1 correlated with the drug effect for the total number of words remembered on Day 2 (r=0.569, p<0.003), the relationship was not significant for the recognition rate on Day 2 (r=-0.084,p=0.689).

Finally, 4 participants did perform poorer in the levodopa than the risperidone session for both the total number of words remembered on Day 2 and the recognition rate on Day 2 (while 15 participants showed positive values in both measures). This last result is not surprising, as these two memory measures are significantly correlated (r=0.638, p<0.001).

Since there was little overlap between these measures we have decided to exclude these analyses from the final version of the manuscript although we are more than willing to include them as supplemental data if the reviewers think that it is of importance.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

As you can see below, there are just a few minor, easily addressable points. These are centered on the following items. First, the authors should qualify the discussion of the results, in particular describing them as fully bidirectional, as a few of the results did not show significant differences across baseline for both drugs. Second, the authors should revise their supplemental across-subject analysis to use each drug separately compared against baseline, as opposed to each correlation on its own. Third, the authors should include the findings (which were detailed in the revision) in the supplement of the number of participants in the L-dopa condition performing worse. Finally, the authors should include additional details of why PAS/anhedonia was used as the measure of reward sensitivity as opposed to other measures.

Below you can see the point-by-point responses by each reviewer.

Reviewer #1:

The authors did a commendable job responding to the revisions in the previous manuscript. I only had a few remaining concerns that should be easily addressable.

We thank the reviewer for all her/his positive comments and for all the suggestions made during this revision, that we believe have greatly improved the quality of our manuscript.

Firstly, the authors throughout the discussion refer to the bi-directionality of the modulation of dopaminergic neurotransmission. Given that some of the findings from the comparisons against placebo were not significant, these claims should be qualified.

We have removed all the claims to the bi-directionality of the dopaminergic modulation throughout the manuscript.

Further, I believe the request for separate across-subject analyses were mis-interpreted in the first round of reviews. Rather, then looking at each correlation in each group separately, I was hoping to see the correlations with scores calculated as Risperidone>Baseline and then separately L-Dopa>Baseline. I believe these will be more compelling correlations as they remove any subject-specific variance.

We thank the reviewer for this suggestion. To comply with the reviewer’s request, we have now calculated the aforementioned analyses. No significant correlations were found between the PAS and the learning and memory scores or subjective ratings calculated as the difference from placebo for the risperidone and levodopa sessions separately (all ps>0.128). There was only a trend for the correlation between the PAS and the percentage of learned words on Day 1 for levodopa minus placebo (r=-0.39, p=0.055) and the total number of words remembered on Day 2 (r=-0.39, p=0.076) also for the changes in the levodopa versus the placebo session. We are not very surprised by these results, as we think that the more compelling correlations are those that describe the relationship between the total drug effects of the different measures (calculated as the subtraction of the percentage of change from placebo of the levodopa session minus the percentage of change from placebo of the risperidone session) and the PAS. As indicated in the manuscript, in our study we used two drugs to "displace" the baseline physiological system of each participant in opposite directions: risperidone to lower the effects of physiological dopamine release and levodopa to enhance dopaminergic neurotransmission. Thus, in our opinion, the subtraction of the levodopa and risperidone drug effects compared to placebo is the measure that better captures the modulation induced by our pharmacological intervention.

In addition, we really want to thank the reviewer for her/his previous suggestion that, although apparently misinterpreted by us, has led to what we believe are very interesting results. As shown in Figure 2—figure supplements 1 and 2, the relationship between the PAS and the different measures of memory and learning were also modulated by the drug intervention: r scores are consistently lower for risperidone and higher for levodopa sessions. Moreover, the PAS was correlated with the pleasantness ratings both during the placebo and the levodopa sessions. We find these last results of special interest, as they show that learning new words was more enjoyable for people who were more hedonic, even under normal circumstances (i.e., the placebo session). The relationship between reward and word learning was the main topic of the thesis dissertation of the co-first and corresponding author of this work (Pablo Ripollés) and he would very much like to include these analyses in the final version of the manuscript.

Taking into account that i) we designed the protocol to compare levodopa against risperidone results and ii) that we have a particular interest in describing the relationship between word learning and reward, we would like to leave the supplemental section and analyses as they are.

Reviewer #2:

1) I think the authors could have done a better job addressing rationale for using the PAS and could include more in the discussion about this. There are many measures of reward sensitivity and anhedonia and there is not much discussion of why this measure was included.

The PAS is one of the easiest to administer, well-validated and intensively used tests for measuring reward sensitivity and anhedonia. In addition, it shows convergent results with more modern tests measuring trait anhedonia, as the Mood and Anxiety Symptom Questionnaire Short Form (Keller et al., 2013) or the Temporal Experience of Pleasure Scale (Favrod et al., 2008). Some other psychometric measures of reward sensitivity and anhedonia that are translated and validated into Spanish, such as the Sensitivity to Punishment/ Sensitivity to Reward Questionnaire (Torrubia et al., 2001) or the BIS/BAS (Caseras, Avila and Torrubia, 2003) show poorer α coefficients—less than 0.78 and 0.73, respectively—than the PAS (0.92; Fonseca-Pedrero et al., 2009). Indeed, we have previously used the PAS in previous studies exploring individual differences in anhedonia trait (Padrao et al., 2013, Mas-Herrero et al., 2014, Martinez-Molina et al., 2016, Mas-Herrero et al., 2018). We honestly think that a discussion regarding the appropriateness of the different tests that measure anhedonia is out of the scope of this work. We want to remind the reviewer that the use of the PAS is discussed both in the Results and the Discussion section of the previous version of the manuscript. We have now further stressed our points regarding the PAS in the Material and Methods section:

“The PAS is one of the most easy to administer and well-validated tests for measuring reward sensitivity and anhedonia. In addition, it shows convergent results with more modern tests measuring trait anhedonia, as the Mood and Anxiety Symptom Questionnaire Short Form (Keller et al., 2013) or the Temporal Experience of Pleasure Scale (Favrod et al., 2008). Some other psychometric measures of reward sensitivity and anhedonia that are translated and validated into Spanish, such as the Sensitivity to Punishment/ Sensitivity to Reward Questionnaire (Torrubia et al., 2001) or the BIS/BAS (Caseras, Avila and Torrubia, 2003) show poorer α coefficients—less than 0.78 and 0.73, respectively—than the PAS (0.92; Fonseca-Pedrero et al., 2009). Indeed, we have previously used the PAS in previous studies exploring individual differences in anhedonia trait (Padrao et al., 2013, Mas-Herrero et al., 2014, Martinez-Molina et al., 2016, Mas-Herrero et al., 2018). Here, the PAS was used as a proxy to reflect the degree of pleasure taken by individuals when engaging in rewarding behavior (Der-Avakian et al., 2012).”

2) Please include the findings about some people performing worse in L-dopa in the supplement.

We have included this information in the new version of the manuscript, as requested:

Results section:

“This suggests that the pharmacological intervention was able to modulate measures of reward, memory and online learning selectively during the main M+ condition (see the Individual Variability of the Drug Effectsection of Appendix I, for a more in depth description of the individual differences found for the drug effect for each measure).”

Appendix I:

“Individual Variability of the Drug Effect

As can be seen in Figure 2A, there were some participants who performed worse in terms of learning and memory on levodopa than risperidone for the main learning condition M+ (i.e., showed a negative drug effect, calculated as the subtraction of the percentage of change from placebo of the levodopa session minus the percentage of change from placebo of the risperidone session). […] Indeed, the drug effects for these two memory measures are significantly correlated (r=0.638, p<0.001).”

3) I apologize for not noticing this before, so this is something for the researchers to consider in the future. There are findings that suggest that individuals who smoke cigarettes who have abstained from smoking have blunted reward anticipation to money (Sweitzer et al., Biological Psychiatry, 2014). In the future I would encourage the authors to perhaps recruit only non-smokers or to include smoking status as a variable in their analyses. I would be curious to know how many participants in their pool were smokers and if their findings differed based on smoking status. I do not think it is necessary to include in the present manuscript.

We thank the reviewer for this interesting suggestion. We will indeed take this into consideration for future studies. Note that participants in our study were excluded if they smoked more than 10 cigarettes a day.

Reviewer #3:

The changes made by the authors have sufficiently addressed my concerns. In particular, I feel the added analyses and figures provide a richer presentation of the data (even though many are supplemental) substantially improve the manuscript.

We thank the reviewer for all the comments provided during the revision process which have certainly improved the quality of our manuscript.

Appendix 2—table 1.

Counterbalancing across treatments, with six different sequences of letters randomly assigned to N = 30.

A corresponded to risperidone, B to placebo and C to levodopa.

Volunteer	Sequence	Treatment order
1	5	B/A/C
2	4	A/C/B
3	5	B/A/C
4	6	C/B/A
5	2	B/C/A
6	2	B/C/A
7	2	B/C/A
8	6	C/B/A
9	1	A/B/C
10	5	B/A/C
11	3	C/A/B
12	5	B/A/C
13	3	C/A/B
14	4	A/C/B
15	1	A/B/C
16	1	A/B/C
17	2	B/C/A
18	5	B/A/C
19	1	A/B/C
20	3	C/A/B
21	4	A/C/B
22	6	C/B/A
23	4	A/C/B
24	6	C/B/A
25	6	C/B/A
26	3	C/A/B
27	1	A/B/C
28	4	A/C/B
29	2	B/C/A
30	3	C/A/B