Reward disrupts reactivated human skill memory.

Accumulating evidence across species and memory domains shows that when an existing memory is reactivated, it becomes susceptible to modifications. However, the potential role of reward signals in these mechanisms underlying human memory dynamics is unknown. Leaning on a wealth of findings on the role of reward in reinforcing memory, we tested the impact of reinforcing a skill memory trace with monetary reward following memory reactivation, on strengthening of the memory trace. Reinforcing reactivated memories did not strengthen the memory, but rather led to disruption of the memory trace, breaking down the link between memory reactivation and subsequent memory strength. Statistical modeling further revealed a strong mediating role for memory reactivation in linking between memory encoding and subsequent memory strength only when the memory was replayed without reinforcement. We suggest that, rather than reinforcing the existing memory trace, reward creates a competing memory trace, impairing expression of the original reward-free memory. This mechanism sheds light on the processes underlying skill acquisition, having wide translational implications.

It is by now widely accepted that our memories are dynamic, even after their initial stabilization through consolidation. Thus, once an existing, already consolidated memory is retrieved or reactivated by a reminder, it becomes susceptible to modification12345. The reactivation and reinstatement of a memory trace can degrade1367, or conversely stabilize or strengthen48 it through the proposed mechanisms of reconsolidation910. However, despite an ever increasing interest in memory reactivation and reconsolidation, originating from the far-ranging translational implications of this line of research, our understanding of the mechanism that leads to memory modifications remains limited. As recent research has identified the unique role of reward and of reinforcement mechanisms in episodic and procedural memory formation11121314151617, here we set out to uncover how reinforcement contributes to memory modification through reactivation. In the first experiment we had a group of subjects learn a sequential motor skill181920 over 12 blocks of training. Then, a day later, subjects performed one reactivation trial of the task, with half of them receiving monetary reward in accordance with their performance and the rest receiving performance feedback alone. Memory strength was then measured on day 3 for all subjects, focusing on the specific mediating role of reactivation in the link between memory encoding and subsequent memory strength. An additional experiment tested the time-dependent effects of reinforcement following reactivation.

Results

We had subjects (n = 24) perform a sequential finger-tapping motor memory task181920, where they were asked to tap a sequence of key presses, as quickly and as accurately as possible during trials lasting 30 s each (Fig. 1a). An Initial 12 trials encoding phase took place on the first day of the experiment (Fig. 1b). Subjects then returned a day later, and were assigned to one of two groups. The first group (the rewarded group) was administered one reactivation trial of the task, with the number of correct sequences tapped during the trial monetarily reinforced. The second group of subjects (the unrewarded group) was administered the same reactivation trial, receiving feedback about their performance, rather than monetary reward (Fig. 1c). This design allowed us to tease apart monetary reward and simple performance feedback per se. All subjects then returned a day later and performed 3 additional trials, where their memory strength for the acquired skill was measured (Fig. 1c). Repeated measures analysis of variance (ANOVA) indicated that both groups showed comparable encoding (ΔDay1end -Day1start) of the skill memory (F1,22 = 0.536, ns, Fig. 1d). Of note, the groups also showed comparable reactivation strength (ΔDay2-Day1end, t22 = −1.017, ns).

Figure 1

Task and design.

(a) Subjects performed a sequential finger-tapping task where they had to repeatedly tap with their left non-dominant hand a 5-element sequence of finger movements as quickly and accurately as possible. Performance in the task was quantified in terms of the number of correct sequences tapped during each trial. (b) The task was composed of separate trials, lasting 30 s, separated by 30 s breaks. (c) The experiment included three sessions. In the first session (Day 1) all subjects performed twelve trials of the sequential finger-tapping task. On the next session (Day 2), participants were divided into two groups, the first performing one reactivation trial with reinforcement and the second with performance feedback instead of reinforcement, allowing to tease apart monetary reward and simple performance feedback per se. In the third session (Day 3) subsequent memory strength was measured by having both groups perform three test trials of the task, with no performance or reward feedback. (d) Both the rewarded and unrewarded groups showed comparable encoding of the skill memory. Shaded lines denote standard errors of the mean.

Not only that applying reward following reactivation did not strengthen (ΔDay3test-Day1end) memory retention compared to unrewarded reactivation (t22 = −1.834, ns), (Fig. 2a), but rather it exhibited a tendency towards inferior retention. To better understand the role of reward in skill memory reactivation we next examined whether the strength of reactivation correlated with subjects’ subsequent memory strength on Day3. Whereas a statistically significant correlation was found between reactivation strength and memory strength on Day3 in the unrewarded group (r = 0.616, p = 0.033, Fig. 2b), this correlation was abolished in the rewarded group (r = −0.243, ns, Fig. 2c). Both correlations were robust against outliers (bend correlation r = 0.606 and r = −0.08 in the unrewarded and rewarded groups respectively), and significantly different from one another (Fisher Z = 2.05, p = 0.0404).

Figure 2

Effects of reactivation on subsequent memory strength.

(a) Retention for the skill memory was inferior in the rewarded group. R, rewarded group; NR, unrewarded group. (b) In the unrewarded group reactivation strength significantly correlated with memory on Day 3 test (r = 0.616, p = 0.017). (c) In the rewarded group the correlation between memory reactivation strength and memory strength on Day 3 was not significant (r = −0.243, ns) and was significantly weaker than in the unrewarded group. Error bars denote standard errors of the mean.

To further uncover the effect of reward on memory reactivation and subsequent memory strength we considered two competing models. According to the first, the magnitude of memory on Day 3 depends on how well the memory was encoded, regardless of reactivation (Fig. 3a). In a second, indirect model, the effects of encoding on Day 3 memory strength are mediated by the strength of reactivation (Fig. 3a). Modeling the behavior displayed by the rewarded and unrewarded groups revealed clear differences among the two. Regression analysis indicated an overall similar strength, among the two groups, in each of the direct paths in the model (i.e., encoding to Day 3 memory strength, encoding to reactivation and reactivation to Day 3 memory strength; Fig. 3b,c, Table 1). However, testing for mediation effects based on an indirect path from encoding to Day 3 memory strength, bias-corrected confidence estimates and bootstrap resampling2122 showed that in the unrewarded group the effect of encoding on Day 3 memory strength was significantly mediated by reactivation (B = 1.284, confidence interval = 0.241 to 2.849, Table 2), but not in the rewarded group (B = 0.469, confidence interval = −0.23 to 1.01, Table 2).

Figure 3

The mediating role of reactivation in the link between encoding and subsequent memory strength.

(a) We considered two different models for the links between encoding and Day 3 memory strength, a first (Model I) assuming a direct link between the two, and a second indirect model (Model II) where reactivation mediates this link. (b) In the unrewarded group the link between encoding and Day 3 memory strength was mediated by reactivation. (c) In the rewarded group, reactivation failed to mediate the link between encoding and day 3 memory strength. B weights (regression coefficients) are shown for each path in the model (in brackets is the path from encoding to Day 3 memory strength when the mediator is also included in the model).

Table 1

Mediation analysis- direct and indirect effects.

	Coef	S.E	t	p
Unrewarded Group
(A path) Encoding->Reactivation	1.125	0.145	7.738	<0.0001
(B path) Reactivation -> Day 3 memory strength	1.094	0.306	3.577	0.0060
(C path) Encoding -> Day 3 memory strength	1.041	0.207	5.02	0.0005
(C’ path)* Encoding -> Day 3 memory strength	−0.189	0.371	−0.507	0.6238
Rewarded Group
(A path) Encoding->Reactivation	0.793	0.118	6.727	0.0001
(B path) Reactivation -> Day 3 memory strength	0.601	0.224	2.685	0.025
(C path) Encoding -> Day 3 memory strength	0.87	0.106	8.187	<0.0001
(C’ path)* Encoding -> Day 3 memory strength	0.393	0.196	2.004	0.076

*Effect of encoding on Day 3 memory strength, when controlling for reactivation.

Table 2

Mediation analysis- bootstrap results for indirect effects.

	Bootstrap results				Bias-corrected CI
	Data	Boot	Bias	S.E	Lower	Upper
Unrewarded Group*	1.230	1.284	0.054	0.444	0.241	2.849
Rewarded Group	0.476	0.469	−0.007	0.183	−0.23	1.01

*Significant effect (CI is above 0).

To separate between the effects of reactivation and the effects of reward per se and to control for possible intervening effects stemming from the timing of reinforcement, we ran a second experiment in which subjects (n = 12) underwent the same experimental procedures, wherein performance on the reactivation trial was reinforced with a fixed delay of 6 h, a period which may surpass the time-window during which memory becomes susceptible to further modifications following reactivation10. In this delayed reward condition reactivation strength significantly correlated with Day 3 memory strength (r = 0.678, p = 0.015; Fig. 4), in a way which did not differ from the unrewarded group (Z = 0.23, ns). Additionally, retention did not differ from that of the unrewarded group (t22 = 1.527, ns). Overall, these results suggest that reinforced memory reactivation disrupts subsequent memory strength.

Figure 4

Relation between the strength of memory reactivation and Day 3 memory strength in delayed reward conditions.

Reactivation strength significantly correlated with Day 3 memory strength (r = 0.678, p = 0.015).

Discussion

The goal of this study was to identify the role of reinforcement in reactivated skill memory. As reward has been shown to mediate the encoding of information in both the procedural and declarative memory systems11121314151617, we reasoned that reinforcing an already encoded procedural memory following reactivation would impact strengthening of the memory trace and facilitate retention. We report however that reactivation with reward did not strengthen memory, relative to reactivation with no reward. Moreover, the relation between the strength of memory reactivation and subsequent memory as found in reward-free reactivation, was reduced for reward-based reactivation. In addition, statistical modeling revealed that unrewarded, as opposed to rewarded reactivation, indirectly mediated the link between encoding and subsequent memory strength.

A viable framework for interpreting the current results is that of competition between memory traces and memory systems. It is by now well accepted that various tasks invoke competition between memory systems that rely on dissociable brain networks and distinct computational processes17232425. Competitive memory dynamics can also be formed between memory traces, and may specifically originate from reinforcement mechanisms. For instance, encoding of reward associated items interferes with the encoding of non-reinforced mnemonic representations25. Moreover, in episodic memory, competitive dynamics between memory encoding and learning from reward has been documented17, consistent with differential engagement of the medial temporal lobes and the striatum during learning25. Thus, a feasible framework for interpreting the disruptive effect of reinforcement following reactivation on subsequent memory strength is that the introduction of reward following reactivation may have resulted in a competition with the original encoded trace, which was averted when the memory trace was replayed with no reward. This framework is consistent with findings in Pavlovian learning demonstrating retroactive interference of new memories on reinstated memories262728, or modification of reactivated memories through counterconditioning with new reinforcers293031.

An alternative but related explanation on the unfavorable effects of reinforcement following reactivation on subsequent memory strength is based upon a prediction error mechanism. Prediction errors are believed to be a prerequisite for memory destabilization and reconsolidation323334. However, it was recently suggested that memories can be weakened when they are mispredicted by the context, which was originally associated with these memories35. The current results may reflect a related prediction error mechanism, whereby the addition of reinforcement during the replay of procedural memory generates a prediction error which subsequently weakens memory. This prediction error is absent when the memory trace is replayed with no reward, leading to superior subsequent memory. Thus, in this respect reward may generate a previously unencountered context which ultimately weakens the memory trace. Future research should take into account that a combination of memory competition and prediction error mechanisms may underlie the disruptive effect reward has on the reactivated skill memory, both generating an impairment in subsequent memory strength. Our results further indicate that delaying the receipt of reward after the reactivation trial results in an intact significant relation between reactivation strength and subsequent memory strength, comparable to unrewarded reactivation. Thus, our results point to time dependent effects of reward following reactivation, on subsequent memory strength.

Taken together, these results open interesting avenues for future research. First, it remains to be tested if extrinsic modulation of reward systems, whether by means of pharmacological interventions3637, or using non-invasive brain stimulation38 exerts a similar influence on memory reactivation and subsequent memory strength. This will allow to further uncover the role of dopaminergic neuromodulation during memory reactivation. Second, the putative competitive dynamics between memory traces and their underlying neural underpinning could be further tested using brain imaging techniques, suitable for probing complex information representations, such as multivariate pattern classification analysis3940.

The notion that existing memories can be modified with external interventions has far reaching clinical implications, as such interventions can be employed, for instance to disrupt maladaptive memories after post-traumatic stress or to reduce drug craving in addiction41. The current results demonstrate the contextual specificity required for these interventions to as found and point to the need for additional studies, to further delineate the role of memory reactivation in shaping subsequent memory strength.

Materials and Methods

Subjects

A total of thirty-six right-handed healthy subjects (13 men, 23 women; mean age 24.8 ± 2.2 standard deviation) participated in the study. All subjects gave their written informed consent, approved by Tel Aviv University’s Ethics committee. All procedures were in accordance with approved guidelines. Musicians (in the past or present) were excluded from participating in the study. We have additionally required at least 6 h of sleep prior to each experimental session.

Task

During the experiment subjects were asked to perform a sequential finger-tapping task18192042. Each trial in the task lasted 30 sec, during which subjects had to repeatedly tap with their left non-dominant hand a 5-element sequence of finger movements as quickly and accurately as possible (the sequence was 4-1-3-2-4, whereby ‘1’, ‘2’, ‘3’ and ‘4’ correspond to tapping of the index, middle, ring and little fingers respectively). Tapping was performed on a 4-key response box (Cedrus, Lumina, Model LU440), placed in front of subjects during the experiment. Performance in the task was quantified in terms of the number of correct sequences tapped during each trial181942. The same sequence was used in all experiments and sessions. Throughout each trial, each key press produced a dot displayed at the top portion of the screen, with the dots accumulating from left to right as the trial progressed. Trials were separated by 30 s breaks.

Experimental procedure

The first experiment comprised three sessions, administered on three consecutive days. In the first session (Day 1) all subjects (n = 24) performed twelve trials of the sequential finger-tapping task. On the next session (Day 2), which was administered 24 hours later, participants in the main experiment were equally divided into two groups. The first, rewarded group, performed one reactivation trial whereby each successful sequence within the trial was reinforced with monetary reward. Subjects were explicitly told that they will be reinforced at the end of the trial. The total reward earned in the trial was displayed on the screen right after the completion of the trial (indicating: “you won X Shekels!” with X being the amount of Israeli Shekels that is equal to the number of total correct sequences performed in the trial). Instructions given to this group of subjects prior to the task explicitly indicated that they will be monetarily rewarded with 1 Israeli Shekel for each of the correct sequences they perform during the task. A second, unrewarded group, performed one reactivation trial with no monetary reward, however this group of subjects received performance feedback at the end of the trial indicating how many successful sequences they were able to tap during the trial (“you tapped X correct sequences!”). Subjects were explicitly told that they will receive performance feedback at the end of the trial. This design enabled to tease apart monetary reward and simple performance feedback per se. Thus, both groups were administered a reactivation trial at Day 2, eliminating differences resulting from retrieval-induced forgetting4344. In the third session (Day 3) all participants performed three regular trials of the task, with no performance or reward feedback. In a second experiment, subjects (n = 12) were reinforced in accordance with their performance in the reactivation trial, similar to the rewarded group. However in this experiment reward feedback was provided 6 hours after the completion of the reactivation trial (in the same way as the reward group). Instructions in this experiment indicated to the subjects that they will “receive feedback” about their performance 6 hours after the reactivation trial (i.e., the subjects were not aware that their performance will be reinforced with monetary reward).

Data analysis

To test for group differences in encoding, a repeated measures analysis of variance (ANOVA) was performed with group serving as the between-subjects factor, and the first and last three trials of Day 1 as repeated measures. The ANOVA was preceded by Mauchly test of sphericity, to confirm that the assumption of sphericity was not violated. Retention was defined as the difference in performance between Day 3 and the last 3 trials of Day 1. As in previous studies using the same task, to better characterize memory strength by minimizing motor fatigue-related decrements in performance, the best two trials were considered for Day 1 post-training and for Day 3 memory strength4546. Two tailed tests were used in all analyses. The relationship between reactivation strength (defined as the difference in performance between Day 2 reactivation and Day 1 post-training), and Day 3 memory strength was tested with a Pearson’s correlation. We have additionally assessed the robustness of these correlations against outliers using the percentage-bend correlation technique47, as implemented in the “Robust Correlation Toolbox” in MATLAB48. Group differences in the strength of correlation were tested with a Fisher’s r-to-z transformation.

Mediation effects were tested using regression analysis and bias-corrected confidence estimates. First, simple linear regression analyses between each component in the mediation model were performed, including the effect of Day 1 encoding on Day 2 reactivation (A path), Day 2 reactivation on Day 3 subsequent memory strength (B path), Day 1 encoding on Day 3 memory strength (C path), and encoding on Day 3 memory strength when the putative mediator (Day 2 reactivation) is also in the model (the C’ path). In this model, the indirect effect estimates the degree to which encoding exerts an indirect effect on Day 3 memory strength through reactivation (the mediator). Mediation effects (the indirect path) were tested using bootstrapping with bias-corrected confidence estimates2149, defining confidence interval (99% to account for multiple comparisons) with 5000 bootstrap resamples22. Confidence intervals that included the value 0 indicated that the null hypothesis (no mediation effects) could not be rejected whereas intervals that did not include 0 indicated that the null hypothesis should be rejected.

Additional Information

How to cite this article: Dayan, E. et al. Reward disrupts reactivated human skill memory. Sci. Rep. 6, 28270; doi: 10.1038/srep28270 (2016).