Reactivation, retrieval, replay and reconsolidation in and out of sleep: connecting the dots.

The neurobiology of memory has taken on a new look over the past decade. Re-discovery of cue-dependent amnesia, wide availability of functional imaging tools and increased dialog among clinicians, cognitive psychologists, behavioral neuroscientists, and neurobiologists have provided impetus for the search for new paradigms for the study of memory. Memory is increasingly viewed as an open-ended process, with retrieval being recognized as an intricate part of the encoding process. New memories are always made on the background of past experience, so that every consolidation is, in fact reconsolidation, serving to update and strengthen memories after retrieval. Spontaneous reactivation of memory circuits occurs during sleep and there is converging evidence from rodent and human studies that this is an important part of the extended off-line memory processing. The noradrenergic neuromodulatory system is engaged at retrieval, facilitating recall. The noradrenergic system is also activated during sleep after learning and noradrenergic neurons fire in concert with cortical oscillations that are associated with reactivation of memory circuits. We suggest that the noradrenergic system and perhaps other neuromodulatory systems, may be a key to linking off-line memory reactivation, retrieval, and memory reconsolidation processes at both synaptic and systems levels, in and out of sleep.

Some Background on Reconsolidation

A decade of memory reconsolidation is really four decades. The story begins in the 1960s when post-retrieval amnesia was first revealed by Donald Lewis. He showed that temporally graded retrograde amnesia could be obtained for a well-established memory in rats, if that memory were activated by a reminder of the original learning experience. (Lewis, 1979, for review). These studies are the real origin of today's “reconsolidation” hypothesis, even though neither Lewis, nor his colleagues, ever used the term reconsolidation. And with good reason. Cue-dependent amnesia is not predicted by the consolidation hypothesis and is, in fact, in direct contradiction to it. Therefore, rather then superimposing a new hypothetical process on top of the already contested consolidation hypothesis, Lewis preferred a phenomenal description, “cue-dependent amnesia,” to account for his results. It was assumed that the cue associated with the learning, as a reminder, activated the memory. Its susceptibility to the amnestic effects of electroconvulsive shock (ECS) suggested that the active memory was labile, as it was immediately after acquisition (Lewis et al., 1972). In the first series of experiments from the Lewis laboratory, thirsty rats learned to lick a drinking spout; when this behavior was well established, a tone, followed by a footshock was presented during the ongoing licking behavior. Subsequent presentations of the tone alone elicited suppression of licking. A day after training, when memory expression was robust and reliable in control rats, the tone was presented alone, followed by ECS, a treatment that produces amnesia when administered after learning. Those rats that were “reminded” before ECS, showed a significant behavioral deficit when tested the following day. ECS in absence of the cue had no effect on subsequent behavior (Misanin et al., 1968). Cue-dependent amnesia could likewise be induced by protein synthesis inhibition in much the same way that newly acquired memories are. Judge and Quartermain (1982) trained mice on the conditioned lick suppression task used by Lewis. The protein synthesis inhibitor anisomycin was injected systemically at different time intervals after a single memory reactivation, consisting of a brief exposure to the training context. There was a clear renewed efficacy of the treatment after reactivation, although the temporal gradient was steeper than for that generated after initial learning. More recent studies have shown that injections of anisomycin directly into the amygdala after reactivation can produce amnesia for a simplified version of the conditioned suppression task (Nader et al., 2000).

Further experiments from the Lewis laboratory showed that the phenomenon of cue-dependent amnesia was not limited to aversive Pavlovian conditioning protocols. Rats were trained in a complex maze over several days using food reward as the incentive. Cue-dependent amnesia could be obtained after presentation of different types of reminders associated with the initial task (Lewis and Bregman, 1973; see Sara, 2000a,b, 2008, for a detailed account of these early studies).

We used a modified version of the same task in several experiments to show cue-dependent improvement in memory after a long retention interval. Contextual cue reminder alone Deweer et al., 1980) or in combination with pharmacological treatment (Sara and Deweer, 1982) or electrical stimulation of the mesencephalic reticular formation (Sara et al., 1980) in the presence of the contextual cue reminder facilitated retrieval. Although these experiments lent support to the notion that the cue activated the memory, they unfortunately, did not address the question of “reconsolidation,” since the rats were tested right after presentation of the cue.

Lewis’ studies clearly demonstrated that memory lability was not time-bound to acquisition, as the consolidation hypothesis holds. Norman Spear published a series of monographs around this time providing a strong theoretical framework for retrieval induced reactivation (see below; Spear, 1973; Spear and Mueller, 1984). Nevertheless, the term “to consolidate” was not used to account for cue-dependent amnesia until 1997, when a serendipitous observation of a delayed amnesic effect of a drug on a well-trained spatial memory led to a rekindling of interest in cue-dependent amnesia in our laboratory. Remembering the work of Lewis, two decades earlier, we pursued the theme with a series of experiments examining amnesic effects of blockade of NMDA receptors after reactivation of different types of robust memory (Przybyslawski and Sara, 1997; Torras-Garcia et al., 2005). A well-consolidated spatial memory, acquired over many days, reactivated by a single errorless trial, was dependent upon NMDA receptors to maintain stability. The memory deficit was robust, in that there was no spontaneous recovery 48 h later (Przybyslawski and Sara, 1997, Experiment 4). However, drug-treated rats could relearn the task in only a few trials, suggesting only partial amnesia (Przybyslawski and Sara, 1997, Experiment 3). We later showed that cue-dependent amnesia could also be obtained by blockade of noradrenergic beta receptors at reactivation of memories for both appetitive and fear-driven tasks (Roullet and Sara, 1998; Przybyslawski et al., 1999; see also Debiec and Ledoux, 2006). The following decade saw a proliferation of studies by many laboratories implicating protein synthesis, specific neurotransmitters, intracellular signaling pathways in reconsolidation process, and defining behavioral “boundaries” for obtaining a reconsolidation effect. After a decade of renewed interest and intensive investigation, reconsolidation remains, as does consolidation for that matter, a hypothetical process, for the most part inferred through cue-dependent amnesia (see Dudai and Eisenberg, 2004; Tronson and Taylor, 2007; Alberini, 2008; Sara, 2008; Nader and Hardt, 2009; Nader and Einarsson, 2010 for reviews with different perspectives).

Sleep, Memory Reactivation, and Consolidation

The notion that off-line memory processing occurs during sleep has been around for a long time. There was a surge of animal studies in the 1960s and 1970s with a focus on the rapid eye movement (REM) stage. It was hypothesized that the high level of cortical activity associated with this sleep stage is associated with reactiviation of memory traces formed during wakefulness and that this stage of sleep serves to consolidate these memories (Hennevin and Leconte, 1971). The hypothesis was tested by selectively depriving rats of REM sleep or delaying its onset after learning (Fishbein, 1971; Hennevin and Leconte, 1971) although these early studies did not adequately control for the stressful effects of the REM deprivation. Another approach involved monitoring and quantifying the proportion of REM sleep after learning. Learning-dependent REM increases were found and were correlated with the rate of acquisition of a task over several sessions (Hennevin et al., 1974). To further test the hypothesis that a reactivated memory trace undergoes a consolidation process during REM sleep, a contextual cue reminder, associated with the learning situation, was administered during the REM sleep episodes following learning. The idea was to activate specific networks that had been active during the encoding phase. Rats reminded during REM sleep expressed better memory when tested the following day, compared to rats that received the same “reminder” treatment during the waking state (Hars et al., 1985). These studies were carried out within a conceptual framework of Lewis, the idea being that a specific memory was activated by the cue, so as to become labile. The cortical activity associated with REM would promote consolidation of the active network.

The discovery of spontaneous “replay” of neural ensembles in rats provided a strong impetus for renewed studies of the role of sleep in memory consolidation. Neurons that are entrained to fire together during awake behavior, tend to fire together again during sleep (Pavlides and Winson, 1989; Wilson and McNaughton, 1994). Replay has been taken to represent a reactivation of circuits underlying the previously encoded information (the memory trace) and reinforces the notion that consolidation of memory occurs during sleep. The initial hypothesis, discussed above, focused on the REM stage, because of the high level of cortical activity. However most replay in rats seems to take place during non-REM sleep, although there are some exceptions (e.g., Louie and Wilson, 2001).

Replay occurs in strong temporal relation to high frequency oscillations in the hippocampus, called sharpwave/ripple complexes (Kudrimoti et al., 1999; Csicsvari et al., 2000; O'Neill et al., 2010, for review). Others have proposed that these sleep-associated high frequency oscillations provide the substrate to promote long-term plasticity and consolidation of the memory trace. Synaptic connections between cells in the reactivated network will be reinforced by the high frequency concerted firing (Buzsaki, 1989; Steriade and Timofeev, 2003). Recording hippocampal local field potentials in the rat, we found a marked increase in the occurrence of ripples in the sleep session immediately following learning of an odor-reward association. Important from the reconsolidation perspective, there was a similar increase in ripples in well-trained rats that had been exposed to a reminder session, just before. Rats that did not learn the task had no increase in ripples (Eschenko et al., 2008). In a multi-session spatial discrimination task, an increase in ripples predicted an increase in memory performance in the following session (Ramadan et al., 2009). These increases in ripple activity, if they are indeed, accompanied by replay of learning related ensembles of neurons, will promote memory consolidation, or might even be viewed as a manifestation of a consolidation process.

Until recently, there have been no strong data linking the content of the replay with the subsequent expression of memory during wakefulness. One study attempted to drive replay of specific networks by presenting a non-waking contextual cue reminder during sleep. (Rasch et al., 2007). Those subjects “reminded” by the contextual cue during sleep, showed better memory for the initial material presented within the context, when tested the next day. This experiment is reminiscent of the earlier study in rats, cited above. In both experiments, the authors assumed that the reminder treatment caused a specific reactivation of memory circuits to be reinforced during the sleep episode.

Two other studies have provided convincing new data relating the reactivation to subsequent memory performance. In one, using in human subjects, the content of dreams was related to previous waking experience and subsequent memory performance (Wamsley et al., 2010). An analogous experiment in rats recorded ensembles of hippocampal neurons during learning, monitored their reactivation during hippocampal ripple activity, and then related this activity to subsequent performance on a memory test (Dupret et al., 2010). These very recent reports take us a step further in relating encoding of memory in ensemble activity, off-line reactivation of the memory trace and subsequent robust memory performance.

Retrieval

Since reactivation seems to be a key to memory consolidation and reconsolidation in and out of sleep, it is essential to understand what governs this reactivation, what brain regions are involved, and whether neuronal assemblies are actually reactivated at retrieval. James (1890) stated a truism in that the study of memory is first and foremost a study of retrieval, since without recall, there is no proof that memory exists. Nevertheless, the past decades have seen relatively little literature dealing with neurobiological mechanisms of memory reactivation or retrieval (Sara, 2000a). A notable exception is the work from Miyashita's laboratory involving an electrophysiological approach combined with lesions in monkeys (Miyashita, 1993). From their unit recordings they are able to distinguish two types of retrieval. Automatic or bottom up, depends on the inferior temporal cortex, where a cue reactivates the neural representation of the memory. “Active” or top down retrieval is effortful and depends upon the prefrontal cortex input to the inferior temporal lobe. (Miyashita and Hayashi, 2000; Osada et al., 2008, for review).

Many imaging studies in humans have confirmed the crucial role played by the frontal regions in executive control of effortful memory retrieval (review Wheeler and Buckner, 2004). Imaging approaches have been able to identify other brain regions that are activated during retrieval from different types of remote episodic memory, implicating among others, hippocampus and amygdala (Spiers and Maguire, 2007). In addition, the parietal cortex has now been identified as a major component of the retrieval network, especially when the retrieval is effortful and is initiated top down. Furthermore, a recent study demonstrates that co-activation of left medial temporal lobe regions and temporal–parietal cortices is greater for correct than incorrect retrieval of episodic memory. (Mendelsohn et al., 2010). Earlier, the amygdala and the noradrenergic nucleus locus coeruleus were also identified as key players in retrieval, specifically of emotional memories. At retrieval these two structures were more tightly coupled during correct responses. In this particular study, no other brain regions that were activated by the retrieval effort showed this increased functional connectivity (Sterpenich et al., 2006).

Noradrenergic System in Retrieval and Reconsolidation

The demonstration of the engagement of the noradrenergic nucleus locus coeruleus in memory retrieval in humans corroborates previous studies in rats concerning the role of this system in memory retrieval. Electrical stimulation of locus coeruleus neurons or pharmacological increase in availability of forebrain NE can facilitate retrieval from very remote memory in rats, if, and only if, it is applied in the context in which the training took place. As with the cue-dependent amnesia studies, the memory must be activated by a cue, in this case the context, in order that stimulation of the noradrenergic system facilitate retrieval (Sara and Devauges, 1988, 1989; Devauges and Sara, 1991). This tiny pontine nucleus is the sole source of noradrenaline to the forebrain and it projects to all cortical regions and thalamic nuclei, having a strong influence on processing sensory information in all modalities (Sara, 2009 for review). In addition there is a substantial noradrenergic projection to hippocampus where it facilitates synaptic transmission and cellular excitability (Harley, 2007). In the frontal cortex, noradrenergic input modulates working memory and focuses attention (Robbins and Arnsten, 2009). While the mediating mechanism of retrieval facilitation by activation of the LC is unknown, any or all of the aforementioned effects should be involved. It is worth emphasizing that all of the key regions revealed by imaging studies in humans shown to be important in retrieval, receive substantial noradrenergic input from the LC.

In addition to its involvement in retrieval processes, the LC–NE system is engaged in post-acquisition consolidation and post-retrieval reconsolidation. A large number of pharmacological studies have established that the noradrenergic input to the amygdala is essential to memory consolidation after initial training (see McGaugh and Roozendaal, 2008 for review). Beta adrenergic receptors are necessary for reconsolidation after reactivation of well-established memories of varying emotional valences. (Przybyslawski and Sara, 1997; Przybyslawski et al., 1999; Debiec and Ledoux, 2006). Those experiments involved systemic injections, precluding a precise determination of the timing of the drug efficacy. In later studies, intraventricular and intra cortical injections revealed a relatively narrow time window, around 2 h after training, when injections were effective. Moreover, amnesia was obtained several weeks after initial training, if memory was reactivated by exposure to a training-associated cue 2 h before the injection (Tronel et al., 2004).

The late involvement of the noradrenergic system in memory processing led us to consider the behavioral state of the animal, at this time window. They were in their home cages, for the most part sleeping. Recording activity of LC neurons while monitoring the sleep state through cortical electrodes, we found that LC neurons increase their firing rate during non-REM sleep after a learning episode and this increase occurs at around 2 h after learning (Eschenko and Sara, 2008). Moreover, we discovered that LC unit activity is time locked to cortical slow oscillations during this phase of sleep. The slow oscillations serve to group faster rhythms such as cortical spindles and hippocampal ripples and therefore provide the background brain state for consolidation (Siapas and Wilson, 1998; Molle et al., 2006). As mentioned in the preceding section, it is during this non-REM sleep-associated ripple activity that the ensemble replay occurs. If indeed, there is potentiation of synapses occurring during this high frequency concerted firing of activate memory networks, then release of NE at a critical period during the replay should reinforce the plasticity in those activated synapses (Harley, 2007). While this scenario is presently highly speculative, it merits further investigation.

Only a few investigators have attempted to integrate the sleep research with current ideas concerning reconsolidation (Sara and Hars, 2006; Stickgold and Walker, 2007). The noradrenergic system and perhaps other neuromodulatory systems, may, indeed, be a key to linking off-line memory reactivation, retrieval, and memory reconsolidation processes at both synaptic and systems levels, in and out of sleep.

Conclusions and Perspectives

The view presented here is that memory is a dynamic property of the nervous system, in constant flux as a result of being retrieved within current cognitive environments. Such a view can be found in many earlier accounts of memory, and in particular, permeates the writings of James (1890); Tulving and Thompson (1973) argued that remembering is an activity similar to perceiving, in the sense that it involves the apprehension and comprehension of contemporary stimuli in the light of past experience. Accordingly, new episodic memory, to be remembered in a meaningful way, must be consolidated within a preexisting semantic memory. An updated version of Tulving's view can be found in a recent extensive review of the literature on the neural mechanisms of source memory. “Both encoding and remembering are constructive and reconstructive; they are selective and influenced by a rememberer's knowledge, beliefs, biases, goals, agendas, and meta-memory assumptions active at the time”(Mitchell and Johnson, 2009, p. 4). In this view, there is no clear demarcation between consolidation and retrieval. Encoding is preceded by retrieval, so that subsequent off-line consolidation, in and out of sleep, is really reconsolidation.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.