The circadian clock, reward, and memory.

During our daily activities, we experience variations in our cognitive performance, which is often accompanied by cravings for small rewards, such as consuming coffee or chocolate. This indicates that the time of day, cognitive performance, and reward may be related to one another. This review will summarize data that describe the influence of the circadian clock on addiction and mood-related behavior and put the data into perspective in relation to memory processes.

Introduction

The circadian system has evolved to align an organism’s behavior and physiology with the earth’s geophysical time. As a consequence, anabolic and catabolic pathways are allocated to specific time periods over the 24-h of a day. Hence, the circadian system structures an organism’s biochemistry and prepares it for daily recurring events. Because the environmental light/dark cycle changes over the year, the circadian system needs to be able to adapt to such seasonal changes to maintain an alignment between external and internal times. This adaptation allows organisms to predict the sunrise and/or sunset and the availability of food.

During the evolution of life on earth, food has always been one of the limiting parameters for the spread and growth of a species. Using the predictive value of the internal body clock in combination with the drive to eat allows animals to remember place and time for a particular food source. Therefore, the clock and memory seem to be important pillars for survival. A variety of studies suggest that learning and memory, as well as reward processes, are sensitive to disruptions of sleep and circadian rhythms (Dijk et al., 1992; Peigneux et al., 2004; Ellenbogen et al., 2006; Wright et al., 2006; Ruby et al., 2008). Interestingly, most clock genes are expressed in brain areas that are involved in learning, memory, and reward, such as the amygdala (Lamont et al., 2005), the hippocampus (Albrecht et al., 1997; Wakamatsu et al., 2001; Chaudhury et al., 2005; Jilg et al., 2010), and the ventral tegmental area (VTA; McClung et al., 2005; Hampp et al., 2008). Mice with mutated circadian clock genes displayed altered performance in a variety of learned behavioral tasks, such as cued and contextual memory (Garcia et al., 2000), drug sensitization and seeking (Abarca et al., 2002; Spanagel et al., 2005), time–place learning (Van Der Zee et al., 2008), and aspects of spatial learning (Jilg et al., 2010; but see also Zueger et al., 2006). However, the mechanism by which the clock integrates memory and reward has not been thoroughly elucidated.

The Reward System and Clock Genes

The reward system is composed of brain structures that regulate and control behavior by inducing pleasurable and aversive effects. A reward, when presented more than once, causes an increase in the intensity of the associated behavior, which is termed reinforcement. Primary rewards are those necessary for survival, such as food, water, and sex. Secondary rewards derive their value from the primary rewards and include pleasant touch, music, and money (the latter 2, presumably and primarily, are for humans only). Rewards (positive or negative) modify behavior and emotions, induce learning and influence mood. For example, drugs of abuse, such as alcohol and cocaine, which positively influence the reward system, improve subjective well-being and encourage repetitive drug use that eventually leads to addiction. In contrast, hunger or pain induce searching behavior or the avoidance of particular circumstances, respectively.

Clock genes are expressed in many brain areas that are involved in the reward system, including the VTA, the prefrontal cortex (PFC), the amygdala (AMY), and the nucleus accumbens (NAc). In these brain structures, clock genes are expressed in an oscillating manner with a period of 24-h. Cycling of expression in an individual brain structure can be out of phase with the cycling in another structure. Importantly, however, a specific phase-relationship between various cycling brain areas is maintained (reviewed in Guilding and Piggins, 2007). This is essential to generate a synchronized systemic output of behavior. The oscillations in most brain areas outside the suprachiasmatic nuclei (SCN) appear not to be self-sustaining, which indicates that they are most likely dependent on input signals from SCN, the pacemakers of the circadian system. Therefore, the brain structures that constitute the reward system appear to be SCN-driven clocks. However, these subordinate rhythmic structures are sensitive to systemic signals, such as hormones, metabolites, temperature, and feeding. These factors can act as synchronizers of the various subordinate brain clocks, ensuring the harmonic orchestration of brain function (Joels and De Kloet, 1994; Lamont et al., 2005). Interestingly, the SCN exhibit low sensitivity to these synchronizers because in contrast to the subordinate brain and peripheral clocks, the cellular SCN clocks are strongly coupled to each other by various mechanisms, and thus neuronal network properties are integral to their synchronization. Clock gene expression within the SCN is insensitive to glucocorticoids (Balsalobre et al., 2000), melatonin (Poirel et al., 2003), temperature (Buhr et al., 2010), food entrainment (Stokkan et al., 2001), and its own paracrine signal prokineticin 2 (Li et al., 2006). However, under specific conditions, the SCN may be affected by food and melatonin (Caldelas et al., 2005). Therefore, alterations of these synchronizer signals by stress or drugs will primarily affect subordinate brain clocks but will barely affect the SCN. Therefore, subordinate brain oscillators may receive signals from the SCN that are in conflict with the synchronizer signals. This phenomenon may lead to a de-synchronization of timed brain functions and eventually cause changes in behavior, including addictive behavior (Uz et al., 2005; Li et al., 2009).

Neuropsychiatric disorders are often associated with an increased risk of drug abuse (Brown, 2005). The mania-like behavior phenotype of Clock mutant mice (Roybal et al., 2007) and their sensitivity to drugs of abuse illustrate this co-occurrence in animals (McClung et al., 2005). The relationship between clock genes and cocaine was first observed in Drosophila (Andretic et al., 1999) and was later extended to mice. Animals with mutated circadian clocks show altered responses to various drugs of abuse, including alcohol, cocaine, methamphetamine, and morphine. A hypersensitized response to cocaine is observed in Clock and Per2 mutant mice (Abarca et al., 2002; McClung et al., 2005), whereas the loss of Per1 leads to a lack of cocaine sensitization in the behavioral responses of mice (Abarca et al., 2002). This opposite effect of Per1 and Per2 genes on the behavioral response of mice to cocaine is paralleled by a behavioral response to light (Albrecht et al., 2001). The similarity of the behavioral response to cocaine to that of light indicates the existence of common aspects of the cocaine-sensitization and light-signaling pathways. This idea is further supported by the findings that the chronic administration of drugs of abuse can directly induce Per1 and Per2 gene expression within the striatum (Yuferov et al., 2003; Uz et al., 2005), the nucleus accumbens (McClung and Nestler, 2003), and the hippocampus (Uz et al., 2005), which parallels the light inducibility of Per1 and Per2 gene expression in the SCN (Albrecht et al., 1997; Yan and Silver, 2004).

The SCN appear to influence, to a certain extent, the diurnal regulation of cocaine reward-related behavior (Sleipness et al., 2007a), and this mechanism may involve dopaminergic transmission in the mesolimbic pathway (Sleipness et al., 2007b). In this pathway, the VTA sends dopaminergic projections to the NAc. The clock may modulate dopamine levels in the VTA and NAc via transcriptional regulation of monoamine oxidase A (Hampp et al., 2008), an enzyme involved in the degradation of monoamines including dopamine. Taken together, clock genes may influence the function of the mesolimbic dopaminergic system and modulate mood-related behavior.

Monoaminergic neurons can be damaged in response to long-term light deprivation in the constant darkness (DD) paradigm, which induces depression-like behavior in rats (Gonzalez and Aston-Jones, 2008). Although extensive neuronal damage in the hippocampus has been observed in samples of depressed patients (Sapolsky, 2001; Sheline et al., 2003; McKinnon et al., 2009), there is evidence for neuronal cell death in brain regions that harbor monoaminergic neurotransmitter systems (Kitayama et al., 1994, 1997, 2008). One recent study also showed that long-term (4 weeks) exposure to DD in mice led to depression-like behavior and a reduction of cell proliferation in the dentate gyrus of the hippocampus (Monje et al., 2011), which indicates a regulatory role of light (or the absence of light) in hippocampal neurogenesis and mood state. This finding is associated with alterations in the levels of inflammatory parameters, such as interleukin-6 (IL-6) and clock proteins PER2 and NPAS2 (Monje et al., 2011). Interestingly, IL-6 can induce human Per1 gene expression in vitro (Motzkus et al., 2002), and it remains to be tested whether this property is also observed for Per2. However, the Per2 gene seems to be involved in the regulation of neurogenesis in the murine dentate gyrus of the hippocampus (Borgs et al., 2009). It appears that links between the circadian system, inflammatory processes, and apoptosis exist through the NF-κB signaling pathway (Lee and Sancar, 2011; Monje et al., 2011) to regulate neurophysiological processes (Monje et al., 2011). This finding is in accordance with a wealth of data that has described a relationship between the circadian system and inflammatory processes in human depression (Boivin, 2000; Anisman et al., 2005; Wirz-Justice, 2006; Dantzer et al., 2008).

In response to drugs of abuse, mesocorticolimbic dopaminergic activity leads to long-lasting plasticity in glutamatergic projections from the PFC to the GABAergic neurons in the NAc. These changes are thought to be important in the development of addiction (Kalivas, 2007). Interestingly, both extracellular glutamate and GABA display a circadian rhythm in which the highest levels are found at night (Castaneda et al., 2004). Moreover, the expression of the vesicular glutamate transporter 1 (vGLUT1) protein in synaptic vesicles displays a diurnal rhythm with high levels at the start of the light period that decline by noon, rise again at the start of the dark period and fall again at midnight (Yelamanchili et al., 2006). A lack of a functional PER2 protein significantly reduces rhythmic expression of the vGLUT1 protein, which suggests a role of this clock component in the regulation of glutamatergic vesicular sorting. A mutation of Per2 in mice leads to an increase in extracellular glutamate levels in the NAc, partially due to the reduced expression of astrocytic glutamate transporter 1 (Eaat1), which is normally responsible for the clearance of synaptic glutamate levels. The elevated glutamate levels that are found in Per2 mutant mice are accompanied by an increase in alcohol consumption (Spanagel et al., 2005). Therefore, Per2 plays not only an important role in the regulation of dopamine levels in the murine brain but also the regulation of glutamatergic transmission in a manner that is not yet understood.

Memory and the Clock

Past drug use often initiates craving for the drug and causes relapses (O’Brien et al., 1998). Therefore, it was predicted that adaptive forms of learning, such as long-term potentiation (LTP) and long-term depression (LTD), may contribute to addiction (Hyman and Malenka, 2001). Synaptic strengthening appears to be important in both learning and the development of addiction. It has been shown that the modulation of excitatory synapses of the mesolimbic dopaminergic system is important in the initiation and maintenance of addictive behavior in animals (Wolf, 1998; Carlezon and Nestler, 2002; Wolf et al., 2004; Hyman, 2005). These findings led to the conceptualization of addiction as a maladaptive form of learning and memory (Kelley, 2004; Saal and Malenka, 2006). Patients suffering from neuropsychiatric disorders often have problems in accessing memories of specific life events (reviewed in Dere et al., 2010). These memory defects involve the mesolimbic system and the hippocampus, which plays an important role in the consolidation of information from short-term to long-term memory.

Clock genes are expressed in the hippocampus, and several reports have highlighted the involvement of the circadian system in hippocampal-dependent memory function (reviewed in Eckel-Mahan and Storm, 2009; Gerstner et al., 2009). Lesions to the SCN or the induction of clock phase shifts by light (e.g., jet-lag) affect hippocampus-dependent long-term memory (Stephan and Kovacevic, 1978; Tapp and Holloway, 1981; Devan et al., 2001). Furthermore, LTP amplitude varies with the time of day in mice (Chaudhury et al., 2005), and mice with various clock gene mutations show defects in certain types of hippocampus-dependent memory formation (Garcia et al., 2000; Jilg et al., 2010; Kondratova et al., 2010), although water-maze experiments indicate no role of Per genes in spatial and contextual learning in standardized tests for hippocampus-dependent learning (Zueger et al., 2006). However, hippocampal circadian oscillations appear to be required for memory formation and persistence because local inhibition of the circadian rhythms of MAPK activity only within the hippocampus blocks long-term memory formation (Eckel-Mahan and Storm, 2009). Of note is that the MAPK signaling pathway is important for light-induced clock phase-shifting and addiction, which highlights again a convergence of clock, addiction, and learning pathways.

Neurogenesis is associated with hippocampal function (see also above its involvement in mood state). The birth of hippocampal neurons is increased after a learning task that involves the hippocampus (reviewed in Deng et al., 2010), and this process may also involve the clock gene Per2 (Borgs et al., 2009). Cell proliferation and neurogenesis appear to be inhibited as a result of experimental jet-lag in hamsters (Gibson et al., 2010). As a consequence, long-term deficits in hippocampal-dependent learning and memory are observed. This finding may be due to the activation of the HPA axis; however, jet-lag only transiently activates the stress axis, and repeated light exposure does not differentially impact on the cortisol levels between jet-lagged and control animals (Gibson et al., 2010). Furthermore, jet-lagged hamsters are not arrhythmic but do not entrain their activity to the light–dark (LD) cycle. This indicates that a de-synchrony between the internal and external time, rather than an absence of the animal’s circadian organization, is the cause of deficits in hippocampal-dependent learning in jet-lagged hamsters. This view is supported by the observation that Per2 is expressed in a constitutive manner in the dentate gyrus of mice, where this gene appears to be part of the intrinsic control of neuronal stem/progenitor cell proliferation, cell death, and neurogenesis (Borgs et al., 2009). Therefore, the effects of jet-lag and of Per2 on neurogenesis may not be directly related to the circadian clock but may be best described as non-clock functions of Per2. As Per2 is a clock component, any potential effect of the circadian clock on neurogenesis and cognitive performance would probably be indirect.

Conclusion

The circadian clock, the reward system, and memory processes appear to share certain nodal points (Figure 1). First, it appears that these systems all influence neurogenesis and neuronal cell death, which involves the NF-κB signaling pathway to regulate neurophysiological processes. Second, light acts on all three systems that employ the MAPK signaling pathway. These systems also seem to be affected by the HPA-axes via cortisol, thereby leading to short-term changes. Long-term changes, however, appear to involve additional mechanisms. Future experiments will show how the circadian clock, reward, and memory are linked at the molecular level.

Figure 1

Schematic diagramm of the potential integration of memory, reward, and circadian clock information. Signaling pathways affecting the learning/memory, reward, and circadian clock systems are depicted.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.