Circadian clocks and memory: time-place learning.

Time-Place learning (TPL) refers to the ability of animals to remember important events that vary in both time and place. This ability is thought to be functional to optimize resource localization and predator avoidance in a circadian changing environment. Various studies have indicated that animals use their circadian system for TPL. However, not much is known about this specific role of the circadian system in cognition. This review aims to put TPL in a broader context and to provide an overview of historical background, functional aspects, and future perspectives of TPL. Recent advances have increased our knowledge on establishing TPL in a laboratory setting, leading to the development of a behavioral paradigm demonstrating the circadian nature of TPL in mice. This has enabled the investigation of circadian clock components on a functional behavioral level. Circadian TPL (cTPL) was found to be Cry clock gene dependent, confirming the essential role of Cry genes in circadian rhythms. In contrast, preliminary results have shown that cTPL is independent of Per genes. Circadian system decline with aging predicts that cTPL is age sensitive, potentially qualifying TPL as a functional model for episodic memory and aging. The underlying neurobiological mechanism of TPL awaits further examination. Here we discuss some putative mechanisms.

Introduction

Time–place learning (TPL) refers to the ability to secure resources when they are available under specific temporal and spatial contingencies (Crystal, 2009). Many environmental aspects show circadian variation. Predators often establish hunting routes and many resources, like food and mates only become available on certain times of the day (Daan and Koene, 1981; Rijnsdorp et al., 1981; Silver and Bittman, 1984). This given, TPL is believed to be functional in optimizing resource localization and exploitation as well as predator avoidance in a circadian (predictable) changing environment, decreasing energy expenditure, and increasing survival chances. Although this is mainly a hypothetical explanation as to why animals possess TPL ability, an evolutionary relevance is strengthened by the fact that TPL has been shown in many species including bees (Gould, 1987), ants (Harrison and Breed, 1987), fish (Reebs, 1996), birds (Krebs and Biebach, 1989), rats (Boulos and Logothetis, 1990), and recently mice (Van der Zee et al., 2008).

The reason why animals are capable of TPL is, however, not the focus of this review, but rather what we can learn from this behavior in neuroscience. Evidence suggests that TPL can depend on the circadian system. This circadian TPL (cTPL) suggests a link between the circadian system and associative memory. However, not much is known about this connection.

The notion that time of day can be relevant in cognitive functions arose long before the concept of circadian clock systems was established. Beling published her first study on “Zeitgedachtnis,” time memory, studying sun compass orientation in honey bees (Beling, 1929). In 1950, Kramer did rather similar experiments in starlings. He showed that these birds also use an internal time of day mechanism to select the appropriate orientation relative to the position of an artificial sun (Kramer, 1950). These findings, of a functional internal timing in vertebrates, helped in the breakthrough of biological clock concepts (Aschoff, 1954; Pittendrigh et al., 1958). Since then we gained much insight into the physiology and molecular determinants of the circadian system itself and on the manifold behavioral and physiological processes under circadian control.

In mammals, a central nervous system based pacemaker, located in the suprachiasmatic nuclei (SCN), tracks light, and dark information and is connected with many peripheral, organ based oscillators (Dibner et al., 2010). In external 24 h light-dark (LD) cycles the circadian system results in entrained daily rhythms in the body. In constant external conditions, these rhythms show a free running pattern of about 24 h (circadian). The circadian sleep-wake cycle, regulation of hormonal, body temperature, and feeding rhythms are well-known examples. General cognitive- and memory performances have also been shown to vary over the circadian cycle (for review see Carrier and Monk, 2000). In line with this, disruption of circadian rhythms due to age, shift work, and shifts of the LD cycle (jet lag) have been associated with impairments of cognitive function (Fekete et al., 1985; Folkard et al., 1985a,b; Antoniadis et al., 2000; Devan et al., 2001; Biemans et al., 2003; Cain et al., 2004b; Craig and McDonald, 2008).

Besides governing innate rhythms in physiology and behavior, it is believed that circadian oscillators can provide phase information to brain systems involved in cognition, like memory, which allows time to be used in adaptive mechanisms (Enright, 1970, 1975; Gallistel, 1990; Mistlberger et al., 1996). It is believed that time of day information derived from an internal oscillator can be “stamped” in memory as a contextual feature to form associations with other contextual features and to be used in decision making processes. Such a mechanism can only function when a clock can be consulted continuously to record time-stamps, and to check whether coded time-stamps match the actual time of the day. This “continues consulted clock” function is thought to underlie cTPL as well as the adaptive time of day compensation in sun compass navigation demonstrated in several insect and bird species (Frisch, 1950; Kramer, 1950; Hoffmann, 1960; Keeton, 1974; Budzynski et al., 2000; Merlin et al., 2009).

Terminology

In order to avoid confusion, it is necessary to explain/define some terminology regarding TPL. TPL is also referred to in the literature as time-place discrimination or time-place association. Two types of TPL paradigms can be found in the literature: Interval- and daily TPL paradigms. Interval TPL paradigms are specifically designed to study interval timing (the ability to keep track of elapsed time). In such paradigms, animals typically receive a single test session each day (with multiple trials/location switches), and these sessions can occur at varying times of the day. For example, animals can learn to anticipate the switch when first food is provided only at location “A,” while after “X” amount of time food is provided only at location “B.” Rats can learn an interval time-place task with at least four different feeding locations and with unequal intervals between rewarding locations (Thorpe and Wilkie, 2002). In daily TPL paradigms, typically the location of a resource depends on the time of day, and animals are trained over multiple days with multiple sessions per day on fixed time-points, so that they learn to visit or avoid specific locations on specific times of the day.

In this review we focus on daily TPL. Thus, when we refer to TPL, more specifically we refer to daily TPL. In the literature, daily TPL is sometimes referred to as cTPL, however, here when we refer to cTPL we refer to daily TPL with the use of a circadian strategy. Animals may use multiple strategies to solve a daily TPL paradigm, as will be explained in the next paragraph.

Multiple possible strategies for TPL

We have mentioned that TPL can depend on the circadian system (in this case we refer to cTPL). This is only true if animals use a so called circadian strategy to master the paradigm. Alternatively, three non-cTPL strategies have been identified (Carr and Wilkie, 1997). These strategies will be explained here along with methods to identify these non-circadian strategies.

First, animals may use the so called contextual cue strategy. This is a non-timing strategy by which animals simply learn to visit or avoid location A in the presence of one (set of-) contextual cue(s), while they learn to visit/avoid location B in the presence of another (set of-) contextual cue(s). To exclude the possibility for animals to use this strategy, any discriminating contextual cues (differences between sessions) should be excluded by a proper research setup and practice.

Second, animals may use an ordinal strategy. In this case animals remember the sequence of (daily) events, e.g., first visit location A, then B. This strategy is also referred to as an alternation strategy and can also be viewed as the establishment of a (daily) route. Two variants of the ordinal strategy are identified: A timing- and a non-timing variant. In case of the timing variant, the sequence is reset daily, while this is not the case in the non-timing variant. The use of an ordinal strategy can be identified by skipping sessions. Note however that skipping the last session of a day will not identify an ordinal strategy when the timing variant is utilized. Animals will show normal location visits in the first session of the next day because the sequence has been reset. Thus, to identify the use of an ordinal strategy, without further distinction between the two variants, the first session(s) of a day need to be skipped. Instead of visiting the second session location, animals will visit the wrong (skipped) first session location in case an ordinal strategy is used.

Third, animals may use an interval timing strategy. In this case specific delay periods relative to one or more external cues are encoded as discriminating cues to know which locations to visit or avoid. For instance, animals may learn that short (or concrete after X amount of time) after lights-on, they should visit location A, while longer after lights on (or concrete after Y amount of time) they should visit location B. Because external cues can start/stop/reset timing, interval timing is also referred to as a stopwatch like mechanism. Interval timing is thought to be utilized to track intervals in a second-to-minutes range (Wilkie, 1995; Buhusi and Meck, 2005; Crystal, 2009), but some studies have suggested that intervals of several hours can be tracked (Pizzo and Crystal, 2006). According to the classical pacemaker accumulator theory, the circadian system may be implicated in interval timing as well. However, recent advances have challenged this view (Staddon and Higa, 1999; Wearden, 2004; Buhusi and Meck, 2005; Yin and Troger, 2011). Interval timing has been shown to be independent of the SCN and Cry1,2 clock-genes (Lewis et al., 2003; Papachristos et al., 2011) and thus seems to be independent of the circadian system. To rule out the possibility that animals use this strategy for TPL, any distinctive external cues should be ruled out from experimentation, for instance by testing under constant light conditions to eliminate LD transitions. Previous test sessions may also start/stop/reset interval timing (intersession interval timing). Together with the ordinal strategy, this possibility can easily be ruled out by performing session skips.

Only when the above non-circadian strategies have been experimentally ruled out, one can conclude that animals must use a circadian timing strategy. In this case TPL is based on an internal clock, independent of external cues. The endogenous nature of cTPL can further be shown by identifying known oscillator characteristics, like persistence in constant conditions, and known limitations imposed by the circadian system (Crystal, 2009).

Time memory: circadian retention paradigms vs. cTPL paradigms

Evidence that animals remember time of day is provided by behavioral paradigms that involve a training (stimulus encounter) followed by a retention test. In such experiments, animals show optimal retention when training and testing times match, indicating memory for the time of training. A basis for this knowledge was provided by Kamin who reported retention in a passive avoidance paradigm to be minimal one hour after training compared to the retention after 24 h, or even 19 days after training (Kamin, 1957). Why retention would be enhanced after a long interval compared to after a short interval was difficult to explain and the phenomenon, still known as the Kamin effect, has long been misinterpreted as a weak transition point in the processing of short-term memory into long term-memory. In 1973, Holloway and Wansley discovered that the Kamin effect was actually the result of a circadian periodicity in memory retention. They showed that, independent of the time of day at which training occurs, retention is always optimal 24 h later, or multiples thereof (Holloway and Wansley, 1973a,b; Wansley and Holloway, 1976). Next to passive avoidance, such periodic retention effects were also shown for active avoidance (Holloway and Wansley, 1973b; Cain et al., 2008), appetitive motivated learning (Wansley and Holloway, 1975), fear conditioning (Chaudhury and Colwell, 2002), conditioned place preference (Ralph et al., 2002; Valentinuzzi et al., 2008), and conditioned place avoidance (Cain et al., 2004a). However, it should be noted that not all mammals show this pattern (Oklejewicz et al., 2001).

The retention paradigms described above do not require animals to remember the time of training. Apparently animals remember time of day automatically. Therefore, Gallistel proposed that animals automatically encode time, together with place and the nature of biological significant events (Gallistel, 1990).

Although periodic retention deficits suggest time-memory, they do not necessarily implicate that animals learn an association between a stimulus and circadian phase. An alternative interpretation is that behavioral output (e.g., freezing) has been entrained by the stimulus pulse in the same way circadian rhythms can be entrained by other zeitgebers, such as LD transitions. However, when an animal is trained to go to different places at different times of day, as in TPL paradigms, more than entrainment of an oscillator must be involved: an association between time and place must have been learned (Biebach, 1989). In cTPL paradigms, memory of time is displayed directly by active choices. Animals are stimulated to remember the time because time discriminates between correct and incorrect choices. The correctness of a choice depends on correct memory and retention of time, based on previous encounters.

In summary, animals may use different strategies for TPL. The use of a circadian strategy is of special interest. cTPL presumes that an internal clock, and time derived from it, can be used by higher cognitive brain systems in adaptive experience based behavior. Correct location choices in cTPL implicate knowledge of current time of day (internal clock consultation), training times being stored as a contextual cue in memory (time-stamping, time-memory) and an association of these contextual time-points with spatial features and the nature of the event to guide behavior (decision making). These features make cTPL a unique circadian system dependent learning and memory ability and a tool to study the circadian system on a functional behavioral level. Next to providing a an up-to-date literature overview on TPL, this review will discuss the perspectives and functional aspects of (c)TPL.

A literature overview on time-place learning

From insects to birds to mammals

First TPL experiments were conducted studying honeybees. It was found that honeybees can be trained to collect food at any time of the day (Wahl, 1932), at multiple times (Koltermann, 1974), and at different places at different times (Finke, 1958). This plasticity implies that honeybees have a circadian oscillator at their disposal that allows for a continuous monitoring of the passage of time (Pittendrigh et al., 1958). Indeed, foraging behavior in bees was found to show underlying oscillator characteristics, like phase shifts, entrainment and limits of entrainment (Renner, 1959; Beier, 1968; Frisch and Aschoff, 1987), thus this internal oscillator could underlie TPL. Moore et al. showed evidence for cTPL in honeybees under laboratory conditions, thereby excluding a number of potential environmental cues which may have allowed for a contextual cue strategy, like circadian temperature-, humidity-, and light intensity fluctuations and position of the sun (Moore et al., 1989). But TPL in single, individually kept bees was never shown and also the use of ordinal- or interval timing strategies has never been ruled out in order to conclude cTPL (i.e., based on an internal clock).

Convincing evidence for cTPL came from experiments in birds. Field studies had already demonstrated anticipatory behavior regarding feeding schedules in kestrels (Falco tinnunculus). Individual kestrels showed an increased probability to hunt in the same area 24 h after having caught prey (Rijnsdorp et al., 1981; Wilkie et al., 1996). In a laboratory setting, Biebach and coworkers individually kept garden warblers (Sylvia borin) in a cage with four connected feeding rooms. The 12 h light phase was divided into four 3-h episodes in which entry to one of the feeding rooms was rewarded by food. Following an entry, all rooms were closed for 5 min, trapping the birds in the chosen location. This served as a punishment for wrong, non-rewarding location choices. Within ten days, the garden warblers made about 75% right (rewarding) choices. This pattern persisted when all rooms were rewarded and kept open, indicating time-place associated memory (Biebach, 1989). Conclusive evidence for cTPL was provided by Wenger and coworkers using starlings in a similar setup: in constant light conditions, the learned time and place associated feeding pattern was shown to free-run for almost a week, showing an important underlying oscillator property (Wenger et al., 1991).

Since Garden Warblers and starlings are both insectivorous birds, it was questioned whether the cTPL shown in these birds is specific for species that depend on a food source that shows daily fluctuations in availability. To test this idea, Falk investigated TPL in two related weaver bird species: an insectivore (Ploceus bicolor) and a granivore or seed-eater (Euplectes hordeaceus). Although the granivorous birds learned faster, they showed disrupted performance both after a session skip (all doors kept closed) and after a 6 h LD advance, suggesting that these birds used a non-cTPL strategy. In contrast, location visiting patterns of the insectivorous birds were unaffected by the session skip. After the 6 h LD advance these birds showed a phase advance in their feeding pattern that was less than 6 h, consistent with expectations based on an underlying circadian oscillator. In conclusion, these results suggest that not all species may have evolved cTPL ability.

TPL in mammals was first demonstrated using Long Evans rats, showing that rats can anticipate food at the correct location when it is made available at two different locations depending on the time of day (Boulos and Logothetis, 1990). It was known that SCN lesions, known to abolish LD entrainment and free-running circadian rhythms, did not interfere with food anticipatory rhythms or with the persistence of food anticipation during food deprivation (for review, see Boulos and Terman, 1980). Therefore, TPL was also investigated in SCN lesioned rats. These rats were able to master the task and this provided first evidence that TPL, like food anticipation, may be independent of the SCN (Boulos and Logothetis, 1990). Although the used strategy was not investigated, these findings were later confirmed by Mistlberger using male Wistar rats in a T-maze that were food deprived to 85–90% of ad libitum feeding weight. SCN intact- and SCN lesioned animals performed equally well. Moreover, session skips and an LD shift (day-night inversion) indicated that both groups used a circadian strategy. The authors suggested that an alternative internal oscillator, presumably a food entrainable oscillator (FEO), was used as a time source (Mistlberger et al., 1996).

Others had difficulties to show cTPL in rats. Carr and coworkers repeatedly found that Long Evans rats used an ordinal timing strategy, even when they tried to make an ordinal strategy unreliable by structural random session skips. They hypothesized that animals tested in a two sessions and two locations TPL setup may be more prone to adapt an ordinal strategy, because animals can simply alternate between locations (or learn to avoid always the previous rewarding location). Nonetheless, Long Evans rats also showed ordinal TPL in a three sessions and three locations TPL setup (Carr and Wilkie, 1997, 1999; Carr et al., 1999). Thorpe and coworkers also reported ordinal timing in this strain (Thorpe et al., 2003). Surprisingly, in multiple paradigms, Long Evans rats have also been reported not to show circadian retention deficits (McDonald et al., 2002; Cain et al., 2004c), indicating that this may be a strain specific deficit for time memory. On the other hand, Pizzo and coworkers found that Long Evans rats used primarily an interval timing strategy, but with evidence for a partial circadian strategy (Pizzo and Crystal, 2002). This indicates that these rats may not be unable to use a circadian strategy, but may less readily- or have more difficulty to do so.

Decisive factors for TPL

Lukoyanov and coworkers underlined the role of food deprivation in TPL. They investigated whether Wistar rats could show TPL in a non-food reinforced time-place task. They used a Morris water maze in which the platform changed location between morning and afternoon sessions. Surprisingly, rats only learned the task when they were food deprived, receiving 60% of the daily ad libitum food consumption. Ad libitum fed animals and animals that were only food deprived to receive 90% of daily ad libitum food consumption did not perform above chance level. Although the use of an ordinal- or interval timing strategy was not excluded experimentally, the authors suggested that food deprivation enables access to the necessary temporal information from the FEO (Lukoyanov et al., 2002).

Widman and colleagues broadened this view. They had shown earlier that an increased response cost can trigger TPL. Female Sprague Dawley rats showed TPL when they had to climb for food in a vertical maze, while rats tested in a horizontal maze did not. Moreover, the number of rats showing TPL increased as the height was increased (Widman et al., 2000). However, in this experiment the rats were still food deprived to 80% of ad libitum feeding weight. In a following experiment, the group of Widman used ad libitum fed male Sprague Dawley rats in a Morris water maze. Again, without food deprivation or an increased response cost, the rats did not show TPL, replicating the earlier findings of Lukoyanov et al. (2002). A second group of rats was also tested under ad libitum feeding conditions, but this time the response cost was increased by adding a weight belt to the rats. These rats did show TPL. Although the authors did not exclude the possibility that animals used an ordinal or interval timing strategy to solve the task, the authors argument that activation of a food system (FEO) is not specifically necessary for TPL. The authors suggested that food deprivation also forms an increased response cost (relative effort). The authors hypothesized that either the FEO or the SCN can be used as a consulted clock for TPL, and that access to-, or activation of these systems is facilitated by an increased response cost (Widman et al., 2004). In line with this, Aragona and coworkers showed that TPL could be facilitated in a two sessions and two locations lever pressing study, simply by placing a water bottle in between the levers (Aragona et al., 2002). This may also be explained by the response cost hypothesis proposed by Widman et al. because animals now had to walk around the water bottle if they had initially chosen a wrong (non-rewarding) lever (location). Thus, a high response cost seems to facilitate TPL, presumably because it stimulates animals to not make incorrect decisions and thus the need or motivation to encode the time-place contingency.

TPL has however also been demonstrated in experimental setups involving a low response cost and without food deprivation, using a palatable food resource (Means et al., 2000; Thorpe and Wilkie, 2007). This suggests that a low response cost may be partly compensated by an increased reward value. From another perspective, increasing the reward value (by using a palatable food reward or more severe food deprivation) may also indirectly increase the response cost, because the satisfaction of a subjectively more intense craving is delayed when animals have initially chosen a non-rewarding location and have to switch between locations.

Taken together, previous TPL studies have identified two critical factors to induce TPL: A reward to induce goal directed behavior and a response cost to stimulate animals to make correct choices. Both of these factors seem to add to the significance or motivational value for an animal to encode the time-place contingency. Recently, these principles have been implemented in a novel TPL paradigm for mice (Van der Zee et al., 2008). This paradigm will be described in the following section along with recently obtained results using this setup.

A novel TPL test for mice

Despite the extended literature on TPL in rats, albeit with differential conclusions, the lack of studies in mice was surprising and perhaps partly based on the absence of a suitable paradigm. This inspired our group to design a novel TPL paradigm for mice. The rational was to induce a conflict between a positive reinforcer (food reward) and a negative reinforcer depending on the time of day. The latter was implemented by applying a mild foot shock through a grid located in front of the food reward (see Figure 1A). Mice had to step on the grid to be able to acquire the food reward. To motivate the mice to search for food, they were food deprived to 85% of their ad libitum body weight. As such, the paradigm emulates the natural situation in which hungry animals seek food while different feeding locations can be predictably safe or unsafe to visit depending on the time of day. We decided to use three times of day (sessions) in a three-arm maze. For example, a mouse was tested at 9:00, 12:00, and 15:00 o'clock. In each of the three daily test sessions, mice had to learn to avoid one of the three baited arms, i.e., the one in which they would receive the mild foot shock if trying to reach the food reward. For example, at 9:00 this was in the left arm, at 12:00 in the middle arm, and at 15:00 in the right arm. A session was performed correctly if the arm with the foot shock was visited last or fully avoided after first having visited both correct locations. After habituation steps, wild-type C57Bl6 mice readily learned this task, reaching an average performance of approximately 80% correct choices in just five days (Figure 1B). Moreover, session skips (to identify the use of an ordinal strategy) and testing under a constant light condition (to identify the use of an interval timing strategy) revealed that the animals were using a circadian strategy (Van der Zee et al., 2008).

Figure 1

Schematic drawing of the TPL maze used by Van der Zee et al. ( Food is placed behind a metal grid on which mice have to stand to reach it, providing the possibility to administer a mild foot shock (A). Schematic representations of the learning curve: wild-type and Per1,2 mutant mice readily learn to avoid the “time-of-day dependent” negatively reinforced location. In contrast, Cry1,2 knockout mice cannot master the task and remain at the 33% chance level (B).

To our knowledge, this was the first time cTPL was shown in mice. This has opened new possibilities to examine the neuronal substrates underlying cTPL, as well as presumably involved clock genes. Various mice, knockout for specific clock genes, can now be tested behaviorally for the functional involvement of these genes in a time-of-day dependent memory test. In following sections, we will review the first published results on Cry1 and Cry2 knockout mice. In addition, we report on preliminary results of a follow up study in Per1 and Per2 knockout mice.

TPL in Cry and Per mutant mice

cTPL is a unique way to study clock-genes on a functional, behavioral level. We introduced clock-gene deficient mice in the context of TPL, and showed that young Cry1−/−Cry2−/− (referred to as Cry1,2) double knockout male C57Bl6 mice were unable to master the paradigm (Van der Zee et al., 2008). Performance stayed around chance level (Figure 1B). Wild-type mice included in this experiment (of same sex, age, and strain) successfully mastered the paradigm (Figure 1B). Session skips under either LD or LL conditions did not affect performance of wild-type mice, replicating that wild-type mice use a circadian strategy in the used setup.

This finding, that cTPL is Cry dependent, confirmed the circadian nature of cTPL and the functional role of Cry genes in circadian rhythms. In the context of this review, these results however raise a relevant question. Other studies have shown that animals can use alternative non-circadian strategies to solve a daily TPL task. Given that Cry1,2 deficient mice have a distorted internal clock thus raises the question why these mice did not use a non-circadian strategy, like an ordinal- or interval timing strategy, to master the paradigm. The possibility that these animals had a general learning deficit was ruled out by control experiments that showed intact spatial memory and association abilities (contextual fear conditioning and spatial learning in a Y maze; Van der Zee et al., 2008). Clearly the used setup initially induces the circadian strategy as seen in wild-type mice. Nevertheless, feedback from the testing (repeated mistakes leading to foot-shocks) theoretically should have led animals to abandon this unreliable strategy for one that produces better outcome. Several reasons may explain why this did not occur. First, animals may only abandon the circadian strategy if the clock signal gets too weak, as with aging, but not when the memory system gets a clear (but distorted) signal as may be the case with young Cry1,2 knockout mice. Second, session skips may have led animals to mark the ordinal strategy as unreliable as well, resulting in sticking with the circadian strategy. Indeed sessions were already skipped during habituation and on the third day of actual testing. Third, total test duration may have been too short. If the testing would have continued for a longer time (without session skips), the Cry1,2 knockout mice may have switched to a non-circadian strategy. Finally, although interval timing has been shown to be independent of Cry genes (Papachristos et al., 2011) ordinal timing may not be. Future studies should confirm if Cry1,2 double knockout mice can learn TPL using a non-circadian strategy. Nonetheless, the results demonstrated that the time-of-day signal as used in our TPL paradigm must originate from an underlying CRY dependent (circadian) mechanism.

The finding that Cry1,2 knockout mice could not learn TPL, raises the question whether this inability is Cry gene specific, or that (c)TPL depends on other clock genes as well. Using the exact same setup and methods as with the Cry1,2 knockout mice, here we report on some new findings investigating TPL in Per1−/−Per2−/− (referred to as Per1,2) double mutant mice (Zheng et al., 1999). We found that Per1,2 mutant mice can learn TPL indistinguishable from wild-type mice (Figure 1B). Hippocampus dependent contextual and spatial learning has been shown to be unaffected in Per1,2 mutant mice (Zueger et al., 2006), indicating that such factors would indeed not prevent these mutant mice to show TPL by using a non-circadian (ordinal-) strategy. However, session skips and prolonged testing under constant light conditions had no effect on performance, indicating that these mice used a circadian strategy (cTPL). This was unexpected from a genotype that is known to lead to arrhythmicity on both a behavioral and molecular level and raises new questions on the role of Per genes in circadian behavior and memory (Mulder et al., 2011; Mulder et al., submitted).

Perspectives and functional aspects of time-place learning

TPL provides animals with an adaptive mechanism to guide spatiotemporal behavior in accordance with prior experience. cTPL implies that distinct phases of an internal circadian rhythm can participate in associative memory. Studying this type of learning and memory has potentials for several fields of cognitive neuroscience. Below we will outlay some of the perspectives and functional aspects of (c)TPL.

Memory

(c)TPL may demonstrate the formation of tripartite memory association codes. Gallistel stated that whenever an animal encounters a biological significant event, it automatically creates a tripartite memory code consisting of the nature of the event, when the event occurred and where the event occurred (Gallistel, 1990; Wilkie, 1995). The historical difficulties with demonstrating TPL in rats have raised doubts regarding this theory. A study by Thorpe et al. suggests that instead of automatically coding tripartite memory codes on the encounter of important events, animals may automatically create bipartite memory codes, i.e., time-event (and event-place under some circumstances). Thorpe et al. hypothesize that these bipartite codes may later be transformed into tripartite memory codes, but only under high response cost conditions (Thorpe and Wilkie, 2007). Indeed, rats readily show time-of-day discriminations using go/no-go designs in which the value of the reward differs between sessions, without a distinction between different locations (Means et al., 2000; Thorpe et al., 2003). In such designs, animals demonstrate time memory by showing a shorter reaction time at session(s) with a higher reward value. Because a distinction between locations is not necessary in such setups, only bipartite memory codes (time-event) are required, while TPL theoretically requires tripartite memory codes and thus a high response cost (Thorpe and Wilkie, 2007). Future TPL studies may shed more light on the organization of associative memory.

Conditional use of the circadian strategy

The distinction between TPL and cTPL is not always apparent in the literature, because the used strategy is not always investigated. Therefore, little is known on the decisive factors for the utilization of particular strategies. Differential conclusions on the used strategy for TPL within the same strain of specie, indicate that the adapted strategy may not only depend on the used organism or strain, as was concluded from a comparison between seed- and insect eating bird species (Falk, 1992), but on other factors as well. For instance, while cTPL was shown in Wistar rats (Mistlberger et al., 1996), Pizzo et al. reported the utilization of an ordinal strategy by this strain. The authors argument that, although rats may have information on time of day available, they do not readily use this information when other strategies are available, and that, when only two locations and two times have to be distinguished, ordinal timing may be the easiest solution to adapt (Pizzo and Crystal, 2004). However, Mistlberger et al. (1996) had also used a two sessions and two locations TPL setup, indicating that other operational factors must play a role in which strategy animals use for TPL. Theoretically, subjective reward- and response cost values may not only determine whether animals show TPL, but also influence the used strategy. When the stakes are high, memory should be optimal and fault proof. The circadian strategy is theoretically the most reliable strategy because it does not depend on external cues which can be unreliable or remain unnoticed. For instance, with an ordinal strategy, a missed feeding opportunity (similar to a session skip) would result in visiting all following locations at the wrong time of day, which may result in less or no resource collection, increased energy expenditure and/or higher risk of encountering a predator. In expense, a circadian strategy, potentially including the formation of tripartite memory codes, may be more costly on brain resources to create and/or maintain, and may therefore only be recruited when “the stakes” are relatively high. As Falk already stated, perhaps the granivorous weaver birds, showing non-cTPL, learned faster because they learned less (Falk, 1992). From another perspective, it makes sense that the brain does not recruit circadian timekeeping mechanisms for insignificant events. In addition, subjective reward- and response cost value thresholds will likely differ between species and even individuals. This may account for some of the discrepancies found in the literature regarding whether animals showed TPL or not and regarding the used strategies. Thus, although speculative, higher reward- and response cost values may not only induce TPL (Widman et al., 2004) and the formation of tripartite memory codes (Thorpe and Wilkie, 2007), but also the use of a circadian strategy (cTPL). Future studies may confirm this hypothesis.

Clockgenes

cTPL in mice offers a way to study the role of clock genes in a functional and behavioral paradigm (Van der Zee et al., 2008). cTPL was found dependent of Cry1 and/or Cry2 genes and encoded proteins. However, internal timekeeping underlying cTPL seems to be buffered against missing Per1 and Per2 clock genes, even though Cry1,2 and Per1,2 double mutant mice are both behaviorally arrhythmic when housed in absence of a (circadian) LD cycle. This provides new yet underexplored insights regarding the role of these clock genes in the circadian system and memory (Mulder et al., 2011; Mulder et al., submitted). The discrepancy in TPL ability between Cry and Per knockout mice indicates that TPL can be a discriminating paradigm to investigate functional behavioral functionality of known clock-genes and validates the investigation of other molecular clock components.

Aging

TPL has potential as an animal model for episodic memory and aging. Such functional behavioral models are scarce, yet essential to test interventions that potentially improve detrimental effects of aging and (episodic) memory related diseases like Alzheimer's disease (AD). Patients suffering from AD are often said to be disorientated in time and place, and memories of when and where things happened (episodic memory) are among the first to be affected in AD patients. TPL research investigates the functional integration of what, when and where and may thus provide insight on the origin of these early onset AD symptoms (Dere et al., 2005a,b). Moreover, deterioration of the circadian system with aging has been related to other age-associated pathologies (Kondratova and Kondratov, 2012). Despite the potential, TPL has not been investigated before in the context of aging. Circadian system deterioration, such as seen with aging or otherwise, predicts that the cTPL strategy will become less reliable. To buffer performance loss, animals may show to switch the dominant strategy that guides their behavior to one that is independent of the circadian system. In a first investigation of TPL with aging, we found preliminary evidence for this switching hypothesis (Figure 2). Middle-aged (45 weeks old) and aged (80 weeks old) mice showed indistinguishable TPL performance. Session skipping revealed that aged mice primarily used an ordinal strategy while TPL performance of middle-aged mice was significantly less affected by session skips (Mulder et al., 2010; Mulder et al., submitted). Although these results are preliminary due to relatively low numbers of animals, these data suggest that cTPL is more age sensitive than TPL. Animals will first abandon the circadian strategy for one that is independent of the circadian system, before showing age related performance loss, adding aging as a factor for the conditional use of a specific TPL strategy.

Figure 2

Schematic representation of the strategy switching hypothesis with aging. Circadian system decline predicts that the circadian strategy becomes increasingly unreliable with age. To buffer performance loss, the ordinal strategy may gain dominance with aging as a reference to guide TPL behavior.

The switching hypothesis supports a general theory of aging as a loss of behavioral and cognitive flexibility, in this case losing the ability to use the circadian strategy. To compensate performance loss, alternative strategies are explored. In humans, compensatory mechanisms have been demonstrated by the activation of more brain areas for a cognitive task compared to younger individuals (Logan et al., 2002). A similar phenomenon is seen in sleep deprivation. Aged mice and sleep deprived young mice perform just as well as non-sleep deprived young mice in Y-maze learning (a two-arm maze setting without fixed time-of-day testing), showing a similar learning curve. However, they show deficits in reversal learning (Hagewoud et al., 2010; Havekes et al., 2011). Studies have shown that sleep deprived mice rely more on an alternative strategy requiring the striatum (procedural memory) instead of the hippocampus, which is true for aged mice as well (Hagewoud et al., 2010). Although speculative, the striatum may not harbor the necessary connectivity with the circadian system. Future studies will have to confirm the switching hypothesis and may further validate TPL as a model for episodic memory.

cTPL presumes a connection between the circadian system and memory, but not much is known about this connection regarding the origin of the underlying circadian oscillator and the molecular/neuronal signaling to the memory system. cTPL provides the means to investigate this connection. This will be the focus of the final section of this review.

Putative mechanism underlying cTPL

The internal circadian process underlying cTPL currently remains elusive. The SCN (central pacemaker) would be a first educated guess. Via neuropeptidergic, nonsynaptic pathways the SCN is connected to brain regions involved in learning and memory (Van der Zee et al., 2009 and references therein). One of the major output systems of the SCN is the neuropeptidergic Vasopressin (AVP) system. AVP released in the third ventricle reaches numerous brain regions involved in learning and memory, including the hippocampus. AVP receptors are expressed in the hippocampus, and AVP is able to induce LTP in various sub-regions of the hippocampus (Dubrovsky et al., 2003). The AVP neurons of the SCN and the formation of time memory are regulated by the cholinergic input (Hut and Van der Zee, 2011), by which the SCN may provide a time cue to support the formation of time-place associations in the hippocampus. Moreover, both the AVP system (Van der Zee et al., 1999) and cholinergic signaling in the SCN (Van der Zee et al., 1991) are strikingly age-sensitive. Aged rats show age-specific alteration in the AVP and cholinoceptive systems in the SCN (Biemans et al., 2003), predicting age-related impairments in cTPL performance. Although the SCN is known to be arrhythmic in Cry1,2 and Per1,2 double mutant mice, it has been found that the SCN in Per1,2 mutant mice retains some functionality (van der Veen et al., 2008), which may explain why Per1,2 mutant mice still showed cTPL. Taken together, the SCN seems a good candidate as the circadian clock underlying cTPL. However, SCN lesioned rats were still able to show cTPL (Mistlberger et al., 1996), indicating that the SCN is not a prerequisite for cTPL. On the other hand, we found that a 3 h phase shift induced by a light pulse affects feeding patterns in our TPL paradigm (unpublished data), suggesting at least partial or modulator involvement of the SCN.

An alternative candidate for circadian regulation of TPL is the FEO (Stephan, 2002). Despite many lesion studies this oscillator has not yet been fully localized (Davidson, 2006). Currently the FEO is thought to be comprised of a network of interconnected brain structures (Carneiro and Araujo, 2009). cTPL has also been shown in experiments that did not involve food (Widman et al., 2004), arguing against the FEO as the timing mechanism behind cTPL. Another identified oscillator is the methamphetamine sensitive circadian oscillator (MASCO) (Hiroshige et al., 2009), but it has been hypothesized that both the FEO and the MASCO may be a manifestation of the same oscillator induced by arousal (Cain and Ralph, 2009). Together, these known oscillators may drive the circadian expression of many potential time cues for cTPL.

Several criteria may narrow the search: (1) Potential candidate cues should obey the characteristic of being expressed in a circadian fashion. (2) As the hippocampus is the key region for learning and spatial memory, the cue should be able to reach the hippocampus through either synaptic or hormonal routes. (3) In turn the hippocampus should be receptive for the cue. Given these criteria, next to the already discussed SCN derived AVP, potential candidates may be hormones like leptin, ghrehlin, corticosteron, glucagon, and insulin (Lukoyanov et al., 2002; Carneiro and Araujo, 2009), or neurotransmitters like dopamine (Aragona et al., 2002). In addition, non-hormonal metabolic signals, like free fatty acids, ketone bodies and glucose could function as a food driven hourglass timer, for instance, through depletion of glycogen stores or through SIRT1, an NAD(+)-dependent deacetylase which is known to modulate CLOCK-BMAL1 activity (Asher et al., 2008). In addition, multiple time signals and underlying oscillators and/or metabolic hourglass mechanisms may influence subjective timing, as for instance seen in the regulation of sleep (Daan et al., 1984).

Recent studies have shown that the hippocampus harbors its own clockwork. Clock genes are expressed in most regions of the hippocampus (CA1, CA3, DG) and peak expression of some clock genes seems to coincide with times at which memory is consolidated (Feillet et al., 2008; Jilg et al., 2010). Moreover, memory deficits have been shown in clock gene knockout animals (Sakai et al., 2004; Lyons et al., 2006; Feillet et al., 2008; Jilg et al., 2010). Recently, memory formation and consolidation were shown to depend on the circadian reactivation of the cAMP/MAPK/CREB pathway in hippocampal neurons (Eckel-Mahan et al., 2008). Although the circadian expression of clock genes (Per2) has been shown in isolated hippocampal slices (Wang et al., 2009), MAPK cycling seems to be driven by the SCN (Phan et al., 2011). In the SCN, activation of the MAPK pathway is light responsive (Obrietan et al., 1998; Butcher et al., 2002). It has been hypothesized that training might function similarly in hippocampal neurons as light activation and clock resetting functions in SCN neurons; inducing a unique, temporally specific molecular profile in memory-specific neuronal ensembles (Gerstner et al., 2009). Indeed, BMAL1 is negatively regulated by MAPK (Sanada et al., 2002), providing a molecular clock resetting mechanism by training. Interestingly, next to “place” cells in the hippocampal CA1 and CA3 region, “time” cells have recently been identified and shown to track the elapsing of an interval (Macdonald et al., 2011; Shapiro, 2011). It is likely that this hippocampal clock may not only be entrained by training (event encounters), but is also modulated by internal cues like SCN and FEO outputs.

In conclusion, TPL offers research potentials in the neurobiological framework of circadian clock regulation, memory function, episodic-like-memory, and aging. A clear next step is to clarify the neurobiological mechanisms underlying cTPL.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.