Entorhinal mismatch: A model of self-supervised learning in the hippocampus.

The hippocampal formation displays a wide range of physiological responses to different spatial manipulations of the environment. However, very few attempts have been made to identify core computational principles underlying those hippocampal responses. Here, we capitalize on the observation that the entorhinal-hippocampal complex (EHC) forms a closed loop and projects inhibitory signals "countercurrent" to the trisynaptic pathway to build a self-supervised model that learns to reconstruct its own inputs by error backpropagation. The EHC is then abstracted as an autoencoder, with the hidden layers acting as an information bottleneck. With the inputs mimicking the firing activity of lateral and medial entorhinal cells, our model is shown to generate place cells and to respond to environmental manipulations as observed in rodent experiments. Altogether, we propose that the hippocampus builds conjunctive compressed representations of the environment by learning to reconstruct its own entorhinal inputs via gradient descent.

Introduction

The hippocampus has been suggested to play a key role in a range of cognitive functions, including spatial navigation (Burgess, Jeffery, & O'Keefe, 1999), memory consolidation (Nadel and Moscovitch, 1997), attentional shift (Devauges and Sara, 1990), working memory maintenance (Axmacher et al., 2010; Olton et al., 1980), and others (Mack et al., 2018). Interestingly, in spite of the variety of cognitive functions at least partially attributed to the hippocampus, a unified theory of hippocampal function anchored in psychology and the cognitive sciences has not been successfully developed. Indeed, descriptions of diverse computational principles inferred mostly from hippocampal connectivity and single-cell firing patterns, such as competitive selection (De Almeida, Idiart and Lisman, 2009), attractor dynamics (Wills, Lever, Cacucci, Burgess, & O'Keefe, 2005), sequences (Pastalkova et al., 2008), short- and long-term plasticity (Alger and Teyler, 1976), and hierarchical processing (Lavenex and Amaral, 2000), have advanced explanations for behavioral and physiological experimental data from multiple paradigms including habituation to novelty (Yamaguchi, Hale, D'Esposito and Knight, 2004) and latent learning (Kimble and BreMiller, 1981). Therefore, a general description of the computational arsenal of the hippocampus should also advance our understanding of hippocampal cognitive processes.

A standout feature of the circuitry of the hippocampus and entorhinal cortex (EC) is its organization in multiple parallel loops (Bartesaghi et al., 2006). The circuitry of the EC allows the convergence of cortical input and hippocampal output pathways within the same structure, setting the conditions for the implementation of a “comparator” between the two (Lörincz and Buzsáki, 2000). Further, such a plastic circuit that minimizes its input-output discrepancy with multiple inner layers may be viewed as implementing a self-supervisory function, i.e., to act as a type of neural network that is capable of learning without a supervising reference (Wang, 2001). A neural network of such characteristics is normally referred to as an autoencoder (Hinton & Salakhutdinov, 2006), which has been previously used as a model of the hippocampus (Gluck et al., 2003; Gluck and Myers, 1993; Japkowicz et al., 1995), showing how sensory cues can be modulated to support learning in other brain regions performing tasks related to classical conditioning. Furthermore, a recent study following a similar principle has also shown that place cells emerge naturally by the information compression carried out in the bottleneck of the autoencoder (Benna and Fusi, 2019). Although it is unclear whether the hippocampal circuitry evaluates the hippocampal-cortical signal discrepancy (i.e., the comparator) and learns accordingly, it can in principle support computations that are classically associated with the hippocampus: the correction of noisy and incomplete input signals, known as pattern completion (Rolls, 2013), and the generation of sparse divergent representations, known as pattern separation (Yassa and Stark, 2011). Importantly, these two functions are thought to be critical for the role of the hippocampus in forming episodic memories (N. Burgess, Maguire, & O'Keefe, 2002). Therefore, we ask the question whether there is a general computational principle that can explain these phenomena as well as the corresponding physiology and behavioral data in a unified theoretical framework. We argue that the input-output mismatch minimization is the core computation carried out in the entorhinal-hippocampal complex (EHC), which optimizes information compression and transfers in a self-supervised manner.

Interestingly, although the concept of the trisynaptic loop emphasizes the role of the sequential feedforward circuitry (Brewer et al., 2013), a growing body of empirical evidence suggests that multiple mechanisms may carry information in the backward or “countercurrent” direction within the hippocampal formation. For example, the phase of theta waves in the subiculum appear to precede the CA1 and CA3 theta phases, with GABAergic mechanisms playing a key role in the temporal coupling (Jackson et al., 2014). Similarly, all major types of CA3 and CA1 GABAergic neurons have been shown to possess significant boundary-crossing axon terminals, carrying CA activity in the form of inhibition back to dentate gyrus neurons (Szabo et al., 2017). In fact, about a fifth of the GABAergic inputs to dentate granule cells have been estimated to originate from the CA3 and CA1 regions, indicating the potential power of this countercurrent GABAergic hippocampal projection system. Feedback inhibition from the CA1 area to CA3 and dentate gyrus is in general agreement with the hypothesis that a countercurrent pathway for synaptic plasticity may exist within the hippocampus (Sik et al., 1994). A further plausible component engaged in promoting synaptic changes in the hippocampal loop as a consequence of the EC comparator is the entorhinal layers IV-VI projections to granule and GABAergic neurons in the dentate gyrus (Deller et al., 1996) and the GABAergic projection from the deeper layers of the EC to the CA1 (Melzer et al., 2012). Together, these experimental observations of multiple pathways providing backward inhibitory signaling with potential roles in modulating synaptic plasticity highlight a potential solution for a biological implementation of the type of error backpropagation needed for self-supervised gradient descent learning within the hippocampus (see (Santos-Pata et al., 2021) for a more thorough review and a related discussion on its biological feasibility).

Here, we evaluate whether the self-supervised model of the hippocampus featuring the EC comparator hypothesis and input-output mismatch minimization can account for diverse physiological findings reported to occur in the hippocampus and adjacent regions, namely, spatial learning and representations (M. B. Moser, Moser et al., 1995), the response to environmental modifications (Colgin et al., 2008), novelty detection (Knight, 1996), and relearning (Clare et al., 2002). To that end, we implemented ENCORE, a self-supervised network with multiple layers associated with each of the main subregions of the EHC: EC, dentate gyrus (DG), CA3, and CA1. The network is fed with realistic spatial signals from medial EC (MEC) and lateral EC (LEC) that propagate sequentially throughout the network. The learning algorithm applied to the synaptic weights, error backpropagation, minimizes the difference between the activity of the first and last layers. We then evaluate whether the emergent spatial representations in the middle layers can capture the reported data considering particular manipulations on the spatial inputs. Therefore, and contrary to previous models (Benna and Fusi, 2019; Gluck and Myers, 1993), we provide a comprehensive set of realistic environmental manipulations allowing us to compare ENCORE responses to numerous physiological benchmarks and in a layer- or subregion-wise manner. Based on our results, we propose that the hippocampus may be able to implement a form of error backpropagation which affords an end-to-end self-supervised optimization of the whole EHC by continuously minimizing its input-output mismatch. In turn, this core principle leads to an information compression throughout the hippocampal bottleneck that is able to reproduce core physiological responses to environmental modifications seen in rodent experiments.

Results

Learning, spatial representations, and responses to environmental manipulations

We have devised a self-supervised autoencoder network to explore the potential role for entorhinal cells in detecting novelty and to modulate synaptic changes throughout the hippocampus through the backpropagation of error (Figures 1A and 1B; see transparent methods). First, we tested the ability of the model to reconstruct the entorhinal inputs (Figures 1C and 1D) at the output layer. As expected from these types of self-supervised models, the model quickly (40 epochs) learned an optimal weight configuration to minimize the error between the activity of the input and output layers (Figure 1E). After learning, the entorhinal reconstruction error at every spatial bin was asymptotically below 0.01 mean absolute error (Figure 1E, left and center), with a strong population vector correlation between input and output at every point in time (Pearson-r, r = 0.99, p < 0.001, Figure 1E, right). Having established the ability of the model to learn to reconstruct the EC input vector, we analyzed the activity of the hidden layers while exploring the environment. As expected from hippocampal place cells, units in our model tuned their maximal activity to specific locations of the squared arena (Figure 2A). Interestingly, the network's hidden layers (an analogy of the DG, CA3, and CA1 regions) showed place field density distributions broadly reflecting experimental data of the rodent hippocampus (Leutgeb et al., 2007) (Figure 2B).

Figure 1

Hippocampal model and training paradigm

(A) Hippocampal trisynaptic circuit with feedback loop through the EC.

(B) Proposed mechanism to learn from the hippocampal loop in the form of a self-supervised model.

(C) Input rate maps mimicking the ones from grid cells (MEC) and sensory cells (LEC). See transparent methods.

(D) Diagram representation of the model input every time (t) as a function of the spatial location.

(E) (Left and center) Loss and mean absolute error (MAE) of the model during learning (epochs). (Right) The correlation between the real input (EC) to the network and the outcome (Pearson-r = 0.998).

Figure 2

Place cell modulation after environmental morphing

(A) Example of hidden layers (DG, CA3, CA1) activity resembling the ones of hippocampal place cells.

(B) Distribution of number of place fields per cell in the EC input layer (LEC and MEC) and hidden layers.

(C) Procedure to study the model response to environmental modifications. (Left) The EC input used to train the model. (Right) Replacing a percentage of the sensory cells rate maps mimics environmental modifications. Environmental modifications of different degrees (levels) could go from slightly distinct (10%) to a completely novel environment (100%). The population vector (PV) at each spatial bin is fed to the model.

(D) Modifying the environment leads to an increase of place fields size (mean ± standard deviation) in CA3 and CA1 layers but not in the DG (Barry et al., 2012).

(E) The DG layer responds with rate modulation (Leutgeb et al., 2007).

(F) Spatial correlation decreases with increased environmental modifications, less drastically for CA layers, as expected from later hippocampal subregions.

Next, to quantifying the model's response to environmental modifications, we froze learning and tested the model on a second stack of LEC rate maps, representing a novel environment (Figure 2C). Thus, we were able to test the model’s accuracy and the effects of environmental modification at different magnitudes, from familiar (0% modification) to completely novel (100% modification). This approach mimics the modulating the incoming sensory signals due to alteration of objects displaced within the environment. The activity of hippocampal neural ensembles has been observed to be modulated as a function of the degree of environmental modifications, suggesting a rate remapping code mediating spatial maps learned in the DG (Leutgeb et al., 2007). At the computational level, rate remapping has been hypothesized to be mediated by the interaction between spatial and sensory inputs arriving at the hippocampus by entorhinal projections (Rennó-Costa et al., 2010). An interesting question, therefore, is whether input-output mismatch error learning is sufficient to reproduce rate remapping. We quantified the response of units in the model DG in relation to the rate of environmental modifications (Figure 2E). As in rodent physiology, we observed a decrease in the population activity correlation as the environment was progressively modified (Pearson test, r = 0.99, p < 0.001; one-way analysis of variance [ANOVA] F(1,278) = 75,635.479, p = 0.0, η2 = 0.996), closely matching the decay function observed in hippocampal rate remapping experiments. Furthermore, we quantified the changes in spatial correlation in cells at the distinct stages of our network with respect to environmental modification. The simulations indicated a steeper decrease for early (DG) compared to later (CA) stages (Figure 2F), a result that directly speaks to the observations that CA3 populations sustain spatial representations over increased levels of sensory modifications (Leutgeb et al., 2007) due to their attractor dynamics (Reno Costa CA3 paper). Moreover, the spatial correlation dropped significantly for the 3 stages after 50% modification, a signature consistent with global remapping (Sanders et al., 2020). Thus, these results suggest that the internal network configuration (i.e., intra-hippocampal connectivity) learned from the input-output mismatch error derived from the EC comparator is sufficient to explain key features of rate and global remapping.

The scale of spatially tuned place fields of hippocampal CA1 place cells has been observed to temporally expand after environmental modifications (Barry, Ginzberg, O'Keefe and Burgess, 2012). To test whether our model responded similarly, we quantified the size place fields along the hippocampal hidden layers during a progressive morphing of the environment (Figure 2D). We observed that our model's CA1 place fields increased their spatial scale when the LEC sensory stream is altered (one-way ANOVA F(1,1430) = 24.34, p = 0.0, η2 = 0.017). Interestingly, this modulation was specific to CA1 cells, with CA3 cells resisting up to 50% of environmental changes (one-way ANOVA F(1,988) = 21.309, p = 0.0, η2 = 0.021), and with DG cells decreasing their scale at greater modifications (one-way ANOVA F(1,767) = 10.465, p = 0.00127, η2 = 0.013). Moreover, a similar observation concerning the changes in spatial scale has also been observed in MEC grid cells during rodent exploration upon environmental modifications (Barry et al., 2012). Because our model is based on self-supervision, we measured the differences in spatial scaling in the predicted MEC grid cells' fields (the output layer of the model). Surprisingly, and as observed in physiological data, we also identified an increase in grid cells firing fields after environmental sensory modifications (Student's t-test, statistic = −8.881363, p < 0.001, Figure 3). Moreover, we believe that these observations based on spatial alterations are in agreement with the grid-place cell interdependence hypothesis which proposes that grid cells rely on excitation activity from hippocampal neurons (Bonnevie et al., 2013).

Figure 3

Grid cells firing field expansion after environmental modification

(A) Hypothesized modulation of grid cell (MEC) reconstruction activity in EC′ after altering 10% LEC input (sensory cells) (see transparent methods).

(B) Grid cells' firing field size in response to SPECIFIY MANIPULATION. Px = pixels.

The aforementioned modulation of place fields upon environmental modification has been previously demonstrated during environmental morphing (O′Keefe and Burgess, 1996). Specifically, it has been observed that environmental boundary elongation or stretching after navigational exploration is sufficient to control the position and geometry of hippocampal place fields, suggesting that boundary signals, likely originating in MEC layer III (Solstad et al., 2008), are involved in modulating the hippocampal spatial firing activity. Moreover, those observations are also in line with the hypothesis that the hippocampus is involved in both allocentric and egocentric spatial representations (Feigenbaum and Rolls, 1991). Because our model simply received local inputs, i.e., no distal cue information, we next aimed to quantify whether the internally generated network dynamics were sufficient to infer short-distance egocentric representations while still modulating the position and geometry of place fields as observed in the rodent hippocampus. To do so, we generated a horizontally elongated arena mimicking the input signals processed during environmental stretching, where the spatial scale of grid cells remained intact, but sensory signals were extended proportional to environmental morphing ( Figure 4A). We exposed the model to the modified arena but without allowing further synaptic updates, thus avoiding learning of the new configuration. Visual inspection of the model's reconstructed place fields revealed that the firing activity of its place cells accompanied the direction and extent of the environmental elongation (Figure 4B). The size (area) of place fields increased after environmental stretching, affecting all three hippocampal populations ( Figure 4D, DG: original 127 x 84 pixels, stretched 190 x 107 pixels, t test, statistic = −6.9193, p < 0.001; CA3: original 139 x 79 pixels, stretched 200 x 138 pixels, t test, statistic = −5.4390, p < 0.001; CA1: original 137 x 85 pixels, stretched 214 x 160 pixels, t test, statistic = −6.8463, p < 0.001). Moreover, the number of place cells within each stage also increased after environmental stretching, an effect strongly affecting CA1 cells (Figure 4C, t test, DG: t- statistic = − 3.2163, p < 0.001; CA3: t-statistic = −4.4941, p < 0.001; CA1: t-statistic = −3.7907, p < 0.001).

Figure 4

Place field elongation after environmental stretching

(A) Procedure to test the model during environmental stretching as in (O’ Keefe and Burgess, 1996).

(B) Example cell with a stretched place field after environmental manipulation.

(C) Difference in the number of place fields between original and stretched environment, suggesting that more place fields emerge.

(D) As in (C), place fields tend to increase their size after environmental stretch. Px = pixels.

Novelty detection, generalization, and relearning

Once the hippocampal loop has been optimized by minimizing its input-output mismatch in a certain environment, changes in the environmental configuration should lead to changes in activity, serving novelty detection. Therefore, we next studied how activity changed across the stages of the model dependent on the amount of environmental modification (see transparent methods section for a detailed description of how the environment was modified). We used the mean squared error (MSE) of the activity vectors per layer between the original and the novel environment as an approximation of the local error signal since it captures the difference in population activity between conditions.

By continuously varying the environment, we observed a strong relationship between the extent of the environmental modification and the MSE in the CA1 output stage (Figures 5A 5C). This indicates that the error signal produced in the EC by the hippocampal output may not only be detecting changes in the environment but also their magnitude. Furthermore, although this linear dependence between environmental change and error is also maintained across layers, we observed that the magnitude of the error was differentially distributed, with the DG having the highest slope ( Figure 5D). This is in agreement with previous physiological reports demonstrating a larger change in neural activity in the DG as compared to CA3 and CA1 due to novel environments (Hunsaker et al., 2008; Lee et al., 2005). Hence, our results emphasize the role of the DG as the layer within the hippocampal loop in both signaling novelty and undergoing error driven plastic changes (Davis et al., 2004; E. Moser, Moser and Andersen, 1993). Moreover, by changing the environment in a space-specific manner and analyzing the error signal during spatial navigation, we also found that increases in the error signal of the network coincided well with the position of the modified place fields ( Figure 5E). The latter result shows that the error signal generated at the EC that drives learning, can encode novelty during navigation and, therefore, be used to trigger learning at specific locations in the environment. Thus, we also studied the performance of the hippocampal model during the relearning of a novel, modified environment (Figures 5F and 5G). We observed that, even when the environment is completely novel, the network maintains a relatively small initial error, demonstrating the generalization capabilities of its previously learned place fields and their distribution (Figure 5G). Moreover, it can be seen that error decreases very rapidly after the initial novelty detection, showing fast relearning under novelty. However, the number of trials (i.e., epochs) to reach the error asymptote of the naive condition (i.e., first learning) varies with the extent of environmental modification. Interestingly, relearning is faster for both small and large changes as compared to intermediate changes (Figure 5F). That is, highly novel environments drive the place fields to readapt faster than similar environments with medium levels of modification. The latter suggests a nonlinear relationship between the extent of environmental modification and relearning speed that could be tested experimentally.

Figure 5

Novelty detection and relearning

(A) Novelty can be simulated by manipulating the rate maps of the sensory cells (LEC) at specific locations.

(B) MSE for the model output (left) and for individual stages (right) showing that error magnitude decreases along the DG to CA1 layers.

(C and D) The error increases as the environmental modification increases and is stage specific.

(E) Example of novelty detection during navigation showing that the model can detect environmental modifications by monitoring its reconstruction error. By altering the activity of the sensory cells within a portion of the environment (hotspot in the left plot), the model increases its error at that same location (center plot). Thresholding the model's output error allows us to detect modified locations, threshold set to > 0.03.

(F) Number of epochs require to XYZ versus the degree of environmental morphing. The number of epochs needed for learning (stabilization) increases with the environmental modification level. Notably, the model converges quicker for largely different, novel environments.

(G) The number of epochs needed for learning (stabilization) increases with the environmental modification level. Notably, the model converges quicker for largely different, novel environments.

Discussion

Over the last four decades, a variety of cell types have been found to encode egocentric and allocentric spatial features and these have highlighted the role of the hippocampus in mapping, localization and planning (Howard and Eichenbaum, 2015). Indeed, the hippocampal trisynaptic circuit and its physiological responses have been extensively studied with special emphasis on spatial navigation (Ekstrom et al., 2003). In this regard, the associative binding of both sensory- and self-motion- related signals (coming from LEC and MEC, respectively) within the hippocampal network serves the animal's ability to learn the statistical regularities of the environment and to recall previously experienced episodes. Thus, we decided to explore whether the wide variety of reported physiological responses to environmental modifications during spatial navigation in rodents could be accounted for by a small set of core computational principles operating in the hippocampus: self-supervision via EC input reconstruction (comparator hypothesis), information compression along the trisynaptic pathway, and gradient descent learning via countercurrent inhibition. For that purpose, we used an neural network optimized by error backpropagation exposed to realistic stimuli coming from both LEC (sensory) and MEC (space). We observed that our model was capable of generating place cell-like receptive fields, modulating individual neuron's and population activity to environmental modifications , performing novelty detection, and generalizing to novel environments reflecting physiological data.

The implementation of hippocampus-like computations in the form of an autoencoder dates back almost three decades (Gluck and Myers, 1993), for instance, suggested that the hippocampus develops new stimulus representations that enhance the discriminability of predictive cues. Importantly, and unlike these previous studies, we do not make an ontological commitment to the autoencoder as a comprehensive and faithful model of the hippocampus. Instead, we argue that a optimized by error backpropagation captures a unifying computational principle operating in the hippocampus: the gradual compression of external information through gradient descent over the self-generated reconstruction error. We call this architecture ENCORE (Entorhinal Compression Reconstruction). The rationale behind considering ENCORE model as an informative model of the EHC is threefold. Firstly, we select the network topology by grounding our understanding of hippocampal information processing on the aforementioned comparator hypothesis (Lörincz and Buzsáki, 2000), whereby the EC generates an error signal from comparing the neocortical inputs with their respective hippocampal reconstructions. Secondly, we propose that the mismatch error signal generated in the EC comparator is fed back to and optimizes the hippocampal trisynaptic pathway by gradient descent and, more specifically, by error backpropagation. We selected the learning rule of error backpropagation after reviewing recent anatomical and physiological literature pointing to a broad inhibitory (GABAergic) network that runs countercurrent to the mainly excitatory trisynaptic pathway (Santos-Pata et al., 2021). This countercurrent inhibitory network consists of boundary-crossing interneurons that could plausibly modulate synaptic plasticity in pyramidal cells in a way that is consistent with recent proposals about how error backpropagation could be carried out in the brain, i.e., by means of somato-dendritic interactions like plateau potentials and backpropagating action potentials (Lillicrap et al., 2020). Thirdly, ENCORE is tested against an up-to-date range of behavioral and physiological benchmarks. ENCORE also provides an updated account of the comparator and error reconstruction hypothesis with respect to our contemporary knowledge of hippocampal physiology and its related behavior in rodents. In this respect, our model exhibits surprising features mimicking its biological counterpart that are neither trivial nor enforced by training: e.g., the expansion of the receptive fields of the reconstructed grid cells right after environmental morphing (Barry et al., 2012) and the quantitative fit to the rate remapping phenomena (Leutgeb et al., 2007).

One important component of our model, which at the same time operates as one of the main principles of our proposal, is the use of gradient descent learning in the form of error backpropagation. Despite the skepticism in using machine learning methods and computational abstractions to approximate functions of brain regions, recent studies have listed biological mechanisms potentially involved in the backpropagation of error (Guerguiev et al., 2017; Lillicrap et al., 2020). Nonetheless, the pervasive use of neural networks to emulate brain function has also raised concerns about their capability to reliably capture the computational principles and mechanisms by which biological neural systems operate (Massaro, 1988). Indeed, the concern that algorithms like error backpropagation can be super powerful seems to be justified given how they excel at solving complicated tasks achieving super-human performance (e.g. (Silver et al., 2016)). However, recent reviews highlight the conditions where neural networks can indeed be highly informative about brain mechanisms, especially when they provide specific falsifiable predictions and hypothesis (Saxe et al., 2021). In this regard, our model not only exhibits a variety of responses comparable to its biological counterpart without explicitly being trained to do so (i.e., it only learns to reconstruct its own inputs) but also is able to make falsifiable predictions that go beyond fitting specific data sets during training. For instance, our model predicts that place cell stabilization and behavioral performance should be achieved faster under completely novel environments than in moderately modified environments of a previously familiar one (Figures 5F and 5G). Indeed, this could explain the differences in pace between the phenomena of partial and global remapping (Sanders et al., 2020). We thus suggest that, under a completely novel environment, the circuit generates error signals much more frequently with respect to its predictions (i.e., reconstructions) and that these error signals in turn boost learning in the hippocampus (e.g., by means of enhanced plasticity mediated by acetylcholine release, which is indeed driven by novelty (Jeewajee, Lever, Burton, O'Keefe and Burgess, 2008)).

By considering the role of the entorhinal comparator in generating error signals mediating learning, we emphasize the role of the EC in performing novelty detection in the hippocampus. Unlike previous models relying on oscillatory interference networks under resonance amplification (Borisyuk et al., 2001), sequential network models relying on self-supervised learning have already shown to be capable of performing match-mismatch judgments and thus novelty detection by virtue of input reconstruction and input-output comparison (Japkowicz et al., 1995). Notably, recent evidence shows that the comparison operation underlying novelty detection is likely to occur at least partly via interneuron populations within the EC (Miao et al., 2017). However, unlike previous models, we also emphasize the putative role of the DG in encoding novelty (Figure 5D) as being the region exhibiting the most prominent changes in spatial tuning after environmental modifications (Figure 2F). These phenomena indeed point to the DG as a very plastic subregion that would be more prone to rate remapping (Leutgeb et al., 2007), with its neuronal activity signaling novelty (Maass et al., 2014).

Despite the set of benchmarks chosen here to test the ENCORE's ability to capture physiological properties characteristic of the rodent hippocampus, it is far from a complete approximation of its biological counterpart. Among many untested hippocampal features, the ordering of neuronal activity in the form of theta sweeps (Gupta et al., 2012), the role of sharp wave ripples in memory formation (Buzsáki, 2015), as well as the generation of cross- frequency coding schemes (Lisman and Jensen, 2013), or the structured relation between individual neuron activity and the overall population activity (e.g., phase precession (Skaggs et al., 1996)) could not, by design, be emulated with the presented model. We argue however that some of these prominent physiological features of the hippocampus might be cast as being of an implementational nature, rather than of a computational one (i.e., functional principles). For instance, theta oscillations could provide the necessary time multiplexing for forward predictions and error backpropagation to take place alternatively in different theta phases within each cycle (O'Reilly, 1996). Similarly, phase coding enabled by cross-frequency coupling could order hippocampal ensembles in time by the magnitude of synaptic activity (Mehta et al., 2002) to facilitate credit assignment during error backpropagation. In addition, we also want to emphasize another relevant feature corresponding to the sequence generation capacity of the hippocampus, which has been argued to be its main function (Buzsáki and Tingley, 2018), acting as a glue to form episodes from temporally contiguous experiences. Further work would include an extension of the ENCORE model to capture this fundamental property by adding recurrent connections to the middle layer (CA3), which indeed is known to be highly recursive (Lebovitz et al., 1971). Also the distinct anatomical and physiological constraints of the hippocampus must be included including its pattern separation and completion capabilities. Also, generative variants such as variational autoencoders (Kingma and Welling, 2014) could be explored in relation to the putative hippocampal role in sequence generation or sampling from internal world models. These features (generativity and recurrency) would potentially shed light onto what properties of the hippocampus are explained by information compression alone or by (sequence-based) prediction.

Limitations of the study

The present study does not address all available benchmarks within the hippocampal literature. Among many untested experiments, we emphasize here the need for a more comprehensive account of the diverse remapping phenomena (Sanders et al., 2020). We believe that a full understanding of remapping and how it relates to generative processes is likely to provide further critical insights into hippocampal function. Also, we did not explicitly test for the presence of the typical range of hippocampal cells usually reported in literature (e.g., border cells, object-vector cells, head direction cells, etc.). In fact, some of them, like head direction cells, are just not possible to account for given the testing paradigm used here, where there is a lack of active navigation within the environment. Hence, actively sampling the respective EC population vectors (Figure 1D) based on velocity or direction vectors and thus mimicking what would be an active exploration of the arena would probably overcome this limitation and possibly lead to the emergence of such variety of hippocampal cells within the hidden layers of the network as we have shown in previous models (Maffei et al., 2015, DACX).

Features that might be relevant to include in further generations of the model would increase its explanatory value include: oscillations and theta-based sequences (Gupta et al., 2012), leading to phase precession (Skaggs et al., 1996) and cross-frequency coding schemes (Lisman and Jensen, 2013), and sharp wave ripples for replay and consolidation (Buzsáki, 2015). As discussed above, the generative properties of the hippocampus, together with the predictive nature of its sequence generation, are probably the most important computational features that should be added to the present model to attempt a more comprehensive account of the related literature.

Overall, the hippocampus is known to play a role in learning features and their associations over multiple dimensions, ranging from spatial location encoding (E. I. Moser, Kropff and Moser, 2008) to semantic relationships (Solomon et al., 2019) to abstract concepts (Quiroga, 2012). Even though our study largely focused on the spatial domain, similar computational mechanisms mediating position and environmental encoding should generalize across domains. The simple principle of EC mismatch error minimization, when applied to biologically realistic EC inputs, is sufficient to explain key physiological phenomena seen in the rodent hippocampus during spatial navigation. Furthermore, it demonstrates how machine learning systems could realize novelty detection, generalization, and rapid re-adaptation to environmental contingencies in an autonomous manner during navigation, thus leading to the holy grail of artificial intelligence: epistemic autonomy (Santos-Pata et al., 2021).

Resource availability

Lead contact

Further information and requests should be directed to and will be fulfilled by Paul FMJ Verschure (pverschure@ibecbarcelona.eu).

Materials availability

This study did not generate new unique reagent.

Data and code availability

The code used to simulate the model and generate the results included in the figures is available at: https://gitlab.com/diogo.santos.pata/encore.

Methods

All methods can be found in the accompanying transparent methods supplemental file.