Cen Yang1,2,3, Yuji Naya1,3,4,5. 1. School of Psychological and Cognitive Sciences, Peking University, Beijing, China. 2. Academy for Advanced Interdisciplinary Studies, Peking University, Beijing, China. 3. Center for Life Sciences, Peking University, Beijing, China. 4. PKU-IDG/McGovern Institute for Brain Research, Peking University, Beijing, China. 5. Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China.
Abstract
The ability to use stored information in a highly flexible manner is a defining feature of the declarative memory system. However, the neuronal mechanisms underlying this flexibility are poorly understood. To address this question, we recorded single-unit activity from the hippocampus of 2 nonhuman primates performing a newly devised task requiring the monkeys to retrieve long-term item-location association memory and then use it flexibly in different circumstances. We found that hippocampal neurons signaled both mnemonic information representing the retrieved location and perceptual information representing the external circumstance. The 2 signals were combined at a single-neuron level to construct goal-directed information by 3 sequentially occurring neuronal operations (e.g., convergence, transference, and targeting) in the hippocampus. Thus, flexible use of knowledge may be supported by the hippocampal constructive process linking memory and perception, which may fit the mnemonic information into the current situation to present manageable information for a subsequent action.
The ability to use stored information in a highly flexible manner is a defining feature of the declarative memory system. However, the neuronal mechanisms underlying this flexibility are poorly understood. To address this question, we recorded single-unit activity from the hippocampus of 2 nonhuman primates performing a newly devised task requiring the monkeys to retrieve long-term item-location association memory and then use it flexibly in different circumstances. We found that hippocampal neurons signaled both mnemonic information representing the retrieved location and perceptual information representing the external circumstance. The 2 signals were combined at a single-neuron level to construct goal-directed information by 3 sequentially occurring neuronal operations (e.g., convergence, transference, and targeting) in the hippocampus. Thus, flexible use of knowledge may be supported by the hippocampal constructive process linking memory and perception, which may fit the mnemonic information into the current situation to present manageable information for a subsequent action.
Declarative memory enables individuals to remember past experiences or knowledge and to use that information according to a current situation [1, 2]. This flexible use of stored information is in contrast to procedural or fear-conditioned memory, in which acquired memory is expressed in a fixed form of associated actions or physiological responses [3-5]. Previous studies revealed the involvement of the hippocampus (HPC) in the medial temporal lobe (MTL) in the formation and retrieval of declarative memory [2, 3, 6–14]. However, the mechanism by which the HPC contributes to the flexibility in the usage of the declarative memory remains largely unknown.The contribution of the HPC to declarative memory was often investigated by examining its spatial aspects in both human subjects [15-17] and animal models [3, 8–10, 13, 14, 18–20]. In the preceding literature, the contributions of the HPC to the spatial memory task were successfully dissociated from those of the other brain areas when the start position in spatial mazes differed between training (e.g., “south” in a plus maze) and testing trials (e.g., “north”) [3, 21], because the fixed action patterns acquired during the training period (e.g., “turn left,” egocentric coordinate) cannot guide the subjects to a goal position (e.g., “west,” allocentric coordinate) in the testing trials. The HPC thus contributes to the memory task by retrieving a goal position, which could be represented in an acquired cognitive map [22]. However, in order to reach the goal position, it is not enough for the subjects to remember the goal on the cognitive map, which represents the allocentric spatial relationship of the environment in mind. In addition, it would be critical for the subjects to locate their self-positions by perceiving current circumstances around them and relate the goal to the self-positions in egocentric coordinate for a subsequent action. The subjects thus need to transform the goal position within the cognitive map into goal-directed information relative to a specific circumstance (i.e., start position) that the subjects currently experience. In the present study, we hypothesized that the goal-directed information in the specific circumstance may be constructed by combining the retrieved memory and incoming perceptual information on the same principle as “constructive episodic memory system” suggested by human neuroimaging studies [1, 23, 24]. In this theory, the constructive episodic memory system recombines distributed memory elements for both remembering the past and imagining the future (i.e., “mental time travel”) [25].We therefore investigated whether and how the HPC neurons combine the retrieved location with the perceived circumstance in order to construct goal-directed information. To achieve this purpose, we devised a new memory task for macaque monkeys, in which memory retrieval and its usage were separated by sequential presentations of 2 cues in a single trial (Fig 1). The first cue presented a visual item (item cue) that would trigger retrieval of the location associated with the item. The second cue presented a background image (background cue) that would be combined with the retrieved location to construct goal-directed information. This task structure allowed us to separate the constructive process from the retrieval of item-location association memory. In addition, the animals were prompted to link the individual items to the preassigned locations on the background image through repetitive trainings. Taken together, we investigated the constructive process to fit the semantic-like memory (cf., episodic memory) to the current situation in the present task. We referred to this new task as the constructive memory-perception (CMP) task.
Fig 1
CMP task.
(a) Item stimuli. (b) Item-location association pattern. Two items, one from set A (e.g., I-A) and the other from set B (e.g., I-B), were assigned to each location (e.g., co-location I) on the background image. Scale bar for both item-cue and background-cue stimuli, 5° visual angle. (c) Schematic diagram of the CMP task. An item cue and background cue were chosen pseudorandomly in each trial. The monkeys should maintain fixation on the center until the end of the background-cue period including Delay 2, then saccade to the target location (red arrow) during the choice period. Monkeys were trained using every 0.1° step in orientation from −90° to 90°, though only 5 orientations (−90°, −45°, 0°, 45°, and 90°) were tested during the data acquisition. Relative sizes of the item-cue stimuli to the background-cue stimuli were magnified for display purpose. CMP, constructive memory-perception.
CMP task.
(a) Item stimuli. (b) Item-location association pattern. Two items, one from set A (e.g., I-A) and the other from set B (e.g., I-B), were assigned to each location (e.g., co-location I) on the background image. Scale bar for both item-cue and background-cue stimuli, 5° visual angle. (c) Schematic diagram of the CMP task. An item cue and background cue were chosen pseudorandomly in each trial. The monkeys should maintain fixation on the center until the end of the background-cue period including Delay 2, then saccade to the target location (red arrow) during the choice period. Monkeys were trained using every 0.1° step in orientation from −90° to 90°, though only 5 orientations (−90°, −45°, 0°, 45°, and 90°) were tested during the data acquisition. Relative sizes of the item-cue stimuli to the background-cue stimuli were magnified for display purpose. CMP, constructive memory-perception.By measuring single-unit activities during the CMP task, we examined whether and how the retrieved memory and incoming perceptual signals were combined in the HPC. One hypothesis might be that the 2 signals would be directly linked to the goal-directed information by a conjunctive representation [20, 26, 27], which binds input elements into a unitary representation and supports the “hippocampus indexing theory” [45]. An alternative hypothesis might be that the memory and perceptual signals converge on the responses of single HPC neurons holding both signal contents [20]. This “convergence” process would require an additional neuronal operation to transfer the retrieved location to the target (“transference” process) and then to represent the target location itself (“targeting” process), which would be analogous to the conjunctive representation.The present study supported the sequentially occurring neuronal operations in the HPC consisting of the “convergence,” “transference,” and “targeting” processes. The HPC may equip the declarative memory with flexibility in its usage by the constructive process combining memory and perception through the 3 neuronal operations.
Results
CMP task
Two rhesus macaques were trained to perform the CMP task. In the CMP task, 4 pairs of visual items were assigned to 4 different locations (co-locations) on a background image (Fig 1A and 1B, S1 Text), which would be stored as long-term association memory linking the items and their locations in allocentric coordinate. We referred to the 2 items in each pair as “co-location” items (e.g., I-A and I-B) because the 2 items were assigned to the same location on the background image. The configuration of the co-location items allowed us to evaluate an item-location memory effect for each single neuron by examining the correlation in its responses to the co-location items. In the present study, we used the same 8 visual items and 1 background image during all the recording sessions. In each trial, 1 of the 8 items was presented as an item cue (e.g., II-A) (Fig 1C). After a short delay, a randomly oriented background image was presented as a background cue (e.g., −90°). The subjects were then required to saccade to the target location (e.g., top-right), which would be represented in egocentric coordinate, determined by the combination of the item and background cues (e.g., co-location II on the −90°-oriented background on the display).In the initial training, the monkeys learned the item-location association through trials with a fixed orientation of background cue (which we defined as 0°). After they learned the association between items and locations in trials with the 0° background cue, orientation of the background cue was randomly chosen from −90° to 90° (in 0.1° steps). During the recording session, the orientation of the background cue was pseudorandomly chosen from among 5 orientations (−90°, −45°, 0°, 45°, and 90°). The 2 monkeys performed the task correctly (chance level = 25%) at rates of 80.9% ± 8.1% (mean ± standard deviation; monkey B, n = 179 recording sessions) and 96.8% ± 3.1% (monkey C, n = 158 recording sessions). Neither of the animals showed any strong bias in the performance among item-cue identities, background-cue orientations, or target locations (S1 Fig). While the monkeys performed the task, we recorded single-unit activity from 456 neurons (n = 247 for monkey B, n = 209 for monkey C) in the HPC of the MTL (S2 Fig, S1 Table).
Representation of the retrieved memory
We first investigated the retrieval process during the item-cue period of the task. Fig 2A shows an example of a neuron exhibiting item-selective activity (item-selective neuron, P < 0.01, 1-way ANOVA). This neuron exhibited the strongest response to item I-A (optimal), whereas an item paired with the optimal item (I-B, pair) elicited the second-strongest response from the same neuron. The neuron thus strongly responded to only the particular co-location items (i.e., I-A and I-B) but not to others (Fig 2B). The selective responses to I-A and I-B could not be explained by eye position (S3 Fig). To examine the item-location association effect, we calculated the Pearson correlation coefficient between the responses to the 4 pairs of co-location items and referred to it as the co-location index (S4 Fig). Therefore, the co-location index was influenced by the responses not only to the optimal and its paired co-location items but also all other items. If a single neuron in a population showed the pattern of stimulus selectivity that was independent of the items’ co-locations, the mean value of the co-location index for the neuronal population would be expected to approach zero as the number of neurons in the population increased. The co-location index of this neuron was extremely high (Fig 2B) (r = 0.99, P < 0.0001, permutation test, 2-tailed), which indicates a strong long-term memory effect on the responses of this neuron.
Fig 2
Representation of the retrieved memory during the item-cue period.
(a) Example of an item-selective neuron with the co-location effect. Black lines indicate SDFs in trials with the Optimal and its Pair items (i.e., best co-location items) of the neuron. Gray line indicates averaged response in the other trials (Other). Brown bar, presentation of the item cue. (b) Mean discharge rates and SEM of the same neuron during the item-cue period for each item. Black bars, set A. White bars, set B. r, co-location index. ***P < 0.0001, permutation test, 2-tailed. (c) Population-averaged response of item-selective neurons (n = 136). SDFs in trials with the best co-location items (i.e., Optimal and Pair) and other items. Shading, SEM. Purple line, time duration indicating a significant (P < 0.01, t-test, 2-tailed) difference between pair and other. (d) Distributions of co-location indices for item-selective neurons (n = 136). r, median value. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. (e) Response discriminability between the optimal and its paired items of the same example neuron in Fig 2A. (Left) ROC curve. (Right) Solid vertical line, AUC value of the example neuron. Gray area and dashed vertical line, distribution of simulated AUC values and its median. (f) Response discriminability between best and other co-locations of the same neuron. ***P < 0.0001, permutation test, 1-tailed. (g) Two-dimensional scatter plots of AUC values between the item (ordinate) and co-location (abscissa) discriminations for item-selective neurons with high co-location index (r > 0.6; n = 109). Each circle indicates one neuron. Arrow, median of real AUC values for each discrimination. Dashed line, median of simulation AUC values for each discrimination. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. s, significant, P < 0.05 for each neuron, permutation test, one-tailed. Source data are available in S1 Data. AUC, area under curve; ns, nonsignificant; ROC, receiver operating characteristic; SDF, spike density function.
Representation of the retrieved memory during the item-cue period.
(a) Example of an item-selective neuron with the co-location effect. Black lines indicate SDFs in trials with the Optimal and its Pair items (i.e., best co-location items) of the neuron. Gray line indicates averaged response in the other trials (Other). Brown bar, presentation of the item cue. (b) Mean discharge rates and SEM of the same neuron during the item-cue period for each item. Black bars, set A. White bars, set B. r, co-location index. ***P < 0.0001, permutation test, 2-tailed. (c) Population-averaged response of item-selective neurons (n = 136). SDFs in trials with the best co-location items (i.e., Optimal and Pair) and other items. Shading, SEM. Purple line, time duration indicating a significant (P < 0.01, t-test, 2-tailed) difference between pair and other. (d) Distributions of co-location indices for item-selective neurons (n = 136). r, median value. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. (e) Response discriminability between the optimal and its paired items of the same example neuron in Fig 2A. (Left) ROC curve. (Right) Solid vertical line, AUC value of the example neuron. Gray area and dashed vertical line, distribution of simulated AUC values and its median. (f) Response discriminability between best and other co-locations of the same neuron. ***P < 0.0001, permutation test, 1-tailed. (g) Two-dimensional scatter plots of AUC values between the item (ordinate) and co-location (abscissa) discriminations for item-selective neurons with high co-location index (r > 0.6; n = 109). Each circle indicates one neuron. Arrow, median of real AUC values for each discrimination. Dashed line, median of simulation AUC values for each discrimination. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. s, significant, P < 0.05 for each neuron, permutation test, one-tailed. Source data are available in S1 Data. AUC, area under curve; ns, nonsignificant; ROC, receiver operating characteristic; SDF, spike density function.Fig 2C shows the population-averaged spike density functions (SDFs) of item-selective neurons (n = 136) to their optimal items, paired items, and other items (average across 6 items). The responses to the items paired with the optimal items were significantly larger than those to the other items during the item-cue period (P < 0.01 for each time step, t-test, 2-tailed). The item-selective neurons also showed extremely large co-location index values (r = 0.89, median) (Fig 2D). We confirmed that the large co-location index values could not be explained by eye position (S2 Text). These results indicated that the HPC showed an item-location association effect on the item-selective activities.The item-location association effect, revealed by the co-location index using the Pearson correlation coefficient, suggests 2 possible response patterns in the HPC during the item-cue period: the neuronal responses representing the locations retrieved from the item cues and those representing individual items that were modulated by the co-locations. If the former holds true, the neurons would not distinguish the co-location items because they would signal the same location. Conversely, if the latter holds true, the neurons would discriminate between the co-location items, although the responses to the co-location items were correlated. To test these alternative assumptions, we conducted the receiver operating characteristic (ROC) analysis for the item-selective neurons that showed disproportionately high co-location index (r > 0.6, 80% of the recorded neurons) (Fig 2D). We calculated the corresponding area under curve (AUC) for each neuron and examined whether the value was significantly (P < 0.05) larger than the chance level, which was estimated by a permutation test (see Methods). Fig 2E indicated response discriminability between the optimal and its paired items of the same neuron in Fig 2A. The ROC curve was close to the diagonal line from (0, 0) to (1, 1), and its AUC value (0.51) was even smaller than an expected value (median AUC = 0.56 in 10,000 permutations). Conversely, the same neuron showed significant response discriminability between best and other co-locations (AUC = 0.84, P < 0.0001, permutation test, 1-tailed) (Fig 2F). Out of the 109 neurons with the high co-location index, 93 neurons (85%) could not discriminate optimal items from their paired items (Fig 2G) even with the use of a liberal threshold of statistical significance (P < 0.05, one-tailed). We confirmed that 89% of the 109 neurons successfully discriminated the best co-location items, including the optimal items and their paired items (e.g., I-A and I-B for the neuron in Fig 2A), from other co-location items. These results indicate that the HPC neurons exhibited “unitized” [28, 29] responses to the co-location items, implying activation of the same location (i.e., co-location) information to be retrieved from the co-location items.
Retrieval signal after background-cue
We examined item-selective activity during the background-cue period using a 3-way ANOVA with item cue, background cue, and target position effects as main factors for each neuron (P < 0.01) and found a substantial number of item-selective neurons (47 out of 456 neurons) (Fig 3A and 3B). The item-selective neurons during the background-cue period showed larger co-location index values (median r = 0.94, P < 0.01, Kolmogorov–Smirnov test) (Fig 3C) than those during the item-cue period. Moreover, 44 out of the 47 item-selective neurons (94%) could not distinguish individual items of the best co-locations significantly (P < 0.05, permutation test, 1-tailed) (Fig 3D). These results suggested a strong unitization effect on the item-selective activities in each co-location during the background-cue period. Therefore, we designed a 3-way nested ANOVA in which individual co-location items were under their co-locations to examine the main effects of the “co-location,” “background,” and “target” on neuronal responses during the background-cue period for each neuron (S1 Table, S3 Text). The 3-way nested ANOVA showed that 66 out of the 456 recorded neurons exhibited significant (P < 0.01) co-location effects on their activities during the background-cue period. Out of them, 30 neurons exhibited the co-location-selective activities only after the background-cue presentation (Fig 3A and 3B, S5A Fig), which might be recruited to signal the retrieved location in the HPC for the necessity of the constructive process during the background-cue period (S5B Fig).
Fig 3
Retrieval signal after background-cue.
(a) Example of an item-selective neuron with co-location effect. Black lines indicate SDFs in trials with the Optimal and its Pair items (i.e., best co-location items) of the neuron. Gray line indicates averaged response in the other trials (Other). Brown bar, presentation of the item-cue. Gray bar, presentation of the background cue. (b) Mean discharge rate and SEM of the same neuron during the background-cue period for each item. Black bars, set A. White bars, set B. r, co-location index. ***P < 0.0001, permutation test, 2-tailed. (c) Cumulative frequency histograms of the co-location index. Black line, item-selective neurons during B-Cue period. Gray line, item-selective neurons during I-Cue period. r, median of the co-location index values. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. †P < 0.01, Kolmogorov–Smirnov test. (d) Two-dimensional scatter plot of AUC values for item-selective neurons during the background-cue period (n = 47). Same format as Fig 2G. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. s, significant, P < 0.05, permutation test, 1-tailed. Source data are available in S1 Data. AUC, area under curve; B-Cue, background cue; I-Cue, item cue; ns, nonsignificant; SDF, spike density function.
Retrieval signal after background-cue.
(a) Example of an item-selective neuron with co-location effect. Black lines indicate SDFs in trials with the Optimal and its Pair items (i.e., best co-location items) of the neuron. Gray line indicates averaged response in the other trials (Other). Brown bar, presentation of the item-cue. Gray bar, presentation of the background cue. (b) Mean discharge rate and SEM of the same neuron during the background-cue period for each item. Black bars, set A. White bars, set B. r, co-location index. ***P < 0.0001, permutation test, 2-tailed. (c) Cumulative frequency histograms of the co-location index. Black line, item-selective neurons during B-Cue period. Gray line, item-selective neurons during I-Cue period. r, median of the co-location index values. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. †P < 0.01, Kolmogorov–Smirnov test. (d) Two-dimensional scatter plot of AUC values for item-selective neurons during the background-cue period (n = 47). Same format as Fig 2G. ***P < 0.0001, Wilcoxon’s signed-rank test, 2-tailed. s, significant, P < 0.05, permutation test, 1-tailed. Source data are available in S1 Data. AUC, area under curve; B-Cue, background cue; I-Cue, item cue; ns, nonsignificant; SDF, spike density function.
Convergence of the retrieved memory and incoming perception
We next investigated how the incoming background-cue information affected the retrieved location signal. Fig 4A shows an example of a neuron exhibiting selective responses to the background cues (P < 0.01, 3-way nested ANOVA). This neuron showed the strong responses across all the co-locations when the orientation of the background cue was either 90° or 0°, whereas it showed only negligible responses when the orientation was −90°. An amplitude of the background-cue effect was exhibited as a time course of the F value (gray curve, middle panel), which characterized the transient increase of the background-cue effect on the neuron’s responses regardless of the co-locations of item-cues (yellow curve) and target positions (black curve). In addition to the neurons showing only background-selective activity (e.g., Fig 4A), we found neurons showing selectivity for both co-locations and backgrounds. An example neuron in Fig 4B began signaling co-locations III and IV at the end of the item-cue period. After the background-cue presentation, this neuron exhibited additional excitatory responses for the best co-locations (i.e., III and IV), especially when the orientation of background cue was 90°. The background-selective responses were thus combined with the co-location-selective responses in this individual neuron (see also S6A Fig), which was shown by the overlap between the co-location (yellow curve) and background (gray curve) effects indicated by their F values (middle panel). We further evaluated the similarity of orientation tuning across co-locations for the example neuron by calculating the Pearson correlation coefficient between the responses to the different orientations of the background-cues for the “best co-location” (III) and those for the “second-best co-location” (IV) (Fig 4B). We found high similarity of orientation tuning across the co-locations (r = 0.95). These results indicate that this neuron signaled the background cue irrespective of which co-location signal the neuron held from the item-cue period.
Fig 4
Convergence of the retrieved memory and incoming perception.
(a) Example neuron signaling background effect. (Top) Each row contains an SDF for each combination of I- and B-Cues. (Middle) Time courses of F values. Brown bar, presentation of the I-Cue. Gray bar, presentation of the B-Cue. (Bottom) Mean discharge rate for each combination of I- and B-Cues during 60–1,000 milliseconds after B-Cue onset. White, gray, and black bars indicate −90°, 0°, and 90° orientations of background cue, respectively. Two bars with the same grayscale indicate the co-location items (e.g., left bar, I-A; right bar, I-B). (b) Example neuron signaling background and co-location effects in a “convergent” manner. Same format as Fig 4A. r, Pearson correlation coefficient between the orientation tunings (responses to −90°, 0°, and 90° during the B-Cue period) for “best co-location” (III) and for “second-best co-location” (IV). (c) Time course of similarity of orientation tuning r(t). Line and shading, means and SEMs of the similarity of the orientation tunings for co-location-selective neurons. Purple line, time duration in which the similarity was significantly positive (P < 0.01, n = 66, Wilcoxon’s signed-rank test for each time step, 2-tailed). Source data are available in S1 Data. B-Cue, background cue; I-Cue, item cue; SDF, spike density function.
Convergence of the retrieved memory and incoming perception.
(a) Example neuron signaling background effect. (Top) Each row contains an SDF for each combination of I- and B-Cues. (Middle) Time courses of F values. Brown bar, presentation of the I-Cue. Gray bar, presentation of the B-Cue. (Bottom) Mean discharge rate for each combination of I- and B-Cues during 60–1,000 milliseconds after B-Cue onset. White, gray, and black bars indicate −90°, 0°, and 90° orientations of background cue, respectively. Two bars with the same grayscale indicate the co-location items (e.g., left bar, I-A; right bar, I-B). (b) Example neuron signaling background and co-location effects in a “convergent” manner. Same format as Fig 4A. r, Pearson correlation coefficient between the orientation tunings (responses to −90°, 0°, and 90° during the B-Cue period) for “best co-location” (III) and for “second-best co-location” (IV). (c) Time course of similarity of orientation tuning r(t). Line and shading, means and SEMs of the similarity of the orientation tunings for co-location-selective neurons. Purple line, time duration in which the similarity was significantly positive (P < 0.01, n = 66, Wilcoxon’s signed-rank test for each time step, 2-tailed). Source data are available in S1 Data. B-Cue, background cue; I-Cue, item cue; SDF, spike density function.After background-cue presentation, a substantial number of neurons (22% of the recorded neurons, P < 0.01, 3-way nested ANOVA) exhibited either co-location-selective activities (14%, 66 neurons) or background-selective activities (14%, 66 neurons). Importantly, a significantly larger number of neurons (n = 32, P < 0.0005, χ2-test) showed both co-location and background-cue effects on their activities than the expected number (i.e., 66/456 × 66/456 × 456 = 9.6) (S1 Table). We further evaluated the background-cue effect on the co-location-selective activities in the HPC by examining the similarity of orientation tuning for each of the co-location-selective neurons during the background-cue period (n = 66). To do this, we calculated the correlation coefficient at each instantaneous time point (100 milliseconds of time-bin) after the background-cue onset. Here, a positive value of the correlation coefficient would imply a similar orientation tuning across co-locations. A similarity between the orientation tunings was observed from 228 to 458 milliseconds after the background-cue onset in the population (P < 0.01 for each time step, Wilcoxon’s signed-rank test) (Fig 4C). These results suggested a convergence of the retrieved location and perceptual information on the single-neurons, which transiently held both signal properties on their activity (“convergent-type”), rather than a conjunctive representation.
Representation of retrieved location and target location
After the background-cue presentation, some co-location-selective neurons exhibited target location selectivity. For example, a neuron in Fig 5A responded to item cues that were assigned to the co-location III during the item-cue period, whereas the same neuron showed selective activity for a particular target location that corresponded to the bottom-left (yellow) during the background-cue period. The bottom-left of the target location matched to the co-location III if we assume the background image with 0° orientation. The responses to the target locations during the background-cue period were largely correlated with those to the co-locations during the item-cue period when the co-locations were assumedly positioned relative to the 0° background image (matching index, r = 0.99) but not to the −90° (r = −0.28) nor the 90° (r = −0.23) background image (Fig 5B). This result may imply that the item-location is retrieved relative to the 0° background image as default. The presence of the default position/orientation of the background image in a mental space of the monkeys may reflect the effect of initial training, during which the monkeys learned the combinations of items and locations in trials with the 0° background cue. To test this implication, we collected 49 neurons showing significant target-selectivity out of the 136 neurons with item-selectivity during the item-cue period. These neurons tended to show the preferred target locations that corresponded to the preferred co-locations relative to the 0° background cue (default orientation) but not to the other orientations during the item-cue period (Fig 5C). It should be noted that if these neurons represented the retrieved location relative to the background image only in allocentric coordinate without projecting it into egocentric coordinates (first person’s perspective), their preferred co-locations and target locations would be independent, and a population average of the correlation coefficients (matching index) would be close to zero value in any orientation of the background cue. The presence of the default position/orientation of the background image implies that the HPC might represent the retrieved location in the egocentric space (first person’s perspective) rather than the allocentric space.
Fig 5
Representation of retrieved location and target location.
(a) Example neuron signaling a particular co-location during the item-cue period and a particular “targeting” location during the B-Cue period. Same format as Fig 4A, except that target locations in the bar graph are indicated by colors (bottom panel). For example, yellow color corresponds to the bottom-left of the target location on the display. (b) Potential matching patterns between the co-location and target location. (c) Median value of matching index for each matching pattern (using neurons signaling both co-location-selectivity during the item-cue period and target-selectivity during the background-cue period, n = 49). Error bar, quarter value. ***P < 0.0001, Wilcoxon signed-rank test, 2-tailed. Source data are available in S1 Data. B-Cue, background cue; BL, bottom left; BR, bottom right; I-IV, co-location I-IV; I-Cue, item cue; TL, top left; TR, top right.
Representation of retrieved location and target location.
(a) Example neuron signaling a particular co-location during the item-cue period and a particular “targeting” location during the B-Cue period. Same format as Fig 4A, except that target locations in the bar graph are indicated by colors (bottom panel). For example, yellow color corresponds to the bottom-left of the target location on the display. (b) Potential matching patterns between the co-location and target location. (c) Median value of matching index for each matching pattern (using neurons signaling both co-location-selectivity during the item-cue period and target-selectivity during the background-cue period, n = 49). Error bar, quarter value. ***P < 0.0001, Wilcoxon signed-rank test, 2-tailed. Source data are available in S1 Data. B-Cue, background cue; BL, bottom left; BR, bottom right; I-IV, co-location I-IV; I-Cue, item cue; TL, top left; TR, top right.
Construction of the goal-directed information
We next investigated how the retrieved location was transformed to the target location when the background cue was presented. Fig 6A shows an example neuron that exhibited strong transient responses to particular combinations of item cues and background cues ([I, 90°] and [III, −90°]) corresponding to the same target location (bottom right, green). However, this neuron did not respond when the background cue was 0° even though the combination (II, 0°) corresponded to the same target location (bottom right). This implies that the neuron responded only when the retrieved location was transferred to the preferred target location of the neuron (i.e., bottom right). The target signal of this example neuron thus depended on the preceding 2 cues, which was demonstrated by the increased F values for all 3 main effects. This “transferring-type” of activity contrasts with the target-selective activity of the neuron shown in Fig 5A, which signaled the preferred target location itself regardless of the co-locations of item cues or the orientations of background cues. We refer to the latter type of target-selective activity as “targeting-type,” which was characterized by a robust increase of the F values only for the target effect (black curve, Fig 5A, S6B Fig). Interestingly, some individual neurons exhibited convergent-type activity first, then transferring-type activity, and finally targeting-type activity (Fig 6B, S6C Fig). These results imply a temporal relationship between the transference effect and targeting effect during the construction of goal-directed information.
Fig 6
Construction of goal-directed information.
(a) Example neuron showing the “transference” effect ([I, 90°] and [III, −90°] for the bottom-right). Same format as Fig 5A. Bottom panel shows a schematic diagram of “transference” from the retrieved location (co-locations I and III) into the same target location (bottom right, green). (b) Example neuron showing multiple operations. (c) Target-selective responses (“best” minus “others”) in trials with −90°, 0°, and 90° background cues for target-selective neurons (n = 72). Curves and shadings depict means and SEMs of population-averaged SDFs. Top lines, time duration in which target-selective responses (“best” minus “others”) were significantly positive in trials with 0° background cues (black) and with either −90° or 90° background cues (gray) (P < 0.05, t test, 2-tailed). Purple line, time duration in which the “best” target responses were significantly larger in trials with either −90° or 90° compared with 0° background cue (P < 0.05). The best target locations of the target-selective neurons in each hemisphere (animal) covered not only the contra-lateral side but also ipsi-lateral side (S2 Table). Source data are available in S1 Data. B-Cue, background cue; I-Cue, item cue; SDF, spike density function.
Construction of goal-directed information.
(a) Example neuron showing the “transference” effect ([I, 90°] and [III, −90°] for the bottom-right). Same format as Fig 5A. Bottom panel shows a schematic diagram of “transference” from the retrieved location (co-locations I and III) into the same target location (bottom right, green). (b) Example neuron showing multiple operations. (c) Target-selective responses (“best” minus “others”) in trials with −90°, 0°, and 90° background cues for target-selective neurons (n = 72). Curves and shadings depict means and SEMs of population-averaged SDFs. Top lines, time duration in which target-selective responses (“best” minus “others”) were significantly positive in trials with 0° background cues (black) and with either −90° or 90° background cues (gray) (P < 0.05, t test, 2-tailed). Purple line, time duration in which the “best” target responses were significantly larger in trials with either −90° or 90° compared with 0° background cue (P < 0.05). The best target locations of the target-selective neurons in each hemisphere (animal) covered not only the contra-lateral side but also ipsi-lateral side (S2 Table). Source data are available in S1 Data. B-Cue, background cue; I-Cue, item cue; SDF, spike density function.To examine the temporal relationship at the population level, we compared time courses of the 2 types of target-related effects for the target-selective neurons (n = 72) by examining the effect of background-cues in different orientations (−90°, 0°, and 90°) on the target-selective responses (Fig 6C). The target-selective responses in trials with the −90° and 90° background-cues became significantly larger than those with the 0° background cue from 309 to 786 milliseconds after the background-cue onset (Fig 6C). The increase in target-selective responses after the −90° and 90° background cues may reflect the transfer of the retrieved location into the preferred locations of individual HPC target-selective neurons (“transferring-type”). Then, the target-selective responses in trials with the 0° background-cue began to increase in the middle of delay 2, and the target-selective responses ultimately became indistinguishable among all the background-cues (Fig 6C), which may represent the target locations themselves (“targeting-type”). In trials with a 0° background cue, target-selective responses were observed not only during the background-cue period but also during the item-cue period (P < 0.05, t test, 2-tailed) (Fig 6C), which confirmed the presence of the default position/orientation of the background image for the representation of the retrieved item-location in the HPC. Considering the fact that the immediate background-cue effect converged on the retrieved location signal, these results suggest involvements of sequentially occurring neuronal operations (convergence, transference, and targeting) in the constructive process in which both memory and perception were combined to generate a goal-directed representation of the memory (S1 Table).
Neuronal signal predicts animals’ behavior
We finally investigated whether the target-selective responses in the HPC were correlated with subjects’ behaviors. For this purpose, we conducted an error analysis for the target-selective activities during the background-cue period. Fig 7A shows an example neuron exhibiting target-selective activities. This neuron showed strong responses during the background-cue period when the animal chose the top-left position (red) not only in the correct trials (Correct trials, red) but also in the error trials (False Alarm, black). In contrast, the neuron did not respond when the animal made mistakes by missing the top-left target position (Miss, gray). We examined whether the target-selective activities in the error trials could be explained by the positions the animals chose or the correct positions of the trials using partial correlation coefficients (see Materials and methods). The activities in the error trials of this neuron were related with the animals’ choice (r = 0.51, P < 0.0001, d.f. = 47) but not with the correct position (r = −0.18, P = 0.94). We calculated the partial correlation coefficients for the target-selective neurons with more than 10 error trials and found that the activities in error trials reflected the animals’ choice rather than the correct position (P < 0.0005, Wilcoxon’s signed-rank test) (Fig 7B). These results suggest that the target-selective activity constructed by the HPC neurons predicts the subsequent animal behavior.
Fig 7
Neuronal signal predicting animals’ behavior.
(a) Example neuron exhibiting target-selective activity. (Left) Raster displays of correct trials sorted by target locations. Colors indicate target locations on display. The neuron exhibited preferred responses when the target location was the top left (red). (Right) Raster displays of error trials and SDFs (σ = 20 milliseconds). False Alarm, top left as the incorrect positions the subjects chose (black). Miss, top left as the correct positions the subjects missed (gray). Correct trials, top left as the correct positions the subjects chose (red). (b) Error analysis for target-selective neurons with at least 10 error trials (n = 50). False Alarm, the false positions the subjects chose. Miss, the correct positions the subjects missed. Each dot indicates 1 neuron. ***P < 0.0005, Wilcoxon’s signed-rank test, 2-tailed. Source data are available in S1 Data. SDF, spike density function.
Neuronal signal predicting animals’ behavior.
(a) Example neuron exhibiting target-selective activity. (Left) Raster displays of correct trials sorted by target locations. Colors indicate target locations on display. The neuron exhibited preferred responses when the target location was the top left (red). (Right) Raster displays of error trials and SDFs (σ = 20 milliseconds). False Alarm, top left as the incorrect positions the subjects chose (black). Miss, top left as the correct positions the subjects missed (gray). Correct trials, top left as the correct positions the subjects chose (red). (b) Error analysis for target-selective neurons with at least 10 error trials (n = 50). False Alarm, the false positions the subjects chose. Miss, the correct positions the subjects missed. Each dot indicates 1 neuron. ***P < 0.0005, Wilcoxon’s signed-rank test, 2-tailed. Source data are available in S1 Data. SDF, spike density function.
Discussion
The present study aimed to investigate whether the flexible use of past knowledge can be explained by a constructive process in the HPC. We found a robust memory signal reflecting the location information retrieved from an item cue (Fig 2), which was substantial even after the onset of background cue (Fig 3, S5 Fig). The perceptual information of the background cue was converged on the retrieved location signal (Fig 4), which transferred the retrieved location to the target location (Figs 5 and 6). The target information was correlated with the animal’s subsequent behavioral response (Fig 7). The present findings thus indicate that the HPC neurons combine mnemonic information with perceptual information to construct goal-directed representations of the retrieved memory (Fig 8), which would be useful in the current situation for a subsequent action.
Fig 8
Constructive process for the flexible use of memory.
Schematic diagram of neuronal signals during a trial of the CMP task, in which the item cue and the orientation of background cue were I-B and 90°, respectively. In the HPC, the retrieved location of the item is represented relative to the 0° background image, which may correspond to the top right in egocentric space. The incoming perceptual signal is integrated with the memory signal to construct an updated information signaling the target location by following sequential neuronal operations: convergence (i.e., memory [co-location I on the 0° background] + perception [90° background]), transference (i.e., from the top right [co-location I on the 0° background] into the bottom right [co-location I on the 90° background]), and targeting (i.e., coding bottom right). It is still unknown which brain area is involved first in the retrieval of item-location association memory and whether the retrieved memory content is same as the memory signal in the HPC. CMP, constructive memory-perception; HPC, hippocampus.
Constructive process for the flexible use of memory.
Schematic diagram of neuronal signals during a trial of the CMP task, in which the item cue and the orientation of background cue were I-B and 90°, respectively. In the HPC, the retrieved location of the item is represented relative to the 0° background image, which may correspond to the top right in egocentric space. The incoming perceptual signal is integrated with the memory signal to construct an updated information signaling the target location by following sequential neuronal operations: convergence (i.e., memory [co-location I on the 0° background] + perception [90° background]), transference (i.e., from the top right [co-location I on the 0° background] into the bottom right [co-location I on the 90° background]), and targeting (i.e., coding bottom right). It is still unknown which brain area is involved first in the retrieval of item-location association memory and whether the retrieved memory content is same as the memory signal in the HPC. CMP, constructive memory-perception; HPC, hippocampus.In previous electrophysiological studies, the effects of the item-location association memory were presented as learning-dependent changes in firing rates (e.g., changing cells) [8, 26] or selective responses to a particular combination of items and locations [26, 27, 30]. However, these studies did not identify the location signal retrieved from an item cue. In the present study, we evaluated the item-location association memory as correlated responses to the co-location items by assigning 2 visually distinct items to each co-location on the background image (Fig 1). We found that the HPC neurons showed the unitized responses to the co-location items, reflecting the location retrieved from the item cues (Figs 2 and 3). In addition, by orienting the background cue randomly, we dissociated the item-location association memory from the item-response association [3] in the CMP task, which was not clearly dissociated in the previous physiological studies [8, 26, 27]. Together, the CMP task allowed us to examine the correlated responses to the items, which were semantically linked via the location information. The correlated responses to semantically linked items were previously investigated using the “pair-association” task [31-33]. Different from the co-location items linked by the locations in the CMP task, a pair of items was directly associated with each other in the pair-association task. In this item-item association memory paradigm, the memory retrieval signal representing a target item appeared first in the perirhinal cortex (PRC) of the MTL and spread backward to the visual area TE [31, 34, 35]. Future studies should aim to determine whether the location information is retrieved within the HPC [36] or derives from other areas such as the PRC (Fig 8), which has been considered as a core brain region responsible for semantic dementia [37, 38] as well as a hub of converging sensory inputs [34, 39].In the CMP task, the sequential presentations of the item and background cues temporally separated the memory retrieval from perceiving the environment and allowed us to observe neuronal dynamics that may underlie the constructive process fitting the retrieved memory to the current situation (Fig 8). Sequential presentations of 2 cue stimuli were also applied in previous studies to investigate a conjunctive representation of the 2 stimuli in the HPC of rats [13] and in the PRC of nonhuman primates [40]. In these studies, animals learned to associate combinations of 2 cues with choice responses [13] or reward deliveries [40] directly because the 2 cue stimuli (e.g., sound and odor) did not have any internal relationship and the combinations of 2 cue stimuli were arbitrarily assigned to the correct responses (e.g., pulling right/left lever) or outcomes (e.g., presence/absence of reward). In this condition, the memory retrieval signal appeared after the second cue [13, 40]. Conversely, the combinations of the item and background cues in the CMP task necessarily determined the correct target positions because the item cue was assigned to a particular position on the background-cue image in allocentric coordinate (Fig 1B). To realize this experimental design in an actual animal experiment, we trained the animals on the preliminary stimulus set (S7 Fig) to train the task rule in a relatively easy condition before the main stimulus set (S1 Text, S3 Table). This training procedure would prevent an animal from learning to “solve” a task in a gradual manner like probabilistic learning or inflexible learning of amnesiapatients, in which participants learned to “solve” the tasks implicitly, and their performances were supported by the striatum or neocortical areas rather than the hippocampus [41, 42]. The present experimental design, including the training procedure, was validated by the correlated responses of the HPC neurons with the animal’s subsequent behavioral response suggesting involvements of the HPC in the CMP.In addition to the constructive process, the present experimental design revealed the default position/orientation for the background image in the HPC when the animals represented the retrieved location of the item cue in their mind (Fig 8). Considering the training history of the monkeys for the item-location association, the default position may depend on their initial trainings. This finding may explain our mental representations of landmarks for their locations, which depends on our experiences [43]. For example, when you remember locations of the Statue of Liberty and Brooklyn Bridge in the New York City, you may automatically put them to the bottom and right, respectively, in your mind. An unanswered question in the present study is whether the mental locations were coded in egocentric space relative to the animals’ head position [45] or in allocentric space relative to a frame of computer screen for the background image [20, 44] or maybe both [46].The presence of the default position/orientation for the background image suggested that the constructive process for the goal-directed information was triggered by an update of the background from its default to a current position/orientation during a trial of the CMP task. The HPC neurons carried both past and present information on their activity. This convergent-type activity may lead the targeting-type activity via the transferring-type activity when the background cue was different from the default position/orientation (i.e., “non-match” condition). These sequentially occurring neuronal operations may be useful to construct (cf., retrieve) a target from multiple signals that have an internal relationship (e.g., the retrieved co-location and the background cue in the CMP task). For example, you may answer a direction of the school gymnasium from your current position/orientation in the campus easily even though you do not have a direct experience to go there from your current position [3, 16, 47].One reasonable question here might be whether the retrieved location was transferred to the target position by a mental rotation [48] of the retrieved location on the background image. If we assume the target information was constructed from the convergent-type of activity including the current background-cue information, which was already oriented, it would be reasonable to consider that the retrieved location was transferred to the target position directly without a transit between the 2 positions in geometric space. The direct transfer may occur only when a subject is familiar with a current environment, in which enough information could be provided to compute the target location accurately in the HPC. Conversely, when the environment is not enough familiar, the subject may recruit an active simulation process like mental rotation, which would be supported by other cortical areas [49-51].Another unanswered question is whether the target-location-selective activity in the HPC encodes an action plan (i.e., endpoint of the saccade) or a mental representation of a target location itself. In the contextual fear memory paradigm [52], the HPC provides the amygdala with context information rather than its associated valence triggering fear responses [53]. Moreover, recent human functional magnetic resonance imaging (fMRI) studies reported activation of the default mode network during future simulations [54] and suggested that the retrieved information spreads from the MTL to medial prefrontal cortex (PFC) [47], which is reportedly involved in decision-making [55]. On the basis of these findings, we hypothesize that the HPC may provide its downstream regions with a target location, which may guide subsequent action selection [56].In the present CMP task, we found the 3 neuronal operations in the HPC that were involved in the construction of goal-directed information. Constructive neural processing is best recognized in the visual system [57]. Based on the anatomical hierarchy, the construction proceeds from the retina to HPC through a large number of distinct brain areas to construct a mental image of an entire visual scene from local visual features (e.g., light spots, oriented bars). A recent electrophysiological study demonstrated constructive perceptual processing in the MTL, which combines an object identity with its location when the monkeys look at the visual object [20, 58]. However, a constructive process for perceiving an entire scene is still an unsolved question. As to the memory system, Schacter and his colleagues proposed the “constructive episodic simulation hypothesis” [1, 59], which assumes that our brain recombines distributed memory elements to construct either past episodes or future scenarios (i.e., “mental time travel”) [24, 25]. However, the neuronal correlates to the constructive memory process for the mental time travel have not been identified as far as we know. There are preceding studies reporting neuronal activities signaling both past and future regarding the performance level during a single recording session in the macaque prefrontal cortex [60] or the task events during a single trial in the rat HPC [13], but neither showed the constructive process. In the present study, we exhibited a constructive process in which HPC neurons combined the past knowledge with incoming perception for its flexible use rather than for perception of an entire scene or for mental time travel.Considering its functional significance as declarative memory, the constructive process operated by the 3 neuronal operations for the flexible use of mnemonic information in the HPC may be shared across species. Moreover, this constructive process combining both memory and perception might be a precedent of the constructive memory process combining only the memory elements for the “mental time travel” in the evolution process of declarative memory system. The underlying neuronal mechanisms of the constructive process in the HPC should be further investigated by theoretical and experimental study across species. The transitions of the 3 neuronal operations for the constructive process might be related with attractor dynamics substantiated by the HPC recurrent networks, which is reportedly involved in spatial memory of rodents [61-64].
Materials and methods
Ethics statements
The experiments were performed in accordance with the National Institutes of Health (NIH) Guide for the Care and Use of Laboratory Animals and were approved by the Institutional Animal Care and Use Committee (IACUC) of Peking University (Psych-YujiNaya-1).
Experimental design
Subjects
The subjects were 2 adult male rhesus monkeys (Macaca mulatta; 6.0–9.0 kg).
Behavioral task
We trained 2 monkeys on a CMP task (Fig 1, S1 Text, S7 Fig and S3 Table). During both training and recording sessions, animals performed the task under dim light. The task was initiated by the animal fixating on a white square (0.5° visual angle) in the center of a display for 0.5 seconds. Eye position was monitored by an infrared digital camera with a 120-Hz sampling frequency (ETL-200, ISCAN). Then, an item cue (diameter, 3.8°) and background cue (diameter, 31.4°) were sequentially presented for 0.3 seconds each with a 0.7-second interval. After an additional 0.7-second delay interval, 4 equally spaced white squares (0.5°) were presented at the same distance from the center (8.5°) as choice stimuli. One of the squares was a target, whereas the other 3 were distracters. The target was determined by a combination of the item cue and the background cue stimuli. The animals were required to saccade to one of the 4 squares within 0.5 seconds. If they made the correct choice, 4 to 8 drops of water were given as a reward. When the animals failed to maintain their fixation (typically less than 2° from the center) before the presentation of choice stimuli, the trial was terminated without reward. Before the recording session, we trained the animals to associate 2 sets of 4 visual stimuli (item cues) with 4 particular locations relative to the background image that was presented on the tilt with an orientation from −90° to 90°. We first trained the monkeys to learn the task rule of the CMP task using a preliminary stimulus set (monochromatic simple-shaped objects [e.g., cross, heart] as item stimuli and a large disk with 4 monochrome colors in individual quadrants as the background stimulus) (S7 Fig) in the preliminary training before the final training using a main stimulus set (S1 Text). In addition, to avoid that the monkeys learn to associate each combination of the item cue and the background cue with a particular target location, the orientation of background image was randomized at a step of 0.1°, which increased the number of combinations (8 × 1,800) and would make it difficult for the animals to learn all the associations among item cues, background cues, and target locations directly (S1 Text). During the recording session, the item cue was pseudorandomly chosen from the 8 well-learned visual items, and orientation of the background cue was pseudorandomly chosen from among 5 orientations (−90°, −45°, 0°, 45°, and 90°) in each trial, resulting in 40 (8 × 5) different configuration patterns. We included the trials with −45° and 45° background cues during the recording session in order to increase the number of configuration patterns and to prevent the animals from linking a combination of the item cue and the background cue to the target location directly, although we did not use these trials in the main analyses. We trained the 2 monkeys using same stimuli but different item-location association patterns. All stimulus images were created using Photoshop (Adobe, https://www.adobe.com/).
Electrophysiological recording
Following initial behavioral training, animals were implanted with a head post and recording chamber under aseptic conditions using isoflurane anesthesia. To record single-unit activity, we used a 16-channel vector array microprobe (V1 X 16-Edge; NeuroNexus) or a single-wire tungsten microelectrode (Alpha Omega), which was advanced into the brain by using a hydraulic Microdrive (MO-97A; Narishige) [11]. The microelectrode was inserted through a stainless steel guide tube positioned in a customized grid system on the recording chamber. Neuronal signals for single units were collected (low-pass, 6 kHz; high-pass, 200 Hz) and digitized (40 kHz) (AlphaLab SnR Stimulation and Recording System, Alpha Omega, https://www.alphaomega-eng.com/). We made no attempt to prescreen isolated neurons. Instead, once we succeeded in isolating any neuron online, we started a new recording session. The offline isolation of single units was performed using Offline Sorter (Plexon, https://plexon.com/) by manual curation to make sure that noise transients were not included as units and that the same cell was not split into several clusters. The cells were isolated depending on the properties of spike waveforms. The cells were included into the analysis if the cells fired throughout the recording session with well-defined fields and a minimal mean firing rate as 1 Hz. On average, 128 trials were tested for each neuron (n = 456). The placement of microelectrodes into target areas was guided by individual brain atlases from MRI scans (3T, Siemens). We also constructed individual brain atlases based on the electrophysiological properties around the tip of the electrode (e.g., gray matter, white matter, sulcus, lateral ventricle, and bottom of the brain). The recording sites were estimated by combining the individual MRI atlases and physiological atlases [65].The recording sites covered between 3 and 16 mm anterior to the interaural line (monkey B, left hemisphere; monkey C, right hemisphere; S2 Fig). The recording sites cover all the subdivisions of the HPC (i.e., dentate gyrus, CA3, CA1, and subicular complex) [11]. A final determination will require future histological verification (both animals are currently still being used).
Statistical analysis
All neuronal data were analyzed by using MATLAB (MathWorks, https://www.mathworks.com/) with custom written programs, including the statistics toolbox.
Classification of task-related neurons during the item-cue period
We calculated mean firing rates of 8 consecutive 300-millisecond time-bins moving in 100-millisecond steps, covering from 0 to 1,000 milliseconds after item-cue onset in each of all correct trials. We evaluated the effects of “item” for each neuron by using 1-way ANOVA with the 8 item-cue stimuli as a main factor (P < 0.01, Bonferroni correction for 8 analysis-time windows). We referred to neurons with significant item effects during any of the 8 analysis-time windows as item-selective neurons. For a comparison of the item-selective activity between the item-cue period and the background-cue period, we also defined the item-selective neurons during the background-cue period (Fig 3).
Classification of task-related neurons during the background-cue period
We calculated the mean firing rates of 8 consecutive 300-millisecond time-bins moving in 100-millisecond steps, covering from 0 to 1,000 milliseconds after the background-cue onset. We evaluated the effects of “co-location,” “background,” and “target” for each neuron by using 3-way nested ANOVA with the 4 co-locations, 3 background-cue orientations, and 4 target locations as main factors, and the 8 item-cues nested under the co-locations (P < 0.01, Bonferroni correction for 8 analysis-time windows). The 3-way nested ANOVA was conducted using the correct trials with −90°, 0°, and 90° background-cues. −45°, and 45° background-cues were excluded from the ANOVA because they would bring about a bias for the target location (S3 Text). For a comparison of the co-location-selective activity between the item-cue period and the background-cue period, we also defined the co-location-selective neurons during the item-cue period using 3-way ANOVA (S5A Fig). Out of the task-related neurons defined by the 3-way nested ANOVA, we further defined a neuron showing both co-location and background effects (“convergence”) during the background-cue period and 2 subcategories of the target-selective neurons (“transference” and “targeting”) during the background-cue period (S1 Table).
Analysis of retrieval signal during item-cue period
To show the time course of activity for an individual item-selective neuron, an SDF was calculated using only correct trials and was smoothed using a Gaussian kernel with a sigma of 20 milliseconds.We examined the retrieval signal of each item-selective neuron by calculating Pearson correlation between the responses to the co-location items {i.e., [f (I-A), …, f (IV-A)] and [f (I-B), …, f (IV-B)]} (4 pairs, d.f. = 2) (S4 Fig). Here, f (I-A) indicates the response to the item I-A. Because individual neurons in the HPC showed various time courses of item-selective activities (e.g., Figs 2A and 3A), we calculated the correlation coefficients during each of the analysis-time windows with a significant item effect for each item-selective neuron (P < 0.01, Bonferroni correction for 8 analysis-time bins). We then averaged Z-transformed values of the correlation coefficients across the significant analysis-time bins for the neuron. The average value was finally transformed into r value (i.e., co-location index) as shown in Figs 2 and 3.The retrieval signal was further examined for each item-selective neurons with high co-location index (r > 0.6) using the ROC analysis [29, 66]. We calculated a mean firing rate during the item-cue period (60 to 1,000 milliseconds from the item-cue onset) in each trial for the optimal item and its paired co-location items. Cumulative proportions of the trials whose firing rates were larger than a criterion were depicted on the 2-dimensional plot with the optimal item and its paired item as the ordinate and abscissa, respectively. A value of the AUC was evaluated for each neuron by a permutation test. We shuffled the trials for the optimal item and its paired co-location item 10,000 times. At each shuffle, we determined the optimal item and its paired item according to their firing rates and calculated a value of the AUC. Using a distribution of the 10,000 values of the AUC, we determined an expected value (median) and the significance level for each neuron. As a control, we also examined a discrimination between the trials of the best co-locations, including the optimal and its paired co-location items, and the trials of the other co-locations and evaluated a value of the AUC using the permutation test. The ROC analysis was also applied for the item-selective neurons during the background-cue period in the same way.
Analysis of task-related signal during background-cue period
To show the time course of activity for an individual task-related neuron, an SDF was calculated using only correct trials and was smoothed using a Gaussian kernel with a sigma of 20 milliseconds. For comparing time courses of proportions of task-related (co-location, background, and target) neurons and their signal amplitudes, we conducted 3-way nested ANOVA for each 100-millisecond time-bin moving by 1 millisecond to test significances (P < 0.01, uncorrected) with F values for each neuron. The 3-way nested ANOVA was conducted using only the correct trials with −90°, 0°, and 90° background-cues.
Analysis of similarity of orientation tuning
To evaluate the effects of background cue on the co-location-selective responses, we used data in the correct trials with −90°, 0°, and 90° background cues. As to the example neuron in Fig 4B, we first calculated mean firing rates for each co-location during the 60- to 1,000-millisecond period from an onset of the background cue and determined the “best co-location” and the “second-best co-location” based on the mean firing rates. We then calculated Pearson correlation between responses to the different orientations (−90°, 0°, and 90°) of background cues for the best co-location and those for the second-best co-location (3 pairs, d.f. = 1). We also examined a time course of the background-cue effect on the co-location-selective responses by calculating the population-averaged correlation coefficients for each 100-millisecond time-bin moving by 1 millisecond as shown in Fig 4C.
Calculation of matching index
To evaluate the relationship between the retrieved location and the target location, we used data in the correct trials with −90°, 0°, and 90° background cues. For each neuron exhibiting both item-selectivity during item-cue period and target-selectivity during background-cue period, we first averaged responses during the 60- to 1,000-millisecond period from item-cue onset in each trial and calculated a grand mean across trials to each of the 4 co-locations. In addition, we averaged responses during the 60- to 1,000-millisecond period from background-cue onset in each trial and calculated a grand mean across trials to each of the 4 target locations. According to the 3 potential matching patterns, we sorted the firing rates to the co-locations and calculated Pearson correlation coefficients between responses to the co-locations in each of the 3 potential matching patterns {e.g., [fic (I), fic (II), fic (III), fic (IV)] on 0° background cue} and those to the target locations [fbc (TR), fbc (BR), fbc (BL), fbc (TL)] (4 pairs, d.f. = 2). Here, fic (I) indicated an averaged response to the items corresponding to co-location I during the item-cue period, and fbc (TR) indicated an averaged response to the top-right target position during the background-cue period.
Analysis of target signal
To evaluate background-cue effect on the target signal, population-averaged SDFs (best–other target locations) were calculated for target-selective neurons across the correct trials with −90°, 0°, and 90° background cues. We first averaged responses during the 60- to 1,000-millisecond period from background-cue onset in each trial and calculated a grand mean across correct trials to each of the 4 target locations to determine the “best target location” for each neuron. The SDFs to each orientation (−90°, 0°, and 90°) of background cues for all target locations were normalized to the amplitude of the mean response to the best target location, and the normalized SDFs for the best target location was subtracted by the mean normalized responses across the other target locations. The population-averaged SDFs (i.e., target-selective response) were smoothed using a Gaussian kernel with a sigma of 20 milliseconds.
Error analysis
We examined whether the target-selective activities signaled positions the subjects chose or correct positions during the background-cue period in error trials by using a partial correlation coefficient. To calculate the partial correlation coefficient for each neuron, we first calculated an average firing rate during each 300-millisecond time-bin moving by 100 milliseconds during the background-cue period (i.e., 8 time-bins in total) for each target position (i.e., 8 positions in total) across the correct trials. We next prepared for 3 arrays for each neuron containing “n” elements in each array (“n” is the number of error trials for each neuron): (1) firing rates in the i-th error trial (i = 1 to n) (dependent variable, ); (2) the mean firing rate across correct trials with the same target position as the subject chose in the i-th error trial (explanatory variable, ); (3) the mean firing rate across correct trials with the same target position as the subject missed (i.e., correct answer) in the i-th error trial (explanatory variable, ). The partial correlation coefficients of the dependent variable, , with explanatory variables, and , were calculated in each time-bin for each neuron when the neuron’s responses in correct trials showed a significant target effect (P < 0.01, Bonferroni correction for 8 analysis-time windows) and the mean firing rate across trials was larger than 1 Hz in that time-bin. The mean partial correlation coefficients were calculated across the active time bins (i.e., P < 0.01, Bonferroni correction for 8 analysis-time windows, >1 Hz) for each neuron using Z-transformation.
Training procedures for the CMP task.
CMP, constructive memory-perception.(DOCX)Click here for additional data file.
Examinations of neuronal responses to co-location stimuli and eye positions.
(DOCX)Click here for additional data file.
Detection of task-related signals during the background-cue period.
(DOCX)Click here for additional data file.
Numbers of task-related neurons.
(DOCX)Click here for additional data file.
Numbers of target-selective neurons selective to each target location.
(DOCX)Click here for additional data file.
Numbers of training sessions for the CMP task.
CMP, constructive memory-perception.(DOCX)Click here for additional data file.
Performance in the CMP task.
Performance during recording sessions (n = 179 for Monkey B, n = 158 for Monkey C). Error bar, standard deviation. Dashed line, chance level = 25%. (a) Performance for 8 item stimuli as item cue. Black bars, set A. White bars, set B. (b) Performance for 5 orientations of background cue. (c) Performance for 8 positions on the display as target locations. Source data are available in S2 Data. B, bottom; BL, bottom left; BR, bottom right; CMP, constructive memory-perception; L, left; R, right; T, top; TL, top left; TR, top right.(TIF)Click here for additional data file.
Recording region.
Magnetic resonance images corresponding to the coronal planes anterior 4 and 10 mm from the interaural line of monkey C (right hemisphere). The recording region is the HPC. A reference electrode implanted in the center of chamber was observed as a vertical line of shadow in the coronal plane at A10. D, dorsal; HPC, hippocampus; L, lateral. M, medial; ots, occipital temporal sulcus; V, ventral.(TIF)Click here for additional data file.
Examinations of neuronal responses and eye positions.
Firing rates plotted as a function of eye positions during the item-cue period for the neuron shown in Fig 2A and 2B. Each circle indicates 1 trial. Filled circles indicate trials with the best co-location stimuli as item cues. Open circles indicate trials with the worst co-location stimuli as item cues. The large overlaps were found in the distributions of the eye positions between the trials with the best and worst co-location items (P = 0.16 for horizontal, P = 0.25 for vertical, t test, 2-tailed), whereas distributions of the firing rates were significantly different between the 2 trial types (P < 0.0001). These results indicate that the item-selective responses shown in Fig 2 cannot be explained by the animal’s eye positions. Source data are available in S2 Data.(TIF)Click here for additional data file.
A schematic illustration of co-location index.
The co-location index was calculated for each neuron as following 5 steps. (1) The item-selectivity was examined in each of 8 consecutive 300-millisecond time-bins using the same threshold (i.e., P < 0.0125 for each time-bin, 1-way ANOVA) as that for the definition of item-selective neurons (P < 0.01, Bonferroni correction for 8 analysis time-bins). (2) If the time-bin showed a significant item-selectivity, we calculated the correlation coefficient (r) between the responses to items from set A and those from set B in the time-bin, and (3) then the r was transformed into Z. (4) We then averaged Z values across the significant time-bins and (5) re-transformed the averaged Z value into value as the co-location index of the neuron.(TIF)Click here for additional data file.
Co-location-selective neurons during the item-cue and background-cue periods.
(a) Percentages of neurons showing a significant co-location effect (P < 0.01, 3-way nested ANOVA) during the item-cue (I-Cue) and background-cue (B-Cue) periods out of the recorded neurons (n = 456). Hatched area, neurons exhibiting co-location-selectivity during both periods. (b) Time courses of percentages of neurons showing significant co-location-selective activity and background-selective activity (100-millisecond time bin, P < 0.01, 3-way nested ANOVA, uncorrected) out of the recorded neurons (n = 456). Brown bar, presentation of the item-cue. Gray bar, presentation of the background cue. Dashed line, chance level = 1%. Source data are available in S2 Data.(TIF)Click here for additional data file.
Example neurons involved in the constructive process.
(a) Example neuron signaling co-location and background cue in a “convergent” manner. This neuron did not show item-cue selective responses during the item-cue period (P = 0.34, 1-way ANOVA), but it exhibited the co-location-selective responses during the background-cue period (P < 0.01, 3-way nested ANOVA). The background-selective responses were combined with the co-location-selective responses. The preferred orientation of the background-cue stimulus was 90° for this neuron across co-locations. The same format as Fig 5A. (b) Example neuron signaling a “targeting” location. This neuron did not show item-selective responses during the item-cue period (P = 0.81), but it exhibited the target-selective responses during the background-cue period (P < 0.0001). The best target location was bottom-right of display (green). (c) Example neuron that changed the activity patterns, showing multiple operations for the construction (i.e., convergence, transference, and targeting) during the background-cue period. This neuron showed item-selective responses during the item-cue period (P < 0.0001), and the preferred items in the item-cue period were I-A, I-B, IV-A, and IV-B. During 300–400 milliseconds after background-cue onset, the background-selective responses were combined with the co-location-selective responses, and the preferred orientation was −90° across co-locations (i.e., “convergence”). During 400–600 milliseconds after background-cue onset, this neuron exhibited strong responses only to the particular combinations of item cue and background cue that corresponded to the top-right target position (blue disk) (blue bars, II-A and II-B, −90° and IV-A and IV-B, 90°) (i.e., “transference”). During 800–1,000 milliseconds after background-cue onset, this neuron exhibited selective responses to the top-right target position regardless of item and background cues (i.e., “targeting”). Source data are available in S2 Data.(TIF)Click here for additional data file.
Stimuli for preliminary training of the CMP task.
(Left) Simple shape objects with monochrome colors as item-cue stimuli. (Right) Large disk with 4 monochrome colors in individual quadrants as a background-cue stimulus. Each item stimulus was assigned to 1 location on the background image. CMP, constructive memory-perception.(TIF)Click here for additional data file.
Source data for the main figures.
The source data used to generate main figures are included under the file name “S1_Data.xlsx.” Source data for each main figure are arranged by sheet and are labeled. The raw spike files for each neuron are available at https://osf.io/nu9ch/?view_only=1faa4cc2d5254b6eb25740a92e6f693c.(XLSX)Click here for additional data file.
Source data for the supplementary figures.
The source data used to generate supplementary figures are included under the file name “S2_Data.xlsx.” Source data for each main figure are arranged by sheet and are labeled. The raw spike and eye position files for each neuron are available at https://osf.io/nu9ch/?view_only=1faa4cc2d5254b6eb25740a92e6f693c.(XLSX)Click here for additional data file.22 Apr 2020Dear Dr Naya,Thank you for submitting your manuscript entitled "Hippocampal cells integrate past memory and present perception for the future." for consideration as a Research Article by PLOS Biology.Your manuscript has now been evaluated by the PLOS Biology editorial staff, as well as by an academic editor with relevant expertise, and I'm writing to let you know that we would like to send your submission out for external peer review. Please accept my apologies for the delay incurred while we sought external advice during these challenging times.However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.Please re-submit your manuscript within two working days, i.e. by Apr 24 2020 11:59PM.Login to Editorial Manager here: https://www.editorialmanager.com/pbiologyDuring resubmission, you will be invited to opt-in to posting your pre-review manuscript as a bioRxiv preprint. Visit http://journals.plos.org/plosbiology/s/preprints for full details. If you consent to posting your current manuscript as a preprint, please upload a single Preprint PDF when you re-submit.Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.Given the disruptions resulting from the ongoing COVID-19 pandemic, please expect delays in the editorial process. We apologise in advance for any inconvenience caused and will do our best to minimize impact as far as possible.Feel free to email us at plosbiology@plos.org if you have any queries relating to your submission.Kind regards,Roli RobertsRoland G Roberts, PhD,Senior EditorPLOS Biology2 Jun 2020Dear Dr Naya,Thank you very much for submitting your manuscript "Hippocampal cells integrate past memory and present perception for the future." for consideration as a Research Article at PLOS Biology. Your manuscript has been evaluated by the PLOS Biology editors, an Academic Editor with relevant expertise, and by four independent reviewers.You’ll see that the reviewers are broadly positive, but that both reviewers #3 and #4 raise some significant conceptual and analytical issues. There seems to be a common theme that the paper lacks clarity, and would benefit from clearer explanations and presentation of additional (existing) data and some new analyses.In light of the reviews (below), we are pleased to offer you the opportunity to address the comments from the reviewers in a revised version that we anticipate should not take you very long. We will then assess your revised manuscript and your response to the reviewers' comments and we may consult the reviewers again.We expect to receive your revised manuscript within 1 month, but do let us know if you require more time, especially under the current circumstances.Please email us (plosbiology@plos.org) if you have any questions or concerns, or would like to request an extension. At this stage, your manuscript remains formally under active consideration at our journal; please notify us by email if you do not intend to submit a revision so that we may end consideration of the manuscript at PLOS Biology.**IMPORTANT - SUBMITTING YOUR REVISION**Your revisions should address the specific points made by each reviewer. Please submit the following files along with your revised manuscript:1. A 'Response to Reviewers' file - this should detail your responses to the editorial requests, present a point-by-point response to all of the reviewers' comments, and indicate the changes made to the manuscript.*NOTE: In your point by point response to the reviewers, please provide the full context of each review. Do not selectively quote paragraphs or sentences to reply to. The entire set of reviewer comments should be present in full and each specific point should be responded to individually.You should also cite any additional relevant literature that has been published since the original submission and mention any additional citations in your response.2. In addition to a clean copy of the manuscript, please also upload a 'track-changes' version of your manuscript that specifies the edits made. This should be uploaded as a "Related" file type.*Resubmission Checklist*When you are ready to resubmit your revised manuscript, please refer to this resubmission checklist: https://plos.io/Biology_ChecklistTo submit a revised version of your manuscript, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' where you will find your submission record.Please make sure to read the following important policies and guidelines while preparing your revision:*Published Peer Review*Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:https://blogs.plos.org/plos/2019/05/plos-journals-now-open-for-published-peer-review/*PLOS Data Policy*Please note that as a condition of publication PLOS' data policy (http://journals.plos.org/plosbiology/s/data-availability) requires that you make available all data used to draw the conclusions arrived at in your manuscript. If you have not already done so, you must include any data used in your manuscript either in appropriate repositories, within the body of the manuscript, or as supporting information (N.B. this includes any numerical values that were used to generate graphs, histograms etc.). For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5*Blot and Gel Data Policy*We require the original, uncropped and minimally adjusted images supporting all blot and gel results reported in an article's figures or Supporting Information files. We will require these files before a manuscript can be accepted so please prepare them now, if you have not already uploaded them. Please carefully read our guidelines for how to prepare and upload this data: https://journals.plos.org/plosbiology/s/figures#loc-blot-and-gel-reporting-requirements*Protocols deposition*To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosbiology/s/submission-guidelines#loc-materials-and-methodsThank you again for your submission to our journal. We hope that our editorial process has been constructive thus far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.Sincerely,Roli RobertsRoland G Roberts, PhD,Senior EditorPLOS Biology*****************************************************REVIEWERS' COMMENTS:Reviewer #1:The authors challenged to uncover the neuronal mechanisms underlying the flexibility of declarative memory, i.e., how to flexibly use stored information in changing situations. They recorded single-neuronal activity from the monkey's hippocampus when it was performing a newly developed task, in which retrieval of item-location association memory and flexible use of it was required. They showed that hippocampal neurons were activated by both mnemonic information representing the location and perceptual information representing the external circumstance. They suggest that the two signals were combined at a single-neuron level in the hippocampus and construct goal-directed information by sequentially occurring convergence, transference, and targeting.The motivation of this study inspired by "constructive episodic memory system" is sound and challenging. The experimental designs including the new behavioral task (CMP task) in particular are thoughtful and the data analysis is comprehensive, just like the previous excellent studies of Dr. Naya. I have only a few minor comments about discussion, that is, about interpretation of the present data and possible hypothesis for future studies of declarative memory.Comment1. In the CMP task, the "transference" process from the retrieved location into a target location might be a type of "mental rotation", i.e., rotating the retrieved cue location in a mental image to the direction indicated by the background cue. Though the authors already discussed similar issues in lines 258-276, a brief statement discussing the relation of their data and mental rotation in HPC and/or related areas with some references, if possible, may be helpful for the readers.2. It might be helpful if the authors would present a summary graph or table of total proportions of the task-related specific neurons, i.e., neurons representing retrieved memory (Fig.2), neurons representing convergence (Fig.3), neurons representing retrieved and target locations (Fig.4), neurons constructing goal-directed information (showing transference effect) (Fig.5), and neurons showing multiple operations (Fig.5). This summary could help us to make an image of information processing of declarative memory in neurons and neuronal population levels in HPC. I expect that the numbers of these neurons are different in a systematic way and more discussion will be possible in future studies.3. The authors reported that the signals of mnemonic information and perceptual information were combined at a single-neuron level in HPC. The mechanism of such convergence of different information for past and present to a single neuron is the great finding by the authors. On the other hand, as a conceptual assumption, convergence of different elementary information could be realized by co-activation of neurons related to elementary information alone. It might be helpful for me and the readers if the authors would add a brief discussion of how the mechanism of convergence to individual neurons (convergence neurons) is advantageous to realize the dynamic features of declarative memory.Reviewer #2:In this paper, Yang and Naya investigated hippocampal neuronal activities related to past memory and future goals. They developed a new task that requires the monkeys to associate items and locations in the background rotated trial by trial. They found that hippocampal neurons displayed the information of the item and target locations during the task. They also demonstrate the transitions of neuronal representations from item cue to direction cue presentations. The behavioral and analysis methods are appropriate and persuasive. The findings in this paper are significant and will be of broad interest. I think that the manuscript is suitable for publication in PLoS Biology. I have several comments which may improve the manuscript:- They classified the target-selective neurons in two types: 'targeting-type' (Fig 4) and 'transferring-type' (Fig 5). The 'targeting-type' neurons were activated when the direction difference between the retrieved location (co-location) and target direction was 0 degree. On the other hand, the 'transferring-type' neurons were activated when the target directions were 90 and -90 degrees from the retrieved location. However, the descriptions of these two types are sometimes confusing in the manuscript. For example, the sentence "these (= item and target selective) neurons tended to show the preferred target locations that corresponded to the preferred co-locations relative to the 0° background-cue (default orientation), but not to the other orientations (Fig 4c)" sounds contradictory because the transferring-type neurons preferred the opposite. (This seems because the analyzed populations for 'transferring-type' and 'targeting-type' are different, but the reason why they selected the different populations for these two types is not clear.) More structural and comprehensive explanations and statistical analysis of these two types of neurons would help readers to understand the importance of the results. Also, adding the numbers of 'targeting-type' and 'transferring-type' neurons in S1 Table would be informative.- Fig 3a, b. The spike density functions (top) do not match the results of the F-values (middle) and the firing rates (bottom). For example, SDF shows almost no firing activity in the trials of the item II-AB, but the bar graph shows high firing rates in Fig 3a. It seems that the SDFs of Fig 3a and b were swapped by mistake.Reviewer #3:[IDs himself as Mingsha Zhang]In the present study, authors trained two monkeys to perform a newly designed task, namely constructive memory-perception task (CMP), in which two visual stimuli were presented sequentially. The crucial point is that the second stimulus was oriented randomly between -90 and 90 degree as the 0 degree being defined as the vertical meridian. Monkeys need to remember the allocentric location of the first stimulus (item-cue) within the second stimulus (background-cue), and determine the goal location of response (saccade) according to the orientation of the background-cue for each individual trial. In this way, authors are able to separate the process of retrieval of remembered item location from the process of perception to current visual stimulus. Then authors explored the neuronal substrate of flexibly using of long-term declarative memory in monkey's hippocampal cortex. They found that hippocampal neurons signaled both mnemonic information, which represented the retrieved item location, and perceptual information, which represented the external circumstance. Thus, authors argue that the hippocampal cortex equips the declarative memory with flexibility in its usage by the constructive process combining memory and perception. While the topic of this study is important and interesting for readers with broad background, there are issues in both conceptual and data analysis that need authors to concern.Major comments:1. Up to my understanding, the long-term declarative memory mentioned in this paper is the spatially-associated memory between item-cue and background-cue. Because two different spatial coordinates were employed in the CMP task, i.e., allocentric coordinate for localizing the item-cue's position within the background-cue; egocentric coordinate for localizing the position of saccadic target, it will be easier for readers to catch up the logic of the task design if authors explicitly describe the exact meaning of item associated location.2. A key argument of this paper is that, after the item-cue (I-cue) onset, the activity of HPC neurons represents the retrieved item-location association memory. However, the results of signal neuronal activity (fig 2a) and population neuronal activity (fig 2c) show significant difference between the optimal and pair items, which does not support above argument. Since these two items share the same location, the neuronal response to these items should be similar if it really represents the memorized item associated location.3. When looking up the results of fig 2d, there is a subset of hippocampal neurons showing the high correlation coefficient activity between optimal item and its pair item. Suggesting authors to analyze the response of these neurons to the optimal and pair items to see whether the activity is similar.4. The CMP task is difficult. To understand the electrophysiology data properly, I would like to suggest authors to present data of monkeys' behavior performance in the results section.Minor comments1. Line 27-29, the words of "south" "north" "turn left" mix the concept of allocentric and egocentric spatial frames. Need to describe these terms clearly.2. Line 32, it is not clearly defined the meaning of "cognitive map". If I understand correctly, the "cognitive map" here means "allocentric map".3. Line 41-42, I am not sure the description is correct. Please refer to the paper of Hasegawa et al., 2000 Science.4. In the caption of fig 2, the subtitle of panel a and c is not suitable because "item-selective neuron" is not consistent with "co-location" neuron in the text.5. Line 112-113, authors wrote "This neuron showed the strongest response across all the co-location when the orientation of the background-cue was 90, while........". However, in fig 3a it seems that, in II-A/B, the data of the background-cue with 90-degree orientation is not significant higher than background-cue with 0-degree orientation.6. In fig 3, the raster plot of panel a and panel b might be miss-displaced, please have a check.7. In fig 3, 4 and 5, the term "I-cue" and "B-cue" were not defined in both text and figure caption.In the results section of text, there is no description of what data are represented by the three curves (yellow, grey and black) in fig 3a-b, as well as in fig 4a and fig 5a-b.8. Line 162, under this subtitle authors show neurons responded to specific target location in fig 5. Are there neurons response to 4 target locations, respectively?9. Authors need to explicitly discuss the advantage of the present study (task design, etc.) in study of memory flexibility, comparing with previous studies.Reviewer #4:The manuscript by Yang & Naya seeks to examine how the hippocampus integrates retrieved information and current perceived information for subsequent decision making. The main strength of the paper is the multi-step task used, which allows the authors to examine different phases in which memory is first retrieved, a subsequent stimulus is presented, and a response is made. Since the response depends on both retrieval and perception, this requires integration of memory and perception. The authors present different flavors of hippocampal neurons which show varied selectivity profiles, showing both neurons that combine retrieved and perceptual information, along with those that show responses related to the choice. The results expand the view of different types of hippocampal neurons and how single neurons integrate a variety of sources of information during a complex task. The link between neural responses and behavior on error trials also reveals that hippocampal signaling is predictive of future behavior rather than accuracy per se. However, I think that several details need to be clarified, and more broadly the manuscript could be improved by preparing the reader for the types of neurons being examined.- The authors frame the paper around the use of memory in a flexible manner, and how this occurs at a single neuron level. The notion of "convergence" or neurons demonstrating selectivity based on a combination or conjunction of cues is commonly queried in hippocampal neurons, so this property was expected (although, it could be motivated based on prior literature in the Introduction, this literature is considered only in the Discussion). However, I found the notion and use of "transference" to be confusing/unclear and not well-motivated or explained in the Introduction. This made it difficult to interpret this property when presented in the Results. It seems like this response property of neurons emerged as an artifact of how the animals were trained and therefore may not be a general feature of how the hippocampus flexibly integrates information; it likely reflects a form of rotation performed during this task. It would be helpful to explain this up front in the manuscript, given that substantial space is devoted to it.- Related to the above point, I wonder how general the authors think their findings are (the demonstration of 3 main types of neurons showing 'convergence', 'transference', and 'targeting'), or whether the specific types of neurons found here are strongly tied to the specific task and the amount of time needed to train animals to perform this kind of task. For example, is there an over-representation of 'transference' neurons simply due to the considerable training on the task and the manner in which the animals were trained (which is already evident in the 'transference' neurons), or is this a general feature of an episodic memory system? It is clear that the authors favor the later interpretation (third paragraph of the discussion), but a broader discussion of whether all of the response properties seen here are related to episodic memory versus considerable training on a specific behavior task should be addressed.- Several correlation analyses are presented in the manuscript (Figure 2, co-location indices, Fig. 3d, similarity of background tuning, and in Fig. 4). Although I understood the theoretical significance of the 'co-location index' (Fig. 2), I generally found these correlation measures to be confusing, because 1) the description of these analyses, i.e. the data going into each of the correlations was not clearly stated and 2) the theoretical significance of each of the different correlations could be much more clearly spelled out in the Results (at least in my reading for those shown in Figs. 3 and 4). Both should be clarified - otherwise these analyses could be completely lost on the reader (I had to expend considerable effort to try to figure out what was being described/shown). For example, it was not clear to me how the correlation for the co-location index shown in 2b (example neuron) and 2d (population) was computed. In the description of the analysis in the Methods (lines 377-382), the sets of data points that are being correlated with each other are not actually stated. Is this the correlation between the mean firing rate in response to items within a co-location pair, such as I-A and I-B, across individual time bins (the 8 relevant time bins) or across individual trials and time bins? Without knowing what the correlation is computed over (i.e. the correlation in response across trials, or the correlation over time bins of mean response), the reader has no sense of the data going into the correlation, so it makes it difficult to interpret. I inferred that the correlation is being computed across time bins using the mean firing rate over trials, between the two co-location stimuli. This is potentially concerning if there are a low number of data points (are all 8 time bins included, or just those with significant F-stats?). Again, this can be addressed by clearly describing the analysis procedure and indicating what the correlation is being computed between (lines 377-382, 397-398, 409-411). Similarly, the data points going into the correlations shown in Figs. 3b and 3d were not clear to me. Is this the correlation between mean responses across the 3 background orientations for the 'best' and 'second-best' co-location stimuli, such that three data points are going into the correlation? Note that additional schematics may be helpful in illustrating these analyses.Minor comments:- The training performed prior to the recording sessions is described in the SI text. It is easy for the reader to miss this description - please reference this text in the Methods where training is mentioned (lines 318-320).- Please include a few more details regarding the training. It looks like there are several phases of training: training on 4 stimuli with a 0 degree background cue, training on 4 stimuli with varied background cues (from -90 to 90), and then training on the 8 stimuli which were actually used in the recording sessions. Please describe the approximate duration of training for each of these phases prior to the recording task- how long did it take for the animals to be trained in each of these phases? In the final part of training, when the animals were trained on the 8 stimuli used in the paper, were they again trained first with the 0 degree background cue and then introduced to the varying direction of the cue? Were they trained on all 8 stimuli at once? Please include these details.- Three-way ANOVAs were performed across sliding windows to examine the effects of 'co-location', 'background' and 'target'. Yet only 3 of the 5 backgrounds were used in this analysis (-45 and 45 degrees were excluded). What is the rationale for removing 2/5 of the data from the analysis? I found this to be particularly confusing when trying to understand the data shown in Figs. 3-5, since the top panels show firing rates over time for all 5 background directions, yet when the data are broken down by background direction in the panels below, only 3 directions are shown.- On lines 127-129 it is stated that "After background-cue presentation…neurons…exhibited either co-location-selective activities (14%) or background-selective activities (14%)." Yet, these numbers conflict with the data shown in Fig. 3C - with ~7% of neurons showing each of these properties after the background cue is presented.- To show 'convergence' of retrieved information with the background cue ('perceptual') information, the authors analyze orientation tuning for co-location selective neurons (in Fig. 3D, 66 neurons). Were the co-location-selective neurons (those showing a 'retrieval' signal) defined only from the time period after the item cue was presented (before the background cue was shown)? Please clarify - I could not find this in the Methods. This definition would provide the clearest evidence that neurons which initially signal retrieval also show selectivity to the background perceptual information.- Related to the above point, i.e. demonstrating 'convergent' retrieval signals and perceptual ('background') signals within individual neurons, I wonder if similar results are obtained by simply looking at the intersection of neurons which show both co-location selective information and background-selective information (intersection of neurons shown the individual effects presented in Fig. 3D). If the proportion of neurons demonstrating both kinds of selectivity are greater than would be expected by chance, this is another way to demonstrate the presence of convergent information at the single neuron level (one main point of the paper).3 Jul 2020Submitted filename: Response Yang Naya.docxClick here for additional data file.6 Aug 2020Dear Dr Naya,Thank you for submitting your revised Research Article entitled "Hippocampal cells integrate past memory and present perception for the future." for publication in PLOS Biology. I have now obtained advice from two of the original reviewers and have discussed their comments with the Academic Editor.Based on the reviews, we will probably accept this manuscript for publication, assuming that you will modify the manuscript to address the remaining points raised by reviewer #4. Please also make sure to address the Data and other policy-related requests noted at the end of this email.We expect to receive your revised manuscript within two weeks. Your revisions should address the specific points made by each reviewer. In addition to the remaining revisions and before we will be able to formally accept your manuscript and consider it "in press", we also need to ensure that your article conforms to our guidelines. A member of our team will be in touch shortly with a set of requests. As we can't proceed until these requirements are met, your swift response will help prevent delays to publication.*Copyediting*Upon acceptance of your article, your final files will be copyedited and typeset into the final PDF. While you will have an opportunity to review these files as proofs, PLOS will only permit corrections to spelling or significant scientific errors. Therefore, please take this final revision time to assess and make any remaining major changes to your manuscript.NOTE: If Supporting Information files are included with your article, note that these are not copyedited and will be published as they are submitted. Please ensure that these files are legible and of high quality (at least 300 dpi) in an easily accessible file format. For this reason, please be aware that any references listed in an SI file will not be indexed. For more information, see our Supporting Information guidelines:https://journals.plos.org/plosbiology/s/supporting-information*Published Peer Review History*Please note that you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out. Please see here for more details:https://blogs.plos.org/plos/2019/05/plos-journals-now-open-for-published-peer-review/*Early Version*Please note that an uncorrected proof of your manuscript will be published online ahead of the final version, unless you opted out when submitting your manuscript. If, for any reason, you do not want an earlier version of your manuscript published online, uncheck the box. Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us as soon as possible if you or your institution is planning to press release the article.*Protocols deposition*To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosbiology/s/submission-guidelines#loc-materials-and-methods*Submitting Your Revision*To submit your revision, please go to https://www.editorialmanager.com/pbiology/ and log in as an Author. Click the link labelled 'Submissions Needing Revision' to find your submission record. Your revised submission must include a cover letter, a Response to Reviewers file that provides a detailed response to the reviewers' comments (if applicable), and a track-changes file indicating any changes that you have made to the manuscript.Please do not hesitate to contact me should you have any questions.Sincerely,Roli RobertsRoland G Roberts, PhD,Senior Editor,rroberts@plos.org,PLOS Biology------------------------------------------------------------------------ETHICS STATEMENT:-- Please include the full name of the IACUC/ethics committee that reviewed and approved the animal care and use protocol/permit/project license. Please also include an approval number.------------------------------------------------------------------------DATA POLICY:You may be aware of the PLOS Data Policy, which requires that all data be made available without restriction: http://journals.plos.org/plosbiology/s/data-availability. For more information, please also see this editorial: http://dx.doi.org/10.1371/journal.pbio.1001797Many thanks for providing the raw data in your OSF deposition. However, we also ask that all individual quantitative observations that underlie the data summarized in the figures and results of your paper be made available in one of the following forms:1) Supplementary files (e.g., excel). Please ensure that all data files are uploaded as 'Supporting Information' and are invariably referred to (in the manuscript, figure legends, and the Description field when uploading your files) using the following format verbatim: S1 Data, S2 Data, etc. Multiple panels of a single or even several figures can be included as multiple sheets in one excel file that is saved using exactly the following convention: S1_Data.xlsx (using an underscore).2) Deposition in a publicly available repository. Please also provide the accession code or a reviewer link so that we may view your data before publication.Regardless of the method selected, please ensure that you provide the individual numerical values that underlie the summary data displayed in the following figure panels as they are essential for readers to assess your analysis and to reproduce it: Figs 2BDG, 3BD, 4AB, 5AC, 6AB, 7AB, S1ABC, S3, S5A, S6ABC. NOTE: the numerical data provided should include all replicates AND the way in which the plotted mean and errors were derived (it should not present only the mean/average values).Please also ensure that figure legends in your manuscript include information on where the underlying data can be found, and ensure your supplemental data file/s has a legend.Please ensure that your Data Statement in the submission system accurately describes where your data can be found.------------------------------------------------------------------------REVIEWERS' COMMENTS:Reviewer #3:[identified himself as Mingsha Zhang]Authors properly answered my questions and made modification in the manuscript accordingly. I don't have further questions and comments.Reviewer #4:[identifies herself as Arielle Tambini]The manuscript has substantially improved with the revision, the authors have thoroughly addressed the points raised. The inclusion of schematics depicting the analysis approaches is much appreciated and I think will benefit the manuscript. I just have a few remaining comments which are minor suggestions.Line 143-4: "One critical concern here might be whether the association effect reflected the locations retrieved from the item-cues." The implication or tone of this sentence (that it is problematic or concerning that the co-location effect may reflect retrieval processes) does not match the rest of the paragraph - which does show that the co-location signal is most consistent with reflecting retrieval of a common location. It may be helpful to revise this sentence.The schematic in the new Fig S7 is a nice addition that helps to clarify the examined memory signals and processes in this task. I'd suggest moving it to the main text/combining it with a figure in the main text so that more readers may see it.16 Sep 2020Submitted filename: Response Yang Naya.docxClick here for additional data file.22 Sep 2020*Dear Dr Naya,On behalf of my colleagues and the Academic Editor, Jozsef Csicsvari, I am pleased to inform you that we will be delighted to publish your Research Article in PLOS Biology.The files will now enter our production system. You will receive a copyedited version of the manuscript, along with your figures for a final review. You will be given two business days to review and approve the copyedit. Then, within a week, you will receive a PDF proof of your typeset article. You will have two days to review the PDF and make any final corrections. If there is a chance that you'll be unavailable during the copy editing/proof review period, please provide us with contact details of one of the other authors whom you nominate to handle these stages on your behalf. This will ensure that any requested corrections reach the production department in time for publication.Early VersionThe version of your manuscript submitted at the copyedit stage will be posted online ahead of the final proof version, unless you have already opted out of the process. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.PRESSWe frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with biologypress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.Thank you again for submitting your manuscript to PLOS Biology and for your support of Open Access publishing. Please do not hesitate to contact me if I can provide any assistance during the production process.Kind regards,Alice MussonPublishing Editor,PLOS Biologyon behalf ofRoland Roberts,Senior EditorPLOS Biology
Authors: Bruce L McNaughton; Francesco P Battaglia; Ole Jensen; Edvard I Moser; May-Britt Moser Journal: Nat Rev Neurosci Date: 2006-08 Impact factor: 34.870
Authors: Karen L Campbell; Kevin P Madore; Roland G Benoit; Preston P Thakral; Daniel L Schacter Journal: Hippocampus Date: 2017-11-17 Impact factor: 3.899
Authors: Allison N Pullin; Victoria S Farrar; Jason W Loxterkamp; Claire T Jones; Rebecca M Calisi; Kristina Horback; Pamela J Lein; Maja M Makagon Journal: Poult Sci Date: 2022-08-30 Impact factor: 4.014
Fig 1
Fig 2
Fig 3
Fig 4
Fig 5
Fig 6
Fig 7
Fig 8