Hippocampus contributes to the maintenance but not the quality of visual information over time.

The hippocampus has recently been implicated in the brief representation of visual information, but its specific role is not well understood. We investigated this role using a paradigm that distinguishes quantity and quality of visual memory as described in a previous study. We found that amnesic patients with bilateral hippocampal damage (N = 5) were less likely to remember test stimuli than comparison participants despite a brief maintenance interval (900 msec). However, estimates of memory quality were similar for all groups. Our findings suggest that the hippocampus contributes to brief maintenance of visual information but does not contribute to the quality of that information.

The ability to retain and integrate visual information across short intervals is critical to everyday life, demonstrated whenever one examines a painting, reads a manuscript, or remembers that a stoplight is still red. The psychological processes and neural mechanisms of these briefly held visual representations have been investigated (Jonides et al. 2008; Chun et al. 2011), but the specific contributions of many brain regions to the maintenance of visual information are unclear. For example, while the medial temporal lobe (MTL) and hippocampus are often associated with the formation of lasting declarative memories (Scoville and Milner 1957; Cohen and Squire 1980), MTL and hippocampus have also been implicated in the representation of visual information over very short intervals in animal models (Eacott et al. 1994; Murray and Bussey 1999; Bussey et al. 2002; Cowell et al. 2006) and more recently in humans (Olson et al. 2006; Barense et al. 2007; Lee et al. 2012; Warren et al. 2012). However, it is not clear whether the hippocampus and MTL contribute to the maintenance of visual representations, the quality of visual representations, or both.

Neuropsychological studies have shown that damage to MTL structures including the hippocampus causes impairment in performance on many tasks at short delays (Barense et al. 2005, 2007; Lee et al. 2005a,b; Hannula et al. 2006; Lee and Rudebeck 2010; Warren et al. 2010, 2011, 2012; Kurczek et al. 2013; Watson et al. 2013). Typically, these investigations have used binary yes/no or forced choice recognition tasks that cannot address how hippocampal damage might change the quality of mental representations. Tasks that permit graded responses have shown that MTL and hippocampal damage increase the variance of responses over time relative to neurologically normal participants (Sidman et al. 1968; Warren et al. 2010), but have not explained any underlying representational changes.

Zhang and Luck (2008) developed a theory and method sufficient to inform this issue by beginning with the premise that mental representations are inherently noisy. From this perspective, representational changes in short-delay tasks (whether in healthy participants or those with hippocampal damage) could be due to: added noise in mental representations that reduce their quality, leading to test–time mismatch with the original stimulus; increased probability that a stimulus is completely forgotten; or a combination of these phenomena (Fig. 1A). Studies of visual working memory in healthy participants suggest that visual representations follow the second course, disappearing from memory rather than decreasing in quality over time (Zhang and Luck 2009, but see Bays et al. 2009). Critically, the method of Zhang and Luck (2008, 2009, 2011) supports independent estimation of the probability of a tested item being represented in memory and the quality of memory representations, providing significant advantages over binary response tasks.

Figure 1.

Model assumptions and trial sequence. (A) Based on the model of Zhang and Luck (2008), we hypothesized that briefly maintained visual representations could change in two dissociable ways. Representations could be remembered or forgotten (left and right columns), and the quality of representations could remain intact or be reduced (top and bottom rows). Each panel diagrams a combination of forgetting and degradation of the visual representation (top), and the associated distribution of responses (bottom) around the target value (0, with maximum response error of π). Expected response distributions (purple) could change in two ways depending on changes to underlying representations: reduced quality would yield a broader distribution of responses; while forgetting some representations entirely would yield a hybrid of a uniform distribution reflecting guesses (blue) and a target-centered distribution of memory-guided responses (red). We predicted that hippocampal damage would reduce probability of memory for studied items but not degrade representations (upper right). (B) In each trial, participants saw 1, 3, or 6 color squares for 100 msec (white text was not presented). After a 900-msec blank interval, the target location was indicated with a thick open square. Participants selected the color that was seen in that location from the color wheel.

Here, we evaluated the necessity of hippocampus for the brief maintenance of simple visual information using a neuropsychological approach. We tested neurological patients with amnesia (N = 5, 1F/4M); patients with brain damage excluding MTL and hippocampus (“BDC”; N = 14, 6F/8M, 1 M later excluded for color blindness); and healthy comparison participants (“NC”; N = 19, 9F/10M) of similar age and educational attainment (Table 1; P > 0.05) using a task that provides insight into the quantity and quality of visual memory representations at short delays (Fig. 1B; Zhang and Luck 2008). Participants saw 1, 3, or 6 color stimuli presented briefly (100 msec) and 900 msec later responded to a memory probe in a particular location by selecting the color previously occupying that location from a color ring. Based on prior research (Sidman et al. 1968; Downes et al. 1998; Warren et al. 2010), we hypothesized that amnesic patients would show a broader distribution of responses than comparisons, reflecting reduced retention of visual information versus comparisons. Importantly, the sensitive nature of our task and analysis also allowed us to form novel hypotheses regarding the specific nature of the impairment. Following the well-established role of the MTL and hippocampus in memory, we predicted that the visual representations of amnesic patients would be more susceptible to forgetting, but that representations which were available at test would be distributed similarly to those of comparisons.

Table 1.

Demographic and neuropsychological data characterizing participants

Task materials were based on those described by Zhang and Luck (2008). Displays consisted of closed squares of specific colors that subtended 2° of visual angle (horizontally and vertically), open black squares subtending 2.04° and 2.2° of visual angle, and a colored ring with an inner radius of 7.1° of visual angle and an outer radius of 9.3° of visual angle (Fig. 1B). For the colored ring, 180 equal-luminance colors were selected from CIE L*a*b* color space by sampling the space around L = 70, a = 0, b = 13 in 180 even steps around the circumference of a circle in the a*b* plane with radius 45. All values were converted to RGB and checked for compatibility with that color space. Colors for the closed square stimuli were drawn from this 180-color spectrum. Visual stimuli were presented at a distance of 50 cm on a 21-in LCD monitor with a vertical refresh rate of 60 Hz (Multi-Sync 2190UXi, NEC Corporation of America). Responses were made with a computer mouse.

Our procedure adapted the Zhang and Luck paradigm (2008). Participants were seated in front of a computer display. At the beginning of each block, written instructions were presented on the screen: “Colored squares will briefly appear near the center of the screen. Remember all of the colors. When one position is cued, indicate the color from that position.” Complementary verbal instructions by the experimenter emphasized key task components, and participant comprehension was evaluated. A practice block (15 three-item trials) preceded the main test phase. The main test phase consisted of three blocks containing 150 trials including 50 trials each for 1, 3, and 6 items in a unique random order.

Participants initiated each trial with a mouse click; a central fixation cross changed color, and ∼1 sec later the trial began (the stimulus onset asynchrony jittered by ±125 msec). The trial sequence (Fig. 1B) was: a study display containing one or more color squares (100 msec); a blank display (900 msec); and the test display (presented until response) which included a mouse cursor, open squares surrounding each position that previously held a colored square, a cue in the form of open square with a thicker outline surrounding the test location, and the color ring. Participants indicated which color had been presented in the cued location by clicking that color on the color ring, and guessed if unsure.

Based on the distribution of test–time responses, three parameters were estimated for each participant: p, the probability that the probe item was in memory at test (i.e., the “quantity” of information in memory); k, the concentration of the response distribution around µ (i.e., the “quality” of information in memory); and µ, the mean of the response distribution. Parameter estimation was based on previously reported methods (Zhang and Luck 2008), and is described in the Supplemental Material. All trials with response times ≤15 sec were used in the parameter estimation procedure. Group differences were evaluated for the three parameters (p, κ, µ). No significant differences were found for µ (see Supplemental Material). In order to address the possibility that nontarget items significantly influenced response distributions, we also estimated parameters for an alternative model (Bays et al. 2009). Results were generally consistent with the main findings; the alternative approach and results are described in the Supplemental Material. Between-group differences for all dependent variables were tested using repeated-measures ANOVA implemented as a hierarchical linear model with participants as a random effect, group as a between-participants fixed effect, and number of items as a within-participants fixed effect. Planned between-group and between-condition comparisons were conducted using nonpaired, equal-variance t-tests. Permutation tests of the planned comparisons are reported as pperm and were calculated as follows: bootstrapped distributions were created by assigning group membership to the data in 105 randomly selected permutations, recording the statistic value for each permutation, and determining the percentile rank of the observed statistic value in the bootstrapped distribution. Effect size was measured with an unbiased variant of Cohen's d that accounts for small sample sizes (dunb) (Grissom and Kim 2012, p. 70). Response time was not a dependent variable of interest, but a similar, exploratory analysis is presented in the Supplemental Material and Table S2.

The probability of an item being present in memory at test (p) differed between groups and was significantly reduced in amnesic patients (Fig. 2A; Supplemental Table S1). Between-group differences in p were significant [F(2,34) = 4.597, P < 0.001], and planned comparisons between groups for each number of items showed that amnesic patients were significantly impaired relative to the NC and BDC groups in the three-item condition [NC, T(22) = 2.879, P = 0.009, pperm = 0.006; BDC, T(16) = 3.020, P = 0.008, pperm = 0.005], at least marginally impaired in the six-item condition [NC, T(22) = 2.070, P = 0.050, pperm = 0.015; BDC, T(16) = 2.152, P = 0.047, pperm = 0.029], but no more than marginally impaired in the one-item condition [NC, T(18) = 1.878, P = 0.077, pperm = 0.073; BDC, T(16) = 1.611, P = 0.127, pperm = 0.133]. The lack of a significant difference for one item may have reflected a ceiling effect as all groups had values of p near 1 (see Supplemental Table S1 and Discussion). The NC and BDC groups did not differ in any condition [one item, T(26) = 0.445, P = 0.660, pperm = 0.328; three items, T(30) = 1.047, P = 0.304, pperm = 0.150; six items, T(30) = 1.089, P = 0.285, pperm = 0.143].

Figure 2.

Damage to the hippocampus and MTL reduced the probability that an item would be remembered without altering the quality of memory representations. (A) Group means for the probability of remembering the tested item. Amnesic patients (Amn) were significantly (*) less likely to remember items at test overall and specifically for the three- and six-item conditions versus both comparison groups. Error bars show SEM, and the performance of individual amnesic patients is indicated by points. In the three- and six-item conditions patient 1846 performed better than the other amnesic patients and near the comparison means; more information and detailed parameter fits are provided in Supplemental Table S3. (B) Group means for the quality of remembered representations (i.e., discounting forgetting) were similar for all item conditions, but quality was significantly (*) greater in the one-item condition than the three- and six-item conditions. Error bars and points as in panel A.

In addition to between-group effects, p was affected by the number of items presented [F(2,64) = 363.874, P < 0.001], but the interaction of the group and number-of-items factors was not significant [F(4,64) = 1.428, P = 0.235]. For all groups the pattern was similar (Fig. 2A): p was greatest for one item; relatively less for three items; and least for six items. Planned within-group comparisons between p for presentations of one versus three items and three versus six items showed that this pattern was significant for all groups [one item versus three items, each T > 4.5, each P < 0.0025, each pperm < 0.005, each dunb > 1.9; three items versus six items, each T > 4.0, each P < 0.005, each pperm < 0.001, each dunb > 2.4]. Thus, when more stimuli were presented, the probability of any given stimulus being present in memory at test was reduced.

The quality of memory representations (κ) was influenced by the number of items in a display, but did not differ between groups (Fig. 2B; Supplemental Table S1). The NC, BDC, and amnesic groups all had similar κ [F(2,34) = 0.229, P = 0.796], and planned comparisons between groups for each number of items found no significant differences [each T < 1.4, each P > 0.19, each pperm > 0.09]. Meanwhile, the number of items presented had a significant effect on κ [F(2,64) = 17.478, P < 0.001]. Planned comparisons revealed the same pattern for each group: displays of one item produced the greatest value of κ [one versus three items, each T > 3.2, each P < 0.025, each pperm < 0.005]; while presentations of three and six items produced values of κ that were lower and not statistically different from each other [three versus six items, each T < 1.7, each P > 0.10, each pperm > 0.501]. There was no significant interaction of the group by condition [F(4,64) = 0.427, P > 0.789]. This pattern of higher quality representations for one item than for either of the larger sets of items may reflect previously reported characteristics of visual working memory (Zhang and Luck 2008).

The observed deficit in brief maintenance of visual information by amnesic patients with hippocampal damage could be attributed to reduced memory capacity, reduced ability to maintain information over time, a combination of these factors, or still further causes. Our findings are compatible with a previously hypothesized role for the hippocampus in the on-line processing of visual information (Gallegos et al. 2006; Barense et al. 2007; Warren et al. 2011, 2012), and congruent with suggestions that short-term and long-term memory systems may not be neurally dissociable (Ranganath and Blumenfeld 2005). Moreover, we suggest that the role of the hippocampus in visual representation is inherently mnemonic and relational. For example, relational memory theory (Cohen and Eichenbaum 1993; Eichenbaum and Cohen 2001, Moses and Ryan 2006, Ranganath 2010; Eichenbaum and Cohen 2014) predicts that the hippocampus is necessary for the binding of arbitrarily related information (e.g., color and spatial location) irrespective of timescale. What our current methodology identifies as outright loss of information may include some responses in the three- and six-item conditions that involve color–location association errors on the part of participants, and relational memory theory predicts that hippocampal damage would increase these errors (Watson et al. 2013). In this context, the relatively preserved performance of amnesic patients in the one-item condition could be due to a lack of relational demands. However, a supplemental analysis using the model of Bays et al. (2009), which attempts to account for responses driven by nontarget items, did not provide strong evidence for increased color–location association errors by amnesic patients (see Supplemental Results).

We suggest that the observed impairment in the brief maintenance of visual information by amnesic patients is due to impairments in on-line processing of relations due to hippocampal damage, but other data and interpretations are relevant. Zhang and Yonelinas (2012) tested a mixed group of unilateral temporal lobectomy and anoxic amnesic patients using similar methodology, and reported a change in the quality of memory representations (i.e., decreased κ) rather than the probability of memory (decreased p). Both studies clearly show that hippocampal damage can impair performance on the Zhang and Luck (2008) task; different patterns of results could be attributable to patient anatomy or details of task implementations. Meanwhile, Jeneson et al. (2010, 2012) and Jeneson and Squire (2011) have suggested that deficits at similar timescales are attributable to the inability of amnesic patients to remember information exceeding the capacity of short-term memory because they lack normal declarative memory systems. We note that our task used a brief maintenance interval (900 msec) and that the amnesic group showed an impairment that was significant for dislays containing as few as three items. Jeneson et al. (2012) have previously described displays with these characteristics as within the capacity of short-term memory, and we concur with that description. We attribute our finding of impairment in this context to the sensitive nature of our experimental methodology.

Despite our robust findings, the study had some limitations. As in most neuropsychological investigations studying severely amnesic patients, our sample size was relatively small. However, the study had enough power to uncover significant differences, and our main findings had substantial effect sizes. Interestingly, while we observed impairment for the amnesic group that was greatest for presentations of three and six items, presentations of one item did not reliably produce impairment. Single items may have been maintained normally by the amnesic group, but it is possible that differences in the maintenance of a single item were obscured by a ceiling effect (especially among comparison participants). Further exploration of the parameter space in this task could attempt to address ceiling effects by including a two-item condition or a reducing item exposure time.

In summary, we found that hippocampal damage was related to reduced probability of remembering briefly maintained mental representations of visual information, indicating that the hippocampus normally makes important contributions to remembering visual information over very short intervals. Our results suggest that future investigations of visual representations in hippocampal amnesic patients could benefit from using graded rather than binary response designs in order to collect rich response distributions. We predict that populations with damage or dysfunction of hippocampus will show deficits at short delays when tested with stimuli of visual or other modalities, further demonstrating the contributions of hippocampus to brief representation and on-line processing.