Medial prefrontal cortex and hippocampal activity differentially contribute to ordinal and temporal context retrieval during sequence memory.

Remembering sequences of events defines episodic memory, but retrieval can be driven by both ordinality and temporal contexts. Whether these modes of retrieval operate at the same time or not remains unclear. Theoretically, medial prefrontal cortex (mPFC) confers ordinality, while the hippocampus (HC) associates events in gradually changing temporal contexts. Here, we looked for evidence of each with BOLD fMRI in a sequence task that taxes both retrieval modes. To test ordinal modes, items were transferred between sequences but retained their position (e.g., AB3). Ordinal modes activated mPFC, but not HC. To test temporal contexts, we examined items that skipped ahead across lag distances (e.g., ABD). HC, but not mPFC, tracked temporal contexts. There was a mPFC and HC by retrieval mode interaction. These current results suggest that the mPFC and HC are concurrently engaged in different retrieval modes in support of remembering when an event occurred.

Memory for sequences of events is a fundamental component of episodic memory (Tulving 1984, 2002; Allen and Fortin 2013; Howard and Eichenbaum 2013; Eichenbaum 2017). While different experiences share overlapping elements, the sequence of events is unique. Remembering the order of events allows us to disambiguate episodes with similar content and make detailed predictions supporting decision-making.

At least two complementary memory processes contribute to the retrieval of events in the correct sequence: ordinal (Orlov et al. 2002) and temporal context (Howard and Kahana 2002) retrieval modes. Whether these disparate retrieval modes operate coincidently or not remains an open question with consequences for understanding basic mechanisms of how we remember the events that unfold throughout our day. According to an ordinal retrieval mode, items are remembered by their position within an event sequence (DuBrow and Davachi 2013; Allen et al. 2014; Long and Kahana 2019), providing sequential memory through well-established semantic or abstracted relationships (first, second, third, etc.). While for a temporal context retrieval mode, events are remembered through a gradually changing temporal context within which specific items have been associated. According to temporal contexts, when an element of a sequence is presented or retrieved (e.g., “C” in ABCDEF), items that are more proximal in the sequence (e.g., the “D” in the sequence) have a higher retrieval rate compared with items that are further away (e.g., the “F” in the sequence). These temporal contexts result from item associations that are dependent on time varying neural activity (e.g., Eichenbaum 2014), and contribute to sequence memory through the reactivation of neighboring items during retrieval (DuBrow and Davachi 2013; Long and Kahana 2019).

The medial prefrontal cortex (mPFC) and hippocampus (HC) are thought to contribute to sequence memory through ordinal representations and temporal contexts, respectively (Agster et al. 2002; Fortin et al. 2002; Kesner et al. 2002; DeVito and Eichenbaum 2011; Allen et al. 2016; Jenkins and Ranganath 2016). In rodents, mPFC disruptions impair sequence memory (DeVito and Eichenbaum 2011; Jayachandran et al. 2019), mPFC “time cells” are evident (Tiganj et al. 2017), and positions within a sequence can be the main determinant of differential activity in mPFC neurons during spatial sequences (Euston and McNaughton 2006). In humans, mPFC activation is sensitive to temporal order memory (Preston and Eichenbaum 2013), and codes for information about temporal positions within image sequences regardless of the image itself (Hsieh and Ranganath 2015). HC activations are also generally associated with temporal order memory (Kumaran and Maguire 2006; Ekstrom and Bookheimer 2007; Lehn et al. 2009; Ross et al. 2009; Jenkins and Ranganath 2010; Tubridy and Davachi 2011; Kalm et al. 2013; Hsieh et al. 2014; Goyal et al. 2018). Prior evidence further shows that the medial temporal lobe, specifically the HC formation, plays a critical role in the use of a TCM retrieval mode in the brain (Manns et al. 2007; Hsieh et al. 2014; Bladon et al. 2019). The HC binds events within temporal contexts (Eichenbaum et al. 2007; DuBrow and Davachi 2013; Bladon et al. 2019) through a gradually changing neural context (Manns et al. 2007; Mankin et al. 2012). Similarly, medial temporal lobe neuronal and BOLD activations in humans have demonstrated evidence for gradually evolving temporal contexts (Howard et al. 2012; Kalm et al. 2013; Kragel et al. 2015).

Here we tested the contributions of the mPFC and HC during a visual sequence memory task that provides behavioral evidence of both ordinal and temporal context retrieval modes (see Fig. 1A; task modified from Allen et al. 2014). Briefly, participants first memorized six visual sequences (six images each) in a single passive viewing phase, and then were instructed to make judgments as to whether individual items were subsequently presented in sequence (InSeq) or out of sequence (OutSeq) over 240 self-paced presentations of each of the six items from each sequence. In the task, the two retrieval modes are parsed using probe trials that place conflicting demands on ordinal (Orlov et al. 2000; Allen et al. 2014, 2015) and temporal context modes (Jayachandran et al. 2019). We first evaluated ordinal retrieval modes using items that were transferred from one sequence to another while retaining their ordinal position (Ordinal Transfers) (Fig. 1B). Evidence for an ordinal-based retrieval mode occurs when these probes are identified as in sequence, because they occur in the same ordinal position as their original sequence. mPFC activations (but not HC) was strongest for these ordinal retrievals. Second, we evaluated a temporal context retrieval mode using items that skipped ahead (Skips) (Fig. 1B) with shorter lag distances (ABCFEF) compared with larger lag distances (AFCDEF). Skips should be most difficult to detect on the shortest lag distances because proximal items in a sequence are more likely to be retrieved (Howard and Kahana 2002; Kragel et al. 2015) and thus judged as InSeq. HC activations (but not mPFC) tracked with lag distance, providing evidence the HC is more reflective of a temporal context-based retrieval mode. Importantly, a significant interaction was observed such that mPFC and HC differentially activated for ordinal and temporal context retrievals. Altogether, our data show that sequence memory involves both retrieval modes. In line with these results, we suggest that understanding episodic memory requires more insight into the neurobiology of ordinal processing, in addition to the more often studied temporal contexts, in the mPFC and HC system.

Figure 1.

Sequence memory task and overall performance levels. Participants were tested on a sequence memory task that differentially burdens different retrieval modes using different out of sequence probe trial types. (A) An example sequence set that included six sequences. Two sequences were low memory demand sequences and four were high memory demand sequences. (B) There were three out of sequence probe trial types: items that were repeated in the sequence (Repeats), items that were presented too early in the sequence (Skips), and items that transferred from one sequence to another, while remaining in their ordinal position (Ordinal Transfers). Repeats and Skips occurred throughout the whole task, whereas Ordinal Transfers occurred during the second half only. (C) Accuracy throughout the task (error bars = ±1SD). Participants performed best on Repeats, then Skips, and poorest on Ordinal Transfers. (D,E) Distributions of response times for all InSeq trials (D, gray bars) and for all OutSeq trials (E, gray bars) for all participants with a fitted two-term Gaussian curve (black line). (F) A bimodal Gaussian curve fit better than a unimodal curve for InSeq and OutSeq trials. A trimodal curve did not improve the fit and increased the root mean squared error (not shown), suggesting distinct decisions decision-making between two decisions. It was rare to observe responses outside of the two distributions.

Results

Sequence memory

First, we investigated behavioral sequence performance in two ways: (1) using a sequence memory index (SMI), and (2) with accuracy measured as the percent correct. SMI is a summary statistic measuring the overall sequence memory performance and normalizes the proportion of in sequence (InSeq) items and out of sequence (OutSeq) items across different conditions. SMI represents sequence memory performance as a single value ranging from −1 to 1 (Eq. 1). An SMI of “1” represents perfect sequence performance (response times >1 sec for all InSeq items and <1 sec for all OutSeq items), and 0 represents chance performance (see “Sequence Memory Analysis” in the Materials and Methods). For accuracy, we evaluated the percentage of OutSeq items (Repeats, Skips, and Ordinal Transfers) that were identified as out of sequence, and InSeq items that were identified as in sequence. We analyzed the low and high memory demand sequences separately (Fig. 1A). As expected, participants performed significantly better than chance (SMI = 0) on both low memory sequences (SMIlow: 0.751 ± 0.173; SMIlow vs. chance: t(33) = 37.065, P = 3.487 × 10−23) and high memory sequences (SMIhigh: 0.582 ± 0.170; SMIlow vs. chance: t(33) = 19.690, P = 5.391 × 10−20), but performance was significantly better with low memory sequences (SMIlow vs. high: t(33) = −4.839, P = 2.958 × 10−5). High memory sequences were used for all subsequent analyses because they optimally taxed the different sequence retrieval modes. Importantly, there was no testing order effect (first or second experimental block) on sequence memory (SMI1st block = 0.571 ± 0.190, SMI2nd block = 0.594 ± 0.152; SMI1st vs. 2nd: t(32) = −0.386, P = 0.702), nor did we observe any sex differences (SMIfemale = 0.566 ± 0.197, SMImale = 0.602 ± 0.133; SMIfemale vs. male = t(32) = −0.612, P = 0.545). Thus, we pooled these groups.

Next, we evaluated overall performance across each of the three memory probes individually using percent correct for ease of interpretation. Overall, accuracy on InSeq items was 95.827% ± 4.396% (95.477% ± 4.393% when excluding the first item of the sequence), and accuracy on OutSeq items was 52.726% ± 15.848%. As in previously published studies (Allen et al. 2014, 2015), we found that Ordinal Transfers (Fig. 1B,C, accuracy = 34.927% ± 4.120%) were the most difficult, followed by Skips (Fig. 1B,C, Skips, accuracy = 49.205% ± 2.686%), with Repeats being the easiest (Fig. 1B,C, Repeats, accuracy = 78.584% ± 2.853%). There was a significant difference in performance across probe types (F(2,66) = 85.714, P = 3.433 × 10−14) suggesting differences in the available retrieval modes driven by the different conditions. Post hoc pairwise comparisons showed that the performance on Repeats was significantly higher compared with Skips (mean difference = 0.294 ± 0.020, P = 8.137 × 10−16) and Ordinal Transfers (mean difference = 0.437 ± 0.039, P = 1.078 × 10−12) and performance on Skips was significantly higher compared with Ordinal Transfers (mean difference = 0.143 ± 0.039, P = 0.001). The same pattern of results was evident when using SMI that controls for idiosyncratic response patterns. Importantly, participants performed each of the three probe types significantly better than chance (Ordinal Transfers: t(33) = 6.164, P = 5.951 × 10−7; Repeats: t(33) = 24.201, P = 1.403 × 10−22; Skips: t(33) = 14.771, P = 4.203 × 10−16). For all probe types, learning was rapid (asymptotic within a few trials), and performance was steady throughout the duration of the experiment (Fig. 1C). Thus, performance on the task, and the accompanying differences in the brain imaging data, primarily reflect the distinct memory retrieval modes driven by the specific probe conditions.

Next, we looked whether participants made distinct responses on trials that were InSeq or OutSeq by examining the distribution of response times in detail (Maloney and Zhang 2010; Wolpert and Landy 2012). Due to the nature of our task, we expected to observe a bimodal distribution of response times (by design in Allen et al. 2014). For all trial types we observed bimodal distributions suggesting two distinct decisions (InSeq vs. OutSeq) were being made under all conditions (Fig. 1D,E, gray bars). To explore the nature of these distributions statistically, we evaluated the InSeq and OutSeq response distributions by fitting Gaussian curves and comparing fits with unimodal (one-term Gaussian curve), bimodal (two-term Gaussian curve), and trimodal (three-term Gaussian curve) models (Fig. 1F). The outcomes indicated that two-term Gaussian curves were a better fit for InSeq trials (R2 = 0.969, root mean square error [RMSE] = 151.150, xpeak 1 = 0.688, xpeak 2 = 1.313) (Fig. 1D, black line) compared with a one-term Gaussian curve (R2 = 0.9568, RMSE = 176.950, xpeak 1 = 1.313) (Fig. 1F) or three-term Gaussian curve (R2 = 0.969, RMSE = 152.918, xpeak 1 = 0.688, xpeak 2 = 1.313, xpeak 3 = 1.313) (Fig. 1F). OutSeq trials were also better fit by two-term Gaussian curve (R2 = 0.946, RMSE = 7.4596, xpeak 1 = 0.688, xpeak 2 = 1.3125) (Fig. 1E, black line) compared with a one-term Gaussian curve (R-quared = 0.6465, RMSE = 18.846, xpeak = 0.713) (Fig. 1F) or three-term Gaussian curve (R2 = 0.946, RMSE = 7.546, xpeak 1 = 0.688, xpeak 2 = 1.313, xpeak 3 = 2.338) (Fig. 1F). The mean response time for detected InSeq items was 1.369 ± 0.089 sec, and for undetected InSeq items 0.708 ± 0.111 sec. For detected OutSeq items the mean response time was 0.717 ± 0.0539 sec, and 1.353 ± 0.110 sec for undetected OutSeq items. These results provide further evidence that two sequence decisions are being made, and that undetected or incorrect responses reflected a sequence memory error. Importantly, it was rare to observe early or late responses outside of these bimodal response distributions. Such responses would be observed with accidental releases or inattentional effects that might confound subsequent behavioral and BOLD fMRI analyses.

Ordinal memory retrievals

An ordinal retrieval mode is known to contribute to memory for sequences of events that we tested for here using Ordinal Transfer probe trials (e.g., ACDEF) (Fig. 2A). If participants exclusively used an ordinal retrieval mode (e.g., A goes in the first position, B goes in the second position, etc.), then these probes would always be remembered as being InSeq (since “2” is in the same position as B). An ordinal retrieval mode would drive performance on these trials to very low detection levels, possibly to chance, if no other retrieval mode was engaged because of this ordinal interference. Moreover, if an ordinal retrieval mode was used, we would not expect large differences in accuracy as a function of transfer position. Conversely, if participants relied exclusively on another process, such as a temporal context mode, then Ordinal Transfers would be easily identified as OutSeq (since “2” does not follow A, and in fact it does not go with any of the items in that sequence). Nonordinal retrieval modes would drive performance to very high levels on Ordinal Transfer probe trials, probably to the same levels as at Repeats (as a good empirical benchmark for asymptotic performance) or higher.

Figure 2.

mPFC activations and ordinal retrievals. (A) Ordinal retrieval modes were probed using Ordinal Transfer trials. (B) Ordinal Transfers occurred in every position in each sequence except the first. For each position, mean performance was significantly higher than the calculated response bias (red dashed line), and significantly lower than the mean performance on Repeats (purple dashed line) and Skips (blue dashed line). Performance did not significantly differ among positions. (C) Response times of all participants were bimodal (green bars) and best fit using a bimodal Gaussian curve (black line) indicating the two distinct decisions (InSeq vs. OutSeq). (D) The normalized performance slope did not significantly differ across positions, and variability, measured by the averaged squared residuals per participant of the linear regression, was high overall. The high variability suggests conflicting cognitive strategies. (E) The variability on Ordinal Transfers is higher for than for Repeats and Skips. For BOLD analysis, we focused on contrasts for the ordinal retrieval mode. When an ordinal retrieval mode is engaged, participants would identify Ordinal Transfers as InSeq, but would identify Repeats and Skips as OutSeq. (F) A BOLD fMRI mPFC ROI analysis showed that on Ordinal Transfers the ordinal contrast (undetected—detected) activation was significantly higher compared with Repeats and Skips (detected—undetected contrasts). (G) A voxel-wise BOLD fMRI analysis using mPFC as a mask (outlined in green) revealed significant activity in a large area of the mPFC on Ordinal Transfers. (H) In whole brain analysis mPFC activation was also evident for Ordinal Transfers. (I) We found activation on Skips was centered on the rostrolateral prefrontal cortex. (J) No significant activation was found on Repeats. (*) P < 0.05, (**) P < 0.01, (***) P < 0.001.

We found Ordinal Transfers were performed significantly better than chance across all positions (Pos2: 0.314 ± 0.357; Pos2vs. Response bias: t(33) = 3.566, P = 0.001; Pos3: 0.338 ± 0.313; Pos3vs. Response bias: t(33) = 4.512, P = 7.712 × 10−5; Pos4: 0.347 ± 0.258; Pos4vs. Response Bias: t(33) = 5.670, P = 2.549 × 10−6; Pos5: 0.354 ± 0.283; Pos5vs. Response bias: t(33) = 5.334, P = 6.892 × 10−6; Pos6: 0.373 ± 0.336; Pos6vs. Response bias: t(33) = 4.811, P = 3.212 × 10−5) (Fig. 2B, red stars; Table 1A), suggesting participants used nonordinal retrieval modes for these trials. However, performance was much lower than Repeats (Pos2vs. Repeats: t(33) = −7.711, P = 7.019 × 10−9; Pos3vs. Repeats: t(33) = −8.338, P = 1.224 × 10−9; Pos4vs. Repeats: t(33) = −9.917, P = 1.996 × 10−11; Pos5vs. Repeats: t(33) = −8.919, P = 2.621 × 10−10; Pos6vs. Repeats: t(33) = −7.175, P = 3.185 × 10−8) (Fig. 2B, purple stars; Table 1C) and Skips (Pos2vs. Skips: t(33) = −2.912, P = 6.390 × 10−3; Pos3vs. Skips: t(33) = −2.870, P = 7.104 × 10−3; Pos4vs. Skips: t(33) = −3.284, P = 2.426 × 10−3; Pos5vs. Skips: t(33) = −2.854, P = 7.400 × 10−3; Pos6vs. Skips: t(33) = −2.075, P = 4.589 × 10−2) (Fig. 2B, blue stars; Table 1B), suggesting a heavy reliance on an ordinal retrieval mode.

Table 1.

Ordinal Transfer performance compared with the calculated response bias, and performances on Skips and Repeats, sorted by position in the sequence

We found no significant difference in performance across positions (F(4,132) = 0.331, P = 0.786) (Fig. 2B). The observed behavioral performance suggests that participants were disproportionally using an ordinal retrieval mode (e.g., lower performance when compared with Repeats and Skips) in combination with other nonordinal retrieval modes to a lesser extent (i.e., performance was still better than the response bias chance levels). A two-term Gaussian curve (Fig. 2C, black line) best fit the Ordinal Transfer response time distribution (R2 = 0.938, RMSE = 4.956, xpeak 1 = 0.688, xpeak 2 = 1.313) (Fig. 2C, green bars) compared with a one-term Gaussian curve (R2 = 0.679, RMSE = 11.182, xpeak 1 = 1.313) (Fig. 1F) or three-term Gaussian curve (R2 = 0.945, RMSE = 4.726, xpeak 1 = 0.713, xpeak 2 = 1.313, xpeak 3 = 2.3875) (Fig 1F). The mean response time for Ordinal Transfers undetected was 1.337 ± 0.097 sec and for Ordinal Transfers detected 0.710 ± 0.119 sec. Again, these data suggest that two distinct sequence decisions were being made.

A prediction from the these conflicting retrieval modes is that performance variability would be high between participants and across positions. In line with this hypothesis, we observed higher Ordinal Transfer variability when compared with Repeats, and much higher when compared with Skips (Mresidual = 0.795 ± 0.074) (Fig. 2D,E). A repeated-measure ANOVA with Greenhouse-Geisser correction showed significant main effect among regression residuals of Ordinal Transfers, Skips and Repeats (F(2,54) = 23.866, P = 1.843 × 10−8). Regression residuals of Ordinal Transfers were significantly higher compared with Skips (mean difference = 0.530 ± 0.089, P = 2.523 × 10−6) and Repeats (mean difference = 0.158 ± 0.028, P = 5.627 × 10−6). Regression residuals of Repeats were significantly higher compared with Skips (mean difference = 0.373 ± 0.099, P = 0.001).

mPFC activations and ordinal retrievals

We hypothesized that using an ordinal retrieval mode would be related to activations in the mPFC (e.g., Hsieh and Ranganath 2015). Algorithmically, this would be evident when contrasting undetected (i.e., using an ordinal retrieval mode) versus detected (i.e., using a nonordinal retrieval mode) during Ordinal Transfer trials. We also reasoned that the use of an ordinal retrieval mode would facilitate the detection of Skips and Repeats (because of the ordinal position mismatch), thus, when assuming an ordinal retrieval mode for Skips and Repeats, we evaluated activations contrasting detected versus undetected trials. We used an anatomical region of interest (ROI) analysis to evaluate mPFC activity across the three probe trial types. In support of our hypothesis the mPFC was significantly more active on Ordinal Transfers relative to both Skips and Repeats for contrasts that assumed ordinal retrieval modes (F(2,56) = 4.038, P = 0.034) (Fig. 2F). Post hoc tests (LSD) further supported the conclusion that the mPFC was most active in the comparison of undetected versus detected Ordinal Transfers (M = 0.708 ± 1.998) relative to detected versus undetected Skips (M = −0.649 ± 1.445, P = 0.051) and Repeats (M = −0.431 ± 1.728, P = 0.005). No significant difference was identified in mPFC activation between detected versus undetected Skips and Repeats (mean difference = 0.061 ± 0.437, P = 0.890). As a follow-up to explore contributions of distinct regions within our anatomical mPFC ROI we evaluated the same contrasts at the voxel-wise level. We observed activations throughout the mPFC bilaterally (prelimbic cortex, anterior cingulate cortex, medial superior frontal gyrus), following the comparison of undetected versus detected Ordinal Transfers (FWE-tfce P < 0.05) (Fig. 2G), while no mPFC clusters survived corrections for multiple comparisons when comparing detected versus undetected Skips (data not shown) and Repeats (FWE-tfce P > 0.05) (data not shown).

One potential confound is that mPFC activations might reflect relative performance levels (incorrect/undetected > correct/detected trials) rather than the utilization of ordinal retrieval modes per se. If true, we would expect similar mPFC activation clusters when examining incorrect compared with correct Skips and Repeats if performance was the main contributor to activation. When evaluating performance related activations for Repeats and Skips, undetected (i.e., incorrect) compared with detected (i.e., correct) Skips exhibited activations predominantly in rostrolateral prefrontal cortex (Fig. 2I) while a cluster for detected greater than undetected Repeats was identified in the right fusiform gyrus (data not shown). We explored this with a repeated measures ANOVA where we compared the mPFC and rostrolateral PFC and the three trial types (Ordinal Transfers, Skips, and Repeats incorrect—correct contrast). For this analysis we had to exclude five participants as these participants either did not detect any Ordinal Transfers or detected every single Repeat trial. There was no main effect for trial type F(2,56) = 0.395, P = 0.678, but there was a trend in main effect for brain region F(2,56) = 3.819, P = 0.061, and no significant interaction effect (F(2,56) = 1.149, P = 0.324). Although not significant, the trend in brain region suggests that unique processes within PFC subregions. These results further support the view that the mPFC contributes to ordinal retrieval modes. However, we were unable to isolate how much mPFC activity is attributable to performance accuracy when making ordinal decisions and/or how much is attributable other ordinal processes that can be explored in future experiments that explicitly manipulate difficulty levels with probe types. The observed activations in the rostrolateral prefrontal cortex during undetected Skips, and the fusiform gyrus for detected Repeats may reflect prospective memory (Umeda et al. 2011; Volle et al. 2011; Benoit et al. 2012) and object processing (Grill-Spector et al. 1998), respectively.

Sequence memory as a function of lag direction and distance

Memory for sequences of events at different lags can be supported by a variety of cognitive processes including working memory (WM) and temporal context memory (TCM) retrieval modes. The use of WM and TCM retrieval modes predict different patterns in behavioral performance across n-forward and n-reverse lags (see Fig. 3A,B; also see Jayachandran et al. 2019), which can be exploited here to test the use of different retrieval modes (e.g., WM versus TCM).

Figure 3.

BOLD fMRI analysis indicates that the right anterior hippocampus (raHC) tracks temporal context across forward lags. (A) Skips and Repeats occurred at different lags away from their InSeq position. (B) Theoretical predictions based on temporal context memory (TCM; blue lines) and working memory (WM; red line) retrieval modes for different lag distances on Repeats and Skips. A mixture of TCM and WM modes is shown by the dashed purple line. (C) The mean performance accuracy for all participants on Repeats and Skips at different lags. Response times for both Repeats (D, purple bars) and Skips (E, blue bars) presented a bimodal distribution and were best fit using a bimodal Gaussian distribution (black line) indicating two distinct decisions (InSeq vs. OutSeq). (F) Overall performance on Repeats was higher than for Skips. Performance was flat on Repeats at different lags reflecting the use of a mixture of TCM and WM retrieval modes (G), and performance has a positive slope on Skips defining of TCM retrieval modes (H). (I) There was significantly higher variability, measured by the averaged squared residuals per participant of the linear regression, in Repeats compared with Skips, which is thought to reflect the integration of variance from using both TCM and WM retrieval modes. (**) P < 0.01, (***) P < 0.001. (J) BOLD fMRI analysis of Skips with a linear contrast of −1, 0, and 1 for lags of +1, +2, and +3, respectively using the bilateral medial temporal lobe as a revealed significant activation in the raHC (right anterior Hippocampus). This activation pattern is unlikely due to general match/mismatch as we found no evidence of this in our brain region X retrieval mode interaction analysis. (K) BOLD fMRI analysis of Repeats with a linear contrast of 1, 0, and −1 for lags of −4, −3, and −2, respectively using bilateral medial temporal lobe as a mask showed no significantly activated clusters.

Skips (forward lags)

OutSeq probe trials that skipped ahead in the sequence (n-forward, e.g., the “D” in ABDDEF) afford the opportunity to evaluate predictions of the use of a TCM retrieval mode during rapid sequence memory decisions. Specifically, successful performance on Skips relies on a TCM retrieval mode as it requires participants to have precise expectations for the subsequent items in the sequence. Accordingly, the likelihood of a memory retrieval is highest for the very next item in the forward direction (lag distance = +1), and drops off (in a graded fashion) for more distal items (from +2 to +4). In the context of this task, the use of a TCM retrieval mode predicts the inverse in performance compared with free recall tasks because of interference. Specifically, performance on Skip OutSeq probe trials should be most difficult to detect for the shortest forward lag distance (+1), and improve at longer distances (+2, +3, or +4) (Fig. 3B, right) precisely because proximal items in a sequence (i.e., Skips with a short forward lag) are more likely to be retrieved (Howard and Kahana 2002; Kragel et al. 2015) and then falsely match up with an out of sequence probe image (and thus be judged as InSeq; undetected).

We tested this prediction by examining performance across n-forward lag distances. First, all forward lags were performed better than response bias chance levels (+1: 0.420 ± 0.178; +1vs. response bias: t(33) = 10.609, P = 3.608 × 10−12; +2: 0.665 ± 0.156; +2vs. response bias: t(33) = 21.299, P = 7.417 × 10−21; +3: 0.747 ± 0.248; +3vs. response bias: t(33) = 15.343, P = 1.402 × 10−16; +4: 0.765 ± 0.431; +4vs. response bias: t(33) = 9.066, P = 1.780 × 10−10) (Fig. 3C, blue bars), suggesting that the use of a TCM retrieval mode does not completely interfere with OutSeq detection at the different forward lag distances. Second, subjects exhibited graded performance improvement as the skip distance increased (F(3,99) = 13.790, P = 1.379 × 10−4). We found that the identification of Skips with a +1 lag were the most difficult to detect compared with all other forward lags (post hoc LSD: +2: mean difference = 0.245 ± 0.029, P = 1.218 × 10−9, +3: mean difference = 0.327 ± 0.049, P = 1.248 × 10−7, +4: mean difference = 0.345 ± 0.079, P = 1.186 × 10−4). These results are consistent with TCM retrieval mode predictions (Fig. 3B,C). Detection of Skips with a +2 lag was significantly lower than Skips with a +3 lag (post hoc LSD: mean difference = 0.082 ± 0.038, P = 0.039). No significant difference in detection was observed, however, when comparing Skips with lags of +2 and +4 (post hoc LSD: mean difference = 0.100 ± 0.074, P = 0.187), or +3 and +4 (post hoc LSD: mean difference = 0.018 ± 0.074, P = 0.814), suggesting that performance approached asymptote. These behavioral results suggest that the participants’ sequence memory was driven by the use of a TCM retrieval mode on Skips.

Repeats (reverse lags)

OutSeq trials that repeated an item from earlier in the sequence (n-reverse; e.g., the second “A” in ABADEF) (Fig. 3A) can be solved by either the use of a TCM retrieval mode, a WM retrieval mode, or a mixture of the two (Fig. 3B, left). Notably, the pattern of OutSeq detections across n-reverse lag distances should differentiate the two competing processes (Jayachandran et al. 2019). For Repeats, a TCM retrieval mode predicts that short backward lag distances (e.g., −2) would be the most difficult to detect due to heightened interference, while further n-reverse lag distances (e.g., −5) would be readily detectable as OutSeq. In contrast, a WM retrieval mode predicts the opposite pattern because the most recently experienced items would be the most accessible and therefore easiest to detect.

We found that performance was better than response bias chance levels for all n-reverse lag distances (−2: 0.859 ± 0.116; −2vs. response bias: t(33) = 38.333, P = 6.058 × 10−29; −3: 0.805 ± 0.171; −3vs. response bias: t(33) = 24.138, P = 1.522 × 10−22; −4: 0.782 ± 0.233; −4vs. response bias: t(33) = 17.204, P = 4.821 × 10−18; −5: 0.529 ± 0.507; −5vs. response bias: t(33) = 4.996, P = 1.865 × 10−5) (Fig. 3C, purple bars). As the n-reverse lag distance increased, detection as OutSeq decreased (F(3,99) = 9.760, P = 0.001) (Fig. 3C, purple bars), suggestive of the dominant use of a WM retrieval mode. Items with an n-reverse lag closest to their original position (e.g., ABCBEF; lag = −2) were easiest to detect as OutSeq compared with all other n-reverse lag positions (post hoc LSD lag −2 compared with lag −3: mean difference = 0.055 ± 0.022, P = 0.020; lag −4: mean difference = 0.077 ± 0.033, P = 0.025; or lag −5: mean difference = 0.330 ± 0.084, P = 4.262 × 10−4). In contrast, items with an n-reverse lag farthest from their original position (e.g., ABCDEA; lag = −5) were the most difficult to detect as OutSeq compared with all other n-reverse lag positions (post hoc LSD lag −5 compared with lag −4: mean difference = 0.253 ± 0.091, P = 0.009; lag −3: mean difference = 0.275 ± 0.091, P = 0.005; or lag −2: mean difference = 0.330 ± 0.084, P = 4.262 × 10−4). No significant differences in detection between −3 and −4 n-reverse lags were observed (post hoc LSD: mean difference = 0.022 ± 0.035, P = 0.532). Taken together, performance at the n-reverse lag extremes (−2 and −5) supports the notion that the use of a WM retrieval mode certainly plays an important role in identifying Repeats, but the absence of a graded performance across n-reverse lags of −3 and −4 supports the idea that a combination of cognitive processes is being used. Overall, these analyses do not support the use of a TCM retrieval mode as an isolated process driving the identification of Repeats in this task.

A two-term Gaussian curve (Fig. 3D,E, black line) fit best for the response time distributions of both Repeats (R2 = 0.950, RMSE = 4.4438, xpeak 1 = 0.688, xpeak 2 = 1.313) (Fig. 3D, purple bars) and Skips (R2 = 0.9205, RMSE = 4.0416, xpeak 1 = 0.713, xpeak 2 = 1.313) (Fig. 3E, blue bars), compared with a one-term Gaussian curve on Repeats (R2 = 0.8859, RMSE = 6.660, xpeak 1 = 0.688) and on Skips (Fig. 1F; R2 = 0.4198, RMSE = 10.796, xpeak 1 = 0.838) (Fig. 1F), or a three-term Gaussian curve on Repeats (R2 = 0.951, RMSE = 4.491, xpeak 1 = 0.686, xpeak 2 = 1.313, xpeak 3 = 2.113) (Fig. 1F) and on Skips (R2 = 0.921, RMSE = 4.087, xpeak 1 = 0.713, xpeak 2 = 1.313, xpeak 3 = 2.363) (Fig. 1F). The mean response times for detected Repeats was 0.710 ± 0.055 sec, for undetected Repeats was 1.320 ± 0.142 sec, for detected Skips was 0.729 ± 0.062 sec, and for undetected Skips was 1.357 ± 0.113 sec. These data suggest that two sequence decisions are being made on Repeats and Skips.

TCM retrievals for skips, but a mixture of retrieval modes for repeats

Differences in the overall ability to detect OutSeq probe trials, and the residuals from a lag-based linear regression model, helped to further elucidate the contributions of either a TCM retrieval mode, a WM retrieval mode, or their combination. First, we predicted that the ability to detect OutSeq probes would be greater for Repeats than Skips because TCM and WM retrieval modes can both contribute to the evaluation of Repeats but not Skips (Fig. 3B, left, purple dashed line). Second, we predicted that a linear regression of Skip detections would positively increase across n-forward lags, whereas a similar analysis across n-reverse lags for Repeats would be essentially flat. Third, we predicted that the residuals from the linear regressions would be, highest on Repeats compared with Skips, reflecting the use of multiple retrieval modes, whereas Skips involve a single mode (i.e., TCM-based retrieval). To test these predictions, we compared the overall ability to detect the two probe trial types, calculated a linear regression based on the z-scores (accounting for individual baseline detection levels) across different lag positions, and averaged the squared residuals as a measure of detection variability across lags. Participants were significantly better at detecting Repeats (Maccuracy = 0.786 ± 0.166) than Skips (Maccuracy = 0.492 ± 0.157; t(33) = 14.435, P = 8.137 × 10−16) (Fig. 3F).

Consistent with our hypothesis, the linear regression in Skips positively increased across forward lags accounting for a large effect on performance (R2 = 0.572, β = 0.757, P = 3.711 × 10−20) (Fig. 3H), whereas a linear regression across reverse lags for Repeats had no significant slope (R2 = 0.024, β = 0.154, P = 0.139) (Fig. 3G). When quantifying variability, Repeats (Mresidual = 0.593 ± 0.243) were significantly more variable compared with Skips (Mresidual = 0.285 ± 0.478) across different lag positions (Fig. 3I; t(30) = 3.051, P = 0.004). The patterns in OutSeq detection and variability further support the conclusion that multiple retrieval modes likely contribute to identifying Repeats as OutSeq, while Skip detection is mediated more exclusively by a TCM retrieval mode.

HC activations and TCM retrievals

Converging evidence indicates that the medial temporal lobe, specifically the HC formation, plays a disproportionate role in the use of a TCM retrieval mode in the brain (Manns et al. 2007; Hsieh et al. 2014; Bladon et al. 2019). To test whether regions of the medial temporal lobe contribute to a TCM retrieval mode during Skip and Repeat probe trials, we looked for linear changes in activation across the different lag distances. We observed a significant activation cluster in the right anterior HC following small volume corrections (bilateral medial temporal lobe, FWE-tfce P < 0.05) that increased its activation across n-forward lags (Fig. 3J). The statistical significance of similar patterns of activation across n-reverse lags did not survive corrections for multiple comparisons (Fig. 3K). These results suggest that the right anterior HC contributes to a TCM retrieval mode during Skips.

Brain region by retrieval mode interaction effect

After finding individual evidence supporting the mPFC contributes to sequence memory by the use of an ordinal retrieval mode and the right anterior HC by a TCM retrieval mode, we wanted to directly compare the mode-related activations (ordinal vs. TCM) across the regions that exhibited voxel-wise activations (right anterior HC vs. mPFC). A repeated-measures factorial ANOVA was conducted with brain region (anatomically defined right anterior HC vs. mPFC) and strategy (ordinal retrieval vs. TCM retrieval modes) as within-subjects factors, and BOLD activations as the dependent measure. We observed greater activations in the mPFC in relation to the use of an ordinal retrieval relative to a TCM retrieval mode, and the opposite pattern in the right anterior HC, evidenced by a significant interaction effect (F(1,31) = 4.782, P = 0.036) (Fig. 4A).

Figure 4.

mPFC and HC retrieval mode interactions. (A) There was a significant interaction effect between mode and brain region (F(1,31) = 4.782, P = 0.036), where we observed greater activations in the mPFC for ordinal retrievals relative to TCM retrievals, and the opposite pattern in the right anterior HC. (B) Based on the results of the current experiment, (1) the mPFC engages in ordinal-based retrieval calling on associations between items and their ordinal position (which could be represented elsewhere) and (2) the HC engages in temporal context-based retrieval.

Discussion

The current study used different out of sequence probe trials during a memory task to test the concurrent use of distinct retrieval modes, adding to a growing literature on sequence memory as a fundamental component of episodic memory (Tulving 1984; 2002; Allen and Fortin 2013; Howard and Eichenbaum 2013; Eichenbaum 2017). Complementary behavioral results across the different out of sequence probe trials in the current task support the conclusion that memory for sequences of events is supported by both ordinal and temporal context retrieval modes. The BOLD fMRI evidence showed that mPFC activations more strongly reflected the use of an ordinal retrieval mode, while activations in the HC better reflected a temporal context memory retrieval mode, thus dissociating the neurobiological substrates of two distinct processes contributing to sequence memory. While considerable evidence indicates that the mPFC and HC are involved in sequence memory, the current study provides new results that the mPFC and HC are concurrently engaged by retrieval modes in support of remembering when an event occurred. This evidence provides an important baseline for further investigation of how sequence memory is impaired in typical aging and diseases such as Alzheimer's disease on the neurobiological level, especially since evidence implicates that the relative dependence on an ordinal retrieval mode increases with age while TCM dependence decreases (Bastin and Van der Linden 2005; Allen et al. 2015).

Ordinal retrieval modes in mPFC

Our results suggest that the positional information associated with retrieving memories in sequence is, at least partially, represented in the mPFC (e.g., Hsieh and Ranganath 2015), and/or that mPFC activations help engage these representations elsewhere such as within HC neurons (e.g., Allen et al. 2016). Ordinal Transfers were detected less often than both Repeats and Skips (Fig. 2B; see also Allen et al. 2014, 2015), but better than response bias chance levels. This suggests that multiple retrieval modes are being used, feasibly the concurrent ordinal retrieval modes and temporal context retrieval modes. These results are not totally surprising because mPFC has been shown to be generally important to temporal order memory (Milner et al. 1985; Shimamura et al. 1990; DeVito and Eichenbaum 2011; Hsieh and Ranganath 2015) and other semantic representations (Preston and Eichenbaum 2013; Hyman et al. 2012). Our study suggests that when an ordinal retrieval mode is strongly engaged, it interferes with the ability to detect Ordinal Transfers, supported by the fact that Ordinal Transfers went undetected at a very high rate and mPFC activation was high at those times.

Ordinal retrieval modes have also been demonstrated in rats (Allen et al. 2014), monkeys (Orlov et al. 2000, 2006) and humans (Allen et al. 2015; Hsieh and Ranganath 2015). In fact, monkeys naturally categorize items within a sequence by their ordinal position that occurs early in the design by Orlov et al. (2000), and only later in trials do monkeys use other strategies such as sequential associations for adjacent items and working memory (Orlov et al. 2000, 2006). Likewise, in humans it has been shown that multivoxel patterns from the mPFC are significantly higher for objects that share the same position information, compared with objects in different positions (Hsieh and Ranganath 2015), suggesting convergent patterns of activation reflect shared ordinal representations within mPFC (Tiganj et al. 2017) despite sensory differences in object identity. Taken together, prior research and the current study show compelling evidence that the mPFC helps remember when events occurred by engaging an ordinal retrieval mode. Notably, while the design of our study facilitated our ability to evaluate different retrieval modes using specific out of sequence probe trials, it is likely that the very presence of out of sequence probe trials and the order in which they are introduced impacted the emergence and utilization of these retrieval modes. Although we observed little changes in behavior throughout the task here, it will be worth exploring both behavioral and brain activation patterns using versions of the sequence task that are designed to explicitly bias ordinal versus serial retrieval modes (Gudmundson et al. 2017).

Theoretically, an ordinal retrieval mode generated by mPFC would be input to the HC during episodic memory through indirect cortical or thalamic pathways (for review, see Dolleman-van der Weel et al. 2019). The engagement of an ordinal retrieval mode in HC might then allow for the rapid formation of conjunctive item-position representations (e.g., first—A, second—B, etc.) (Fig. 4B), and provide sequential structure without an explicit need to represent the elapsing time between items (a useful form of neural compression for temporal information). There is indirect evidence that item-position representations are reflected in rodent CA1 neurons during spatial sequence tasks (Euston and McNaughton 2006), and direct evidence for conjunctive item-position representations during an analogous odor sequence task (Allen et al. 2016). Importantly, item-position representations are learned and retrieved before sequential item-item associations in nonhuman primates (Orlov et al. 2000), although this may be task specific. This raises the question whether episodic memories typically rely on an ordinal retrieval mode for recalling (or encoding) events with timelines. Subsequent replay events or other consolidation processes could either strengthen temporal context modes and/or weaken ordinal modes, although we suspect the former. Interestingly, in monkeys, ordinal transfers show graded interference over lag distances, suggesting they share similar properties with temporal context retrievals when studied in this way (Orlov et al. 2006), indicating two temporal dimensions may be complementary and normally integrated. Future neuroimaging studies in humans using ordinal transfers distributed across lag distances will be useful for examine the neural activity when these processes are both contributing to retrieval patterns.

Temporal context retrieval modes in HC

Consistent with the literature, our results show that the HC contributes to sequence memory through the use of a temporal context retrieval mode. The HC has been shown to be important for sequence memory (Hsieh et al. 2014; Goyal et al. 2018) but its precise contribution has remained an open question. According to TCM, the HC associates items in sequences through a drifting contextual representation (Howard et al. 2005; Polyn and Kahana 2008). As such, the presentation or retrieval of an item from a sequence elicits the retrieval of neighboring items that share a temporal context, decreasing in likelihood or strength for more distal items (Howard and Kahana 2002). According to this framework, a temporal context retrieval mode accounts for the increased likelihood that adjacent items in word lists are recalled (Howard and Kahana 2002). This pattern in free-recall performance predicted by temporal context memory has been validated in computational models (Howard and Kahana 2002), behavioral studies (Kahana 1996; Sederberg et al. 2010; Morton and Polyn 2016), and neurobiological studies (Polyn and Kahana 2008; Jenkins and Ranganath 2010; Hsieh et al. 2014; Bladon et al. 2019). In the current sequence memory task, we reasoned that the presentation of InSeq items would elicit the retrieval of neighboring items that shared temporal contexts and lead to graded impairments in the detection of Skip probes. This is because, as the forward lag increased, the upcoming representations were less likely to be retrieved and thus less likely to interfere with an out of sequence determination. Similar patterns in performance are observed in other tasks probing temporal memory (Allen et al. 2014, 2015; DuBrow and Davachi 2014).

The HC has shown activations consistent with the use of a TCM retrieval mode. Activation in the HC is elevated during the processing of overlapping compared with nonoverlapping sequences (Kumaran and Maguire 2006; Brown et al. 2010; Brown and Stern 2013). Population activity in CA1 drifts across both small- and large-time scales (Manns et al. 2007; Mankin et al. 2012; Ziv et al. 2013; Rubin et al. 2015; Mau et al. 2018). Additionally, HC lesions impair the discrimination of overlapping odor sequences (Agster et al. 2002). An fMRI study showed that HC multivoxel pattern similarity was higher for pairs of adjacent trials that belonged to the same temporal context within a sequence compared with pairs of sequence items that bridged between sequences, even when the temporal distance between the pairs of items was similar (Hsieh et al. 2014). The same study observed that the HC carries information about the temporal context between items within a sequence, rather than information about the objects themselves (Hsieh et al. 2014). Specifically, Hsieh et al. (2014) demonstrated that when the same sequence item is repeated, hippocampal voxel patterns were dissimilar, unless the temporal context was reinstated. Our results add to this by showing that as Skips lag further away from their InSeq location, HC activity also increases closely matching predictions of TCM.

Limitations and theoretical considerations

While the use of specialized out of sequence probe trials provided important insight regarding different retrieval modes, and their related neurobiological substrates, several limitations of the current study remain. First, the task was designed for cross species investigation (Allen et al. 2014), thus the timing of the task and self-paced design of the experiment precluded detailed item-based analyses at the neurobiological level (see intertrial intervals in the Materials and Methods for relevant event timing). Our neuroimaging analysis thus, can only capture retrieval modes with uncertainties related to surrounding individual events. At the expense of repetitions of trials, future studies should include temporal jitter between sequence items to isolate signals at the different item positions. Second, similar to multivariate approaches, univariate approaches are subject to interpretational ambiguities (Ritchie et al. 2017; Hebart and Baker 2018). However, to avoid overfitting issues, especially with time-varying representations, we chose to use a univariate approach over a multivariate approach as it is better suited to our task design to differentiate retrieval modes using different out-of-sequence probe trials. Third, an in-sequence response bias could develop given the vast majority of individual items were in sequence. A response bias should impact all probe trial types similarly given their similar number of presentations. We did not observe evidence that performance was based on a response bias or changed over trials. All probe trial types were responded to better than chance and detailed response time analysis showed similar bimodal distributions across all conditions, notably lacking a rightward skew. While it would be helpful in the future to minimize a bias to hold, it does not change the overall interpretation of in sequence responses for any of our probe trial types. An additional limitation is that we are unable to cleanly isolate relative performance related brain activations and separate them from the retrieval mode activations on a given probe trial. This would be an interesting question to analyze in a future study using a modified task design that explicitly manipulates difficulty on all probe trials.

Key questions remain concerning the interactions between ordinal and temporal context retrieval modes, and their related neurobiological constituents (mPFC and HC). While the current study did not include probe trials that investigated ordinal transfer and transpositions, future studies should examine these critical probe trial types and expect to see increased interactions between the mPFC and HC. mPFC-HC coupling may lead to conjunctive representations of temporal contexts and positional coding. As shown here, mPFC activation contributes to ordinal retrieval modes (Fig. 4B) that interact with temporal contexts in the HC (Fig. 4B). In theory, this information could be merged allowing the formation of conjunctive temporal context and item-position association in HC neurons. Direct evidence for this latter possibility was provided by recent studies that showed HC neurons encode item-context and item-position conjunctions (Komorowski et al. 2009; Allen et al. 2016), suggesting that these conjunctive representations provide the neuronal basis for time and place integrations in episodic memory. However, future studies are required for examining mPFC-HC interdependent interactions directly in humans.

Conclusions

The results from our study highlight novel evidence that ordinal and temporal context retrieval modes both contribute to remembering items within a timeline. In particular, we showed that mPFC and HC activity differentially contribute to ordinal and TCM retrieval modes. Further experiments in both animals and humans are necessary to delineate the precise mechanisms by which the mPFC engages ordinal retrievals, and how this process interacts with HC to integrate ordinality and temporal contexts in support everyday episodic memory.

Materials and Methods

Participants

Thirty-nine right-handed volunteers were recruited from Florida International University (FIU) and University of Miami to perform a magnetic resonance imaging (MRI) study that included a sequence memory task designed to investigate the ability of humans to learn and remember arbitrary sequences of items, and a temporal reward discounting task. Task order was counterbalanced across all participants. Here we report results from the sequence memory task. All participants provided written consent in compliance with the local Institutional Review Board. Five participants were excluded from the final analysis due to failure to complete the task (n = 1) or because of poor performance (n = 4; sequence memory index score ≤0; see Sequence Memory Analysis section below). Participants excluded for poor performance had d-prime scores two standard deviations below the mean (M = 2.030, SD = 0.807). The final sample consisted of 34 individuals (19 females; mean age = 21 yr, SD = 2).

Task apparatus

The sequence task was run on a Dell computer using Matlab (R2015b) with custom scripts that included functions from Psychtoolbox (Psychtoolbox-3 distribution; http://www.psychtoolbox.org). Images were back-projected and viewed by participants with an angled mirror mounted on the head coil. Responses were recorded using a Current Designs MR-compatible four-button inline response device (https://www.curdes.com).

Experimental design and statistical analysis

Prescan training

All participants began with a practice session to become acquainted with the structure of the task and the method of responding. During the practice session, participants viewed four low-memory demand sequence sets, each comprising six unique images: (1) individual arrows at 0°, 60°, 120°, 180°, 240°, and 300°, presented in a clockwise fashion (Fig. 1A, Seq 1); (2) a dot moving from the top left to the bottom right corner (Fig. 1A, Seq 2); (3) bars of different colors moving from left to right (data not shown); and (4) letters A, B, C, D, E, and F positioned in the center of the screen (data not shown). Participants were asked to memorize sequences after a single “study” presentation during which the items were passively viewed. Later the practice sequences were tested with 15 unique memory probes per sequence. Testing was self-paced; each sequence was preceded by a screen with the words “Press the button to begin.” To initiate an image in the sequence, participants were required to press and hold a button. If the image was in sequence (InSeq), participants were instructed to hold down the button until the image disappeared on its own at 1 sec (the decision threshold), after which they could release the button. If the item was out of sequence (OutSeq), participants were instructed to release the button prior to the decision threshold (<1 sec), at which point the image would disappear upon button release. The self-pacing resulted in a mean interval between items in a sequence of 0.412 ± 0.532, and a mean interval between sequence sets of 0.760 ± 0.252. Prescan training was conducted on a Dell desktop computer.

Sequence memory task

During the Sequence Memory Task, participants were presented with two sequences from prescan training (low-memory demand) (Fig. 1A, Seq 1–2), and four novel sequences consisting of six unique fractal images (high-memory demand) (Fig. 1A, Seq 3–6). The exact composition of the novel fractal sequence sets was different for each participant. Sets were selected randomly, without replacement, from a bank of 240 unique fractal images. Similar to the prescan training, participants were asked to view and memorize the sequences after a single presentation in a passive viewing phase. Following the passive viewing phase, sequences were presented in a pseudorandom order, participants made judgments as to whether each item in a sequence was presented InSeq or OutSeq. Testing was self-paced and followed the same structure as the prescan training. The words “Press the button to begin” preceded the beginning of each sequence. A button depression with the right index finger initiated the presentation of each image in a sequence. If the participants judged the current image to be InSeq, they would hold the button until the decision threshold (1 sec, when the image disappeared), after which they could release the button. If the image was OutSeq, they were instructed to release the button before the 1 sec decision threshold. Sequence order was determined by the following rules: (1) Each sequence was presented first with all images in the correct order. (2) In the first half of testing (first 120 sequences), OutSeq items were either the same image appearing twice (Repeats) or an item appearing too early (Skips) (see probe trial description below). In the second half of testing (second 120 sequences), OutSeq items could now include images appearing from a different sequence, but in the correct ordinal position (Ordinal Transfers). Ordinal Transfers were introduced later so participants would not adopt an explicit ordinality strategy at the outset of the study given the dominance of ordinal retrieval modes early in sequence memory (Orlov et al. 2000). The six sequences were presented 40 times each, for a total of 240 sequence presentations. Half of the total sequences contained one OutSeq and five InSeq images; the remaining consisted only of correctly sequenced items. InSeq and OutSeq sequence sets were randomly presented throughout testing. The Sequence Memory Task consisted of 15-min blocks of continuous performance separated by a brief break (<1 min) to provide participants a rest. The number of blocks was dependent on the pace of the participant. Out of all the analyzed participants, most participants completed the task in four blocks (n = 29), while the remainder finished in either three (n = 1) or five (n = 4).

OutSeq probe trials

Three distinct types of OutSeq probe trials were used during the test phase: Ordinal Transfers, Repeats (reverse lags), and Skips (forward lags). OutSeq items were counterbalanced across sequence sets, never presented in first position (Pos1), and each OutSeq instance was unique. Due to the nature of the OutSeq trials, there could not be equal distribution of probe trials across all possible positions, and occurred as follows: Pos1, 0%; Pos2, 7.5%; Pos3, 17,5%; Pos4, 50%; Pos5, 17.5%; and Pos6, 7.5%.

Ordinal Transfers

OutSeq trials where an item from one sequence was “transferred” to a different sequence while retaining its correct ordinal position were considered Ordinal Transfers (Fig. 1B, Ordinal Transfer). For example, consider two sequences consisting of ABCDEF and UVWXYZ. An ordinal transfer into the first sequence might have Y at the fifth position ABCDYF. While Y occupies its original fifth position, it otherwise does not belong to the current sequence (Y does not normally follow D). Distribution of Ordinal Transfers across all possible positions: Pos1, 0%; Pos2, 7.5%; Pos3, 17,5%; Pos4, 50%; Pos5, 17.5%; and Pos6, 7.5%.

An OutSeq item was considered a Repeat if the image previously appeared in the current sequence and was repeated (Fig. 1B, Repeat). Repeats occurred at multiple lag distances, represented as negative values from repeat presentation to the original presentation, ranging from −2 (e.g., ABCBEF) to −5 (e.g., ABCDEA). A lag distance of −1 was not used due to the lack of an intervening item. Distribution of Repeats across all possible positions: Pos1, 0%; Pos2, 0%; Pos3, 7.5%; Pos4, 50%; Pos5, 27.5%; and Pos6, 15%.

OutSeq images were considered Skips when presented too early in the sequence (e.g., ABEDEF) (Fig. 1B, Skip). Skips occurred at all lag distances, represented as positive values from early presentation to original presentation, ranging from +1 (e.g., ABCDFF) to +4 (e.g., AFCDEF). Distribution of Skips across all possible positions: Pos1, 0%; Pos2, 15%; Pos3, 27.5%; Pos4, 50%; Pos5, 7.5%; and Pos6, 0%.

Sequence memory analysis

To evaluate whether participants demonstrated sequence memory, we compared the observed and expected frequencies of InSeq and OutSeq responses using G-tests (Sokal and Rohlf 1995). Responses to each item were sorted into a 2 × 2 matrix based on accuracy (correct/incorrect) and sequence condition (InSeq/OutSeq), as done previously (Allen et al. 2014; Jayachandran et al. 2019). For the sequence memory analysis, only “high memory” sequence sets were included. Responses to the first item of each sequence were excluded from analysis. Ordinal Transfers were excluded from this analysis because they were used to parse retrieval modes rather than measure performance. For comparison we evaluated sequence memory using d-prime, a measure of memory specificity derived from signal detection theory. Hits were defined as InSeq trials that were correctly identified as InSeq, misses were InSeq trials that were incorrectly identified as OutSeq, correct rejections were OutSeq probe trials that were correctly identified as OutSeq, and false alarms were defined as OutSeq probe trials that were incorrectly judged to be InSeq. The same conclusions were drawn with both approaches, thus, here we only report the results from the G-tests given its robustness to response biases as the current task has an overall greater number of InSeq responses.

Additionally, we examined overall sequence memory performance, using a summary statistic called the sequence memory index (SMI) (Eq. 1). SMI normalizes the proportion of InSeq and OutSeq items across different conditions and represents sequence memory performance as a single value ranging from −1 to 1. A SMI of “1” represents perfect sequence performance (response time >1 sec for all InSeq items and <1 sec for all OutSeq items), and 0 represents chance performance. Note that an SMI score of −1 would indicate an incorrect response to every single item (response times <1 sec for all InSeq items and >1 sec for all OutSeq items). Negative SMIs rarely occurred, poor performance was typically captured by SMI scores close to zero. SMI was calculated for both low and high memory demand sequences and has been used in previous human studies to facilitate comparisons (Allen et al. 2014, 2015).

Response time analysis

To evaluate whether participants made distinct InSeq and OutSeq decisions for the trial types, we analyzed the distribution of response times for all the participants for all InSeq trials, all OutSeq trials, and then the OutSeq trials divided up by Ordinal Transfers, Skips, and Repeats. A distribution was created with response times bins of 0.025 sec on the x-axis, and count on the y-axis for all the participant response times combined. If two distinct decisions were being made, we would expect to see a bimodal distribution; one peak occurring before 1 sec (OutSeq decision) and the other peak occurring after 1 sec (InSeq decision). To statistically evaluate whether the response time distribution fit better using a bimodal Gaussian distribution rather than a unimodal or trimodal Gaussian distribution, we fitted a one-term (unimodal), two-term (bimodal), and three-term (trimodal) Gaussian curve onto the data using cftoolbox in Matlab. To compare which Gaussian model was the best fit for the response time distribution, we recorded the R2, root mean squared error and the x-values of the peaks of the models.

Detailed position and lag analysis

To evaluate strategies or mechanisms of sequence memory performance, a detailed analysis based on ordinal position and lag distance was performed for each OutSeq probe type (Ordinal Transfers, Skips, and Repeats). For Ordinal Transfers, we evaluated performance across positions (Pos2 thru Pos6). For Skips, we evaluated performance across n-forward lag distances (Fig. 3A; n-forward lags: +1, +2, +3, and +4). Smaller n-forward lags occurred more often because more combinations were available. For Repeats, we evaluated performance across the n-reverse lags (Fig. 3A, n-reverse lags: −2, −3, −4, and −5). Smaller n-reverse lags occurred more often because more combinations were available. We compared each position or lag distance performance (accuracy and SMI) using repeated-measures ANOVAs followed by one-sample t-tests. To account for the potential of a response bias (the assumption by participants of each item being in sequence) in the accuracy measures, we calculated an adjusted response bias level as follows: (1) We calculated the observed response bias for holding the button for >1 sec on Pos 2 through Pos 6, irrespective of the InSeq or OutSeq status, which was 90.466%, and (2) we calculated the complement of that response bias and set that as the response bias for probe trials, which was 9.534%. To assess differences in variability among probe trial types trial specific performance was converted to a z-score across positions (Ordinal Transfers) and lags (Skips and Repeats) for each participant. Three mixed general linear models were performed with subject as a random effect and position or lag as a fixed effect. The resulting residuals were subsequently squared and averaged across positions or lags resulting in probe trial type specific residuals for each participant. We conducted a repeated measures ANOVA with Greenhouse-Geisser corrections and LSD post hoc testing to compare the averaged squared residuals across probe trial types (Ordinal Transfers, Skips, and Repeats).

Neuroimaging acquisition

Neuroimaging data were collected on a General Electric Discovery MR750 3.0T scanner using a 32-channel head coil at the University of Miami Neuroimaging Facility. Structural T1-weighted images were collected (186 slices, flip angle = 12°; TE = 3.68 msec, TR = 9.184 msec, TI = 650 msec, matrix = 256 × 256 mm; FOV = 256 mm, slice thickness = 1.0 mm). Whole-brain T2*-weighted, blood oxygen level dependent (BOLD) echo-planar imaging data (42 slices, interleaved, bottom-up; flip angle = 75°; TE = 25 msec; TR = 2000 msec; matrix = 96 × 96 mm; FOV = 240 mm, slice thickness = 3.0 mm, voxel size = 2.5 × 2.5 × 3.0 mm3) were collected during the task.

Neuroimaging preprocessing

Preprocessing was performed using a pipeline developed in Nipype v0.1 (Gorgolewski et al. 2011), wrapping tools from Analysis of Functional Neuroimages (AFNI v16.3.18) (Cox 1996), FSL (v5.0.10), FreeSurfer (v5.1.0) (Dale et al. 1999), Advanced Normalization Tools (v2.2.0) (Avants et al. 2008), and Artifact Detection Tools (ART; http://www.nitrc.org). Following DICOM conversion, cortical surface reconstruction and cortical/subcortical segmentation was performed on all T1-weighted structural scans. Functional data were first “despiked” to remove and replace intensity outliers in the functional time series. The data were simultaneously slice-time and motion corrected (Roche 2011). An affine transformation matrix was calculated during coregistration of the mean of each participant's functional scans to their structural scan using FreeSurfer's boundary-based registration algorithm (BBregister). Brain masks were created by binarizing the aparc + aseg (automatic cortical parcellation and automatic segmentation volume) file created using FreeSurfer, and dilating by one voxel. The resulting binary masks were subsequently coregistered to the functional data by applying the inverse of the affine coregistration matrix. Motion and intensity outliers were then identified using the rapid art artifact detection tool as implemented in nipype. Time-points at which intensity either exceeded three standard deviations or composite frame-wise displacement was >1 mm were flagged as outliers to serve as subsequent regressors of no interest in the first-level general linear models. Finally, functional data were spatially filtered with a 5 mm FWHM maximum Gaussian kernel using the SUSAN algorithm (FSL).

Neuroimaging normalization

A study-specific template was generated using Advanced Normalization Tools (ANT). Each structural scan was skull-stripped by multiplying the T1-weighted structural scan by the binarized and dilated aparc + aseg file in structural space. Each skull-stripped brain was then rigid-body-transformed (no scaling or shearing) to Montreal Neurological Institute space using FSL's FLIRT algorithm. This first pass was used to minimize large spatial shifts between participants and generate a template close to a commonly used reference. Following visual inspection, a study template was created using the buildtemplateparallel.sh script from ANTs. After template generation, each participant's skull-stripped brain was normalized using nonlinear symmetric diffeomorphic mapping implemented by ANTs. The resulting warps were applied to contrast parameter estimates following fixed-effects modeling for subsequent group-level tests.

Neuroimaging analysis

Functional data were analyzed according to a general linear model approach using FMRIB's Software Library (http://www.fmrib.ox.ac.uk/fsl). Two separate univariate general linear models were used for first-level analyses: (1) a performance (undetected/detected) model; and (2) a lag model.

All first-level models included event and nuisance regressors. The performance analysis contained eight event regressors of interest: detected and undetected InSeq, Repeats, Skips, and Ordinal Transfers. Event regressors were convolved with FSL's double γ hemodynamic response function with an onset beginning at the first stimulus item of a sequence and a duration equal to the length of time required to evaluate all the items in that sequence (mean = 9.917 sec per subject). The lag analysis comprised six event regressors of interest: Repeats at different n-reverse lags (n-reverse lags: −2, −3, and −4), and Skips at different n-forward lags (n-forward lags: +1, +2, and +3). InSeq and Ordinal Transfers irrespective of behavioral performance were also included. We evaluated linear changes in activation across lags with the following contrast weights: −1, 0, 1 and 1, 0, −1 for Repeats (n-reverse lags: −2, −3, and −4) and Skips (n-forward lags: +1, +2, and +3). We did not include a lag of −5 or +4, as the numbers of trials with these lags per participant was not sufficient (one trial). Regressors for baseline or low memory sequences were also included in both models. Nuisance regressors included motion parameters (x, y, and z translations; pitch, roll, and yaw rotations); first and second derivatives of the motion parameters; normalized motion; first-, second-, and third-order Lagrange polynomials; and each outlier time-point that exceeded the artifact detection thresholds identified during preprocessing.

Following the first-level analysis, a fixed effects analysis across experimental runs was performed for each participant for the respective contrasts of interest (e.g., performance analysis: undetected vs. detected probe trials; lag analysis: linear contrasts). The performance analysis contrasts were limited to sequences with undetected versus detected probe trials in an attempt to maintain parity in the number of events that were being compared. Contrast parameter estimates from the fixed effects analysis were normalized to the study-specific template and group-level analyses were performed using FSL's randomize threshold-free cluster enhancement (tfce) one sample t-test. To test a priori hypotheses with respect to the functional contributions of the mPFC and HC during sequence memory retrieval, we constrained our voxel-wise analyses at the group level to the bilateral mPFC and medial temporal lobe using masks of these regions. Whole-brain exploratory analyses were used to follow up our anatomically directed tests. In the performance analysis, some participants were missing key events of interest (e.g., detected Ordinal Transfers or undetected Repeat probes). These participants were not included in the relevant analyses, reducing the sample size for the detected versus undetected Ordinal Transfer contrast to n = 26, and for the detected versus undetected Repeat probe trial contrast to n = 31.

To directly examine the relation between sequence memory retrieval mode and regional contribution, as a post hoc follow up to our voxel wise analyses, we used an anatomical region of interest analysis to perform a 2 × 2 repeated measures factorial ANOVA with brain region (right anterior HC vs. mPFC) and strategy (ordinal retrieval mode [Ordinal Transfer undetected > Ordinal Transfer detected] vs. temporal context retrieval mode [TCM; positive linear Skip contrast]) as the within subjects factors and participants as the repeated measure. Anterior HC region of interest was manually drawn in coronal slices. The anterior most boundary of the hippocampus was defined by the white matter separating the hippocampus from the amygdala, the lateral and medial boundaries were defined by the cerebral spinal fluid of the lateral ventricle, the inferior boundary was white matter of the parahippocampal gyrus, while the superior boundary was the wavelike contour of the pes digitations/alveus/horizontal line connecting the middle of the medial border of the lateral ventricle to the surface of the uncus. The anterior HC was delineated from the posterior HC by the presence of a small anatomical protrusion of medial HC into the lateral ventricle that was absent in the posterior HC–uncal apex. The mPFC region of interest was created by binarizing the medial orbitofrontal, superior frontal, rostral anterior cingulate, and the caudal anterior cingulate FreeSurfer labels.