Across time and space: spatial-temporal binding under stress.

Successful episodic memory requires binding of event details across spatial and temporal gaps. The neural processes underlying mnemonic binding, however, are not fully understood. Moreover, although acute stress is known to modulate memory, if and how stress changes mnemonic integration across time and space is unknown. To elucidate these issues, we exposed participants to a stressor or a control manipulation shortly before they completed, while electroencephalography was recorded, an encoding task that systematically varied the demands for spatial and temporal integration. Associative memory was tested 24 h later. While early event-related potentials, including the P300 and Late Positive Component, distinguished different levels of spatiotemporal discontinuity, only later Slow Waves were linked to subsequent remembering. Furthermore, theta oscillations were specifically associated with successful mnemonic binding. Although acute stress per se left mnemonic integration largely unaffected, autonomic activity facilitated object memory and glucocorticoids enhanced detail memory, indicative for mnemonic integration. At the neural level, stress amplified the effects of spatiotemporal discontinuity on early information processing. Together, our results indicate that temporal and spatial gaps recruit early neural processes, providing attentional resources. The actual binding success, however, appears to depend on later processes as well as theta power and may be shaped by major stress response systems.

Episodes of experience unfold over time and are made up of multiple interrelated stimuli. Episodic memory requires binding these stimuli into a durable and coherent trace, thereby bridging spatial and temporal discontinuities between event elements. Such spatial-temporal integration in episodic memory is thought to be a core function of the hippocampus (Wallenstein et al. 1998; Eichenbaum 2017). Indeed, several studies using functional magnetic resonance imaging (fMRI) in humans showed that the hippocampus is involved in successful binding of elements presented discontinuously across time and space (Davachi and Wagner 2002; Jackson and Schacter 2004; DuBrow and Davachi 2016). Moreover, one study varied the demands for spatial-temporal integration systematically and showed that the engagement of the hippocampus during successful relational binding was directly modulated by the extent of spatial and temporal discontinuities (Staresina and Davachi 2009). While these findings demonstrate convincingly that the hippocampus is implicated in spatial-temporal binding, fMRI has only a low temporal resolution and the neuronal dynamics underlying mnemonic binding across time and space remain largely elusive.

In addition to the neural dynamics of spatial-temporal integration, the factors that may modulate mnemonic binding across spatial and temporal gaps are not well understood. It is, however, well established that acute stress is a powerful modulator of hippocampal functioning. Several of the many hormones and neurotransmitters that are released during stressful encounters, in particular glucocorticoids and catecholamines, are known to affect hippocampal activity and neuroplasticity (Kim and Diamond 2002; de Quervain et al. 2003; Roozendaal et al. 2004; Diamond et al. 2006; Henckens et al. 2012; Qin et al. 2012; Schwabe et al. 2012b; Zoladz et al. 2012). Accordingly, it has been shown repeatedly that stress and stress hormones may alter memory processes that are dependent on the hippocampus (de Quervain et al. 1998; Smeets et al. 2007; Schwabe et al. 2009; Schmidt et al. 2014; Dandolo and Schwabe 2016), albeit the direction of these stress effects may vary, for example, depending on the exact timing of the stress exposure or the specific function tested. Specifically, stress around the time of learning is thought to facilitate mnemonic processing, whereas stress at longer time intervals before learning is assumed to have detrimental effects (Diamond et al. 2007; Joëls et al. 2011; Zoladz et al. 2011; Schwabe et al. 2012a). Moreover, there is initial evidence that stress and glucocorticoids might influence the formation of associative memory (van Ast et al. 2013, 2014; Goldfarb et al. 2019). Based on this evidence, we hypothesize that stressful events affect, through the action of glucocorticoids and catecholamines, mnemonic binding across spatial and temporal discontinuities. Given that previous studies on stress and learning yielded inconsistent findings (Schwabe et al. 2012a; Shields et al. 2017), it is difficult to predict whether stress would facilitate or impair mnemonic binding. Although such stress-induced changes in spatial-temporal integration would be highly relevant for our understanding of episodic memory in general and potentially for eyewitness testimony or stress-related psychopathologies, this hypothesis has not been tested so far.

Therefore, the aims of the present study were twofold: we aimed (i) to elucidate the neural dynamics of spatial-temporal binding in episodic memory and (ii) to test whether this binding and its neural underpinnings are modulated by acute stress. To this end, we first exposed healthy volunteers either to the socially evaluated cold pressor test (SECPT, Schwabe et al., 2008b) or a nonstressful control manipulation. Twenty-five minutes later, when cortisol—a known modulator of hippocampal functioning (Kim and Diamond 2002; Diamond et al. 2006; Lovallo et al. 2010; Henckens et al. 2012)—was expected to be elevated, participants completed a memory encoding task probing temporal-spatial integration (Staresina and Davachi 2009). In this task, we varied systematically the extent to which a target and an event detail had to be integrated across spatiotemporal discontinuities (Fig. 1). Specifically, the event detail was either combined within the target, spatially separated from the target or presented spatially as well as temporally separated. About 24 h later, item and event detail memory were tested. In order to track the neural dynamics of spatial-temporal integration, we recorded electroencephalography (EEG) during encoding. Although EEG does not provide information about hippocampal involvement, its high temporal resolution makes it exceptionally well suited for studying the neural dynamics involved in spatial-temporal integration. Our EEG analyses focused on event-related potentials (ERPs) which have been linked to successful memory formation. Namely, we analyzed the P300 (Karis et al. 1984; Kamp et al. 2013), a component reflecting the early allocation of attention to the encoding material (Polich and Kok 1995; Polich 2007; Olofsson et al. 2008). Additionally, we analyzed the Late Positive Component (LPC, Fernández et al. 1998; Friedman and Johnson 2000) as well as Slow Waves (Weyerts et al. 1997; Mangels et al. 2001; Kim et al. 2009). Both later components have been linked to successful associative encoding before (Kim et al. 2009; Kamp and Zimmer 2015) and were expected to also be critical for spatial-temporal integration. Further, there is first evidence that the effects of stress on memory formation are also reflected in the Slow Wave (Kamp et al. 2018). Beyond the ERP components, we focused on brain oscillations, especially in the theta band, which have been linked to successful associative memory formation (Summerfield and Mangels 2005; Clarke et al. 2018) and might thus be important for mnemonic binding across spatial and temporal gaps.

Figure 1.

Experimental procedure. On day 1, participants underwent either a stressor or a nonstressful control manipulation before encoding photographs of 300 everyday objects. The color of these object was either presented together with the object (combined), in a frame around the object (spatial) or in a frame before object presentation (spatial-temporal), thus posing different demands of spatial-temporal integration on the participants. On day 2, 24 h later, participants returned for a self-paced recognition and color memory test. If participants indicated a presented object was “old,” they were subsequently asked for the color of the object and indicated the confidence of their color judgement. Green frames indicate responses of a fictive participant.

Results

Successful stress induction through the SECPT

Linear regression models were used to assess the impact of the SECPT, compared to the control manipulation, on subjective and physiological measures. The comparisons over time reflect sliding difference contrasts between neighboring time points, separately for each group. These analyses confirmed that the exposure to the SECPT, but not the control manipulation, led to significant increases in subjective stress, blood pressure, pulse, and salivary cortisol (Fig. 2). Participants in the stress group experienced the manipulation as significantly more challenging, painful, stressful and unpleasant, compared to participants in the control condition (all t(76) > 9.52, all P < 0.001; Table 1). Neither positive nor negative subjective mood changed over the course of day 1 (Main effect time: all |t| < 1.18, all P > 0.238, all |β| < 0.47; time × group interaction: all |t| < 0.75, all P > 0.452, all |β| < 0.42), which is not surprising given that the used mood questionnaire was rather broad and not very specific to stressful events. Systolic and diastolic blood pressure as well as pulse increased from baseline to the measurement during the SECPT (all t > 6.04, all P < 0.001, all β > 1.23) and decreased after the SECPT (all t < −7.72, all P < 0.001, all β < −1.58). Likewise, salivary cortisol increased significantly in the stress group after the SECPT (t = 2.91, P = 0.004, β = 0.58) and decreased over the first half of encoding (t = −5.33, P < 0.001, β = −1.07). In the control group, in turn, there was no increase in systolic blood pressure over the course of day 1 (all |t| < 1.39, all P > 0.166, all |β| < 0.41), while diastolic blood pressure first increased from baseline to the measurement during the manipulation (t = 2.73, P = 0.007, β = 0.44), then decreased to the measurement 25 min after manipulation (t = −4.09, P < 0.001, β = −0.67) and tended to increase during the first half of learning (t = 1.71, P = 0.090, β = 0.27). Pulse first decreased (baseline vs. during manipulation: t = −6.66, P < 0.001, β = −0.97), increased afterwards (during manipulation vs. 25 min after manipulation: t = 2.33, P = 0.021, β = 0.34) and finally decreased during the second half of learning (t = −3.28, P = 0.001, β = 0.48). Furthermore, salivary cortisol concentrations in the control group decreased significantly from baseline to the measurement 25 min after manipulation (t = −3.82, P < 0.001, β = −0.50) and on trend-level during the first half of learning (t = −1.75, P = 0.084, β = −0.23), most likely due to the diurnal rhythm of cortisol.

Figure 2.

Physiological stress response. In response to the socially evaluated cold pressor test (SECPT), but not in response to the control manipulation, there were marked increases in (A) systolic blood pressure, (B) diastolic blood pressure, (C) Pulse and (D) salivary cortisol. Error bars represent standard error of the mean (SEM). (#) P < 0.1 and (***) P < 0.001 for stress group versus control group.

Table 1.

Subjective response to the stress manipulation

Additionally, t-tests were used to test for group differences at each time point. For systolic and diastolic blood pressure as well as pulse, the groups differed during the manipulation (all t > 4.64; all P < 0.001) but not during baseline measurements (all t < 0.40; all P > 0.691) or after the manipulation (all t < 1.80; all P > 0.076). For salivary cortisol, the groups did not differ at baseline (t(73.45) = 0.30; P = 0.764) but 25 min after manipulation, when the task started, the stress group had significantly higher salivary cortisol levels than the control group (t(51.80) = 3.90; P < 0.001) and these differences remained, at trend-level, during the encoding task (t(61.44) = 1.86; P = 0.067). There were no group differences in salivary cortisol levels after the encoding task t(71.22) = 0.70; P = 0.487). The groups did not differ in positive or negative mood in any of the measurements during day 1 (all t < 1.19; all P > 0.238).

Distinct roles of cortisol and autonomic activity in object and color memory

Twenty-five minutes after the onset of the SECPT or control manipulation, when the peak in stress-induced cortisol was expected, participants completed, while EEG was recorded, a memory encoding task that has been introduced before to probe mnemonic binding across spatial and temporal gaps (Staresina and Davachi 2009). In brief, participants were presented objects and were requested to imagine (and memorize) the shown object in a given color. Three different trial types were implemented (Fig. 1): (i) in “combined” trials the object was digitally colored, that is, it was presented in a specific color and therefore required only minimal integrative effort; (ii) in “spatial” trials the object was presented in grayscale at the center of the screen and the object color was presented in a frame around the object, thus requiring participants to integrate object and object color spatially; and (iii) in “spatial-temporal” trials the object was again presented in grayscale at the center of the screen but this time the frame indicating the color of the object was presented 1000 msec before the object itself was presented in grayscale, thus requiring participants to integrate object and object color across time and space. Memory for object and object color were tested 24 h after encoding. In this 24 h-delayed memory test, participants correctly recognized 61 percent of the objects they had seen 24 h before (SD = 20 percent, combined: M = 61, SD = 21, spatial: M = 61, SD = 20, spatial-temporal: M = 61, SD = 20, see Supplemental Fig. S1) and for 42 percent of these (SD = 14 percent, combined: M = 47, SD = 16, spatial: M = 42, SD = 12, spatial-temporal: M = 37, SD = 12), the color was successfully remembered as well. While the trial type (combined, spatial, spatial-temporal) left object memory unaffected (combined vs. spatial: z = −0.59, P = 0.552, β = −0.01; spatial vs. spatial-temporal: z = −0.30, P = 0.767, β < 0.01), the extent of temporal and spatial discontinuity of color and object during item picture presentation had a significant impact on subsequent color memory. As shown in Figure 3, the spatial discontinuity hampered subsequent color memory (combined vs. spatial: z = −2.13, P = 0.033, β = −0.04) and the spatial-temporal discontinuity led to an even more severe impairment in color memory (spatial vs. spatial-temporal: z = −3.65, P < 0.001, β = −0.07). Linear regression models analyzing the confidence ratings of the correct color judgements revealed a trend for higher confidence ratings in the stress group than in the control group (t = −1.74, P = 0.089, β = −0.06). We accounted for this difference by including confidence judgements as a predictor in the models of color memory and electrophysiological data.

Figure 3.

Memory performance. (A) While there was no effect of trial type (combined, spatial, spatial-temporal) on recognition performance, (B) color memory was significantly impeded by increasing discontinuity. (C) Recognition memory was positively correlated with systolic blood pressure during the manipulation. (D) Color memory was positively correlated with peak salivary cortisol levels. Error bars represent SEM. (*) P < 0.05 (***) P < 0.001.

Beyond this trend for higher confidence in correct color memory judgements, stress per se had no significant influence on object recognition memory (z = −1.47, P = 0.143, β = −0.18), nor on color memory (z = −0.08, P = 0.934, β = 0.00). However, in order to elucidate the influence of the physiological stress response on memory performance, we fit in a next step additional models with the predictors salivary cortisol (using the measurement 25 min after stress induction, when cortisol reached peak levels and the encoding task started) and autonomic response (systolic blood pressure during the manipulation). Interestingly, these analyses revealed distinct effects of cortisol and autonomic response on memory performance: Salivary cortisol was not associated with object memory (z = 1.14, P = 0.255, β = 0.03), but was positively associated with color memory (z = 2.46, P = 0.014, β = 0.03). Conversely, systolic blood pressure was positively associated with object memory (z = 2.08, P = 0.037, β < 0.01), but not with color memory (z = 0.43, P = 0.666, β < 0.01; Fig. 3).

Event-related potentials linked to successful color memory

In order to identify processes critical to mnemonic integration, we fitted linear mixed models (LMMs) by modeling the influence of group, trial type, and subsequent color memory (successful color memory vs. mere object recognition) on the mean amplitude of each ERP component.

P300

Attention to the presented object, as indicated by the P300 amplitude, was significantly influenced by the trial type. As shown in Figure 4, the P300 amplitude was reduced for spatial compared to combined trials (t = −7.55, P < 0.001, β < −0.27). However, there was no difference between spatial and spatial-temporal trials (t = −0.21, P < 0.836, β = −0.01) suggesting that spatial discontinuity, but not additional temporal discontinuity affected the P300 amplitude. No other effects of stress or subsequent memory reached statistical significance (all |t| < 1.37, all P > 0.169, all |β| < 0.30).

Figure 4.

ERP amplitudes for the posterior electrode cluster. Amplitude differences due to discontinuity and (A) subsequent memory and (B) stress. ERP amplitude was decreased for spatial trials in (C) the P300 and (D) the LPC. (E) The stress effect in the posterior Slow Wave amplitude was modulated by subsequent color memory. Error bars represent SEM. (**) P < 0.01 (***) P < 0.001.

In order to take the physiological stress response into account, additional models were established with the predictors salivary cortisol peak (measurement 25 min after the stress/control manipulation) and systolic blood pressure during the manipulation instead of group. Systolic blood pressure during the manipulation was positively associated with P300 amplitude (t = 2.40, P = 0.019, β = 0.03). Additionally, there was a trend-wise salivary cortisol × subsequent memory interaction (t = −1.77, P = 0.077, β = −0.02), driven by a more positive effect of salivary cortisol for successful color memory trials than on mere object recognition trials. No other effects of blood pressure or peak salivary cortisol response reached statistical significance (all |t| < 1.61, all P > 0.108, all |β| < 0.06).

LPC

The LPC amplitude, probably reflecting early, item-level processing of the presented object (Kim et al. 2009; Weymar et al. 2012) was lower for spatial trials compared to combined and, interestingly, also compared to spatial-temporal trials (combined vs. spatial: t = −8.40, P < 0.001, β = −0.30; spatial vs. spatial-temporal: t = 3.12, P = 0.002, β = 0.11). Further, there was a trend-level trial type × subsequent memory interaction (t = 1.84, P = 0.067, β = 0.07), driven by a stronger decrease in amplitude for spatial compared to combined trials when color was later correctly remembered than when only the object was recognized. No other effects containing the factors group, trial type, or subsequent memory reached statistical significance (all |t| < 1.62, all P > 0.105, all |β| < 0.07). Models examining the effect of the physiological stress response (systolic blood pressure and salivary cortisol) did not reveal any additional effects (all |t| < 1.51, all P > 0.134, all |β| < 0.02).

Slow waves

The anterior Slow Wave, an indicator of deep encoding (Guo et al. 2004), was more positive for subsequently correct color memories compared to trials with only correct object recognition (t = −1.96, P = 0.050, β = −0.10). Further, there was a trend-level increased amplitude for spatial compared to combined trials (t = 1.78, P = 0.075, β = 0.07). No other effects of group, subsequent memory or trial type reached significance (all |t| < 1.51, all P > 0.134, all |β| < 0.02). Modeling the anterior Slow Wave amplitude with the physiological stress response (salivary cortisol and systolic blood pressure) instead of group revealed a trial type × salivary cortisol interaction (t = −2.52, P = 0.012, β = −0.02), suggesting a positive association of salivary cortisol with the anterior Slow Wave amplitude for combined trials but no effect for spatial trials. Apart from that, the models yielded only trend-level effects which were not possible to follow up (t = 1.66, P = 0.097, β = 0.01) and nonsignificant effects (all |t| < 1.61, all P > 0.108, all |β| < 0.03).

The posterior Slow Wave was significantly decreased for spatial compared to combined trials (t = −4.01, P = 0.001, β = −0.17) and trend-wise also for spatial compared to spatial-temporal trials (t = 1.83, P = 0.068, β = 0.08). Thereby, spatial trials yielded the lowest posterior Slow Wave amplitudes, similar to the dynamics observed in the LPC. There was no main effect for correct color memories compared to mere object recognition (t = −0.38, P = 0.708, β = −0.02), however, there was a significant subsequent color memory × group interaction (t = −2.07, P = 0.038, β = −0.11). This interaction was driven by a more negative effect of subsequent color memory on the Slow Wave amplitude in the stress group compared to the control group. No other effects of trial type, group or subsequent memory reached significance (all |t| < 1.49, all P > 0.137, all |β| < 0.07). The model was additionally run using the physiological stress response (salivary cortisol and systolic blood pressure) as a predictor, but this model yielded no significant results (all |t| < 1.49, all P > 0.135, all |β| < 0.06).

In sum, spatial-temporal discontinuities affected early as well as late ERPs. In particular, the bridging across spatial gaps resulted in marked changes in amplitude for all analyzed components, while spatial-temporal gaps had rather subtle effects appearing only in later components, namely the LPC and Slow Waves. Further, the later components, especially Slow Waves, but on trend-level also the LPC, yielded subsequent memory effects, suggesting that they reflect processes critical to successful memory integration. Stress modulated this subsequent memory effect in the posterior Slow Wave.

Brain oscillations linked to successful color memory

In the next step, we examined time-frequency clusters of oscillatory power, which were critical for mnemonic binding as reflected in successful color memory. For the anterior electrodes, the cluster-based permutation test comparing trials with later successful color memory to trials with mere object recognition, identified two significant time-frequency clusters (see Fig. 5): the first one spanned the time window from 500 to 1500 msec after stimulus onset and the theta frequency (4–8 Hz). The second cluster began 1500 msec after stimulus onset lasting to the end of the stimulus presentation period (2000 msec after onset) and extended over the theta and alpha frequency range (4–15 Hz). No significant cluster was found for the posterior electrodes, so oscillatory power was only analyzed for the two clusters found in the data from the anterior electrodes.

Figure 5.

ERP amplitudes for the anterior electrode cluster. (A) Amplitude differences due to discontinuity and subsequent memory. (B) Anterior Slow Wave amplitude was significantly increased for correct color memory compared to trial with mere object recognition. (C) Amplitude differences due to discontinuity and stress. All error bars represent SEM. (*) P < 0.05.

Early time-frequency cluster

Oscillatory theta power in the early cluster was decreased in trials for which the object color was subsequently correctly remembered (vs. trials with correct object but incorrect color memory: t = 2.09, P = 0.037, β = 0.07). Apart from that, an effect of trial type emerged, with decreased theta power for spatial compared to combined trials (t = −2.33, P = 0.021, β = −0.04), while spatial and spatial-temporal trials did not differ significantly (t = 0.65, P = 0.519, β = 0.01). This main effect was further qualified by a trend-level trial type × subsequent memory interaction (t = 1.69, P = 0.093, β = 0.03), suggesting a more pronounced decrease in oscillatory power for spatial compared to combined trials when color was later correctly remembered. Apart from that, there were only trend-level effects, which could not be resolved in a meaningful way (subsequent memory × group × trial type: t = −1.93, P = 0.055, β = −0.04), and nonsignificant effects (all |t| < 1.10, all P > 0.270, all |β| < 0.02). Additional models testing the effects of salivary cortisol and systolic blood pressure yielded no reliable effects (all |t| < 1.90, all P > 0.06, all |β| < 0.01).

Late time-frequency cluster

Power in the second cluster spanning theta and alpha frequencies was decreased for correct color judgements compared to mere object recognition (t = 3.40, P < 0.001, β = 0.11). In addition, spatial trials were associated with lower theta-alpha power compared to combined trials (t = −2.90, P = 0.004, β = −0.06), whereas spatial and spatial-temporal trials did not differ significantly (t = 0.50, P = 0.619, β = 0.01). In addition, separate models were run using the physiological stress response (salivary cortisol levels and systolic blood pressure) as predictors instead of group. However, no significant effects of salivary cortisol or systolic blood pressure emerged (all |t| < 1.49, all P > 0.136, all |β| < 0.03).

Taken together, spatial discontinuous trials were associated with theta power decreases, similar to the effects observed in the ERP components. However, this discontinuity effect was only present in trials with later color memory, suggesting that these power decreases in the theta band are directly linked to the successful bridging of spatial gaps. However, stress did not affect oscillatory power in the context of subsequent color memory.

Although we focused here on the neural processes involved in mnemonic integration, as reflected in color memory, we also analyzed neural processes implicated in object memory. These additional analyses are presented in the Supplemental Material.

Visual control condition

In order to rule out the possibility that potential effects of the spatial-temporal discontinuity (i.e., the trial type) are confounded by visual processing, we included also visual control trials, in which a colored frame was presented, which was, however, not followed by an object. We contrasted these visual control trials (presentation of a colored frame without an object) against all other trial types for the analyzed ERP components as well as time-frequency clusters. For the earlier ERP components, the P300 and LPC, the amplitude was decreased for visual control trials compared to the other trial types (both t > 6.08, both P < 0.001, both β > 0.43). Further, oscillatory power in both the early and the late time-frequency cluster was higher for visual control trials compared to the other trial types (t = 2.16, P = 0.032, β = 0.35 and t = 7.16, P < 0.001, β = 1.09, respectively). Thus, integrative processing of an object with a color elicited neural activity beyond the processing of a color alone, showing that the analyzed electrophysiological measures are sensitive to the cognitive processes underlying memory integration and not only to the perceptual processing of a presented color. For the anterior and posterior Slow Waves, the visual control trials did not differ from the other trial types (both |t| < 1.51, both P > 0.133, both |β| < 0.07). This lack of a difference may be owing to noise accumulating over the course of the trial effectively aligning the different waveforms and making it difficult to identify differences between visual control trials and other trial types. However, as later components are known to reflect top-down rather than bottom-up processes (Olofsson et al., 2008), it is unlikely that Slow Wave effects in our study are affected by perceptual processing.

Control variables

The stress and control groups did not differ in subjective chronic stress levels, depressive mood, state or trait anxiety (all t(79) < 0.39, all P > 0.698; see Table 2). Further, the groups were similar in their initial levels of blood pressure, salivary cortisol, and subjective mood on day 1 (all t < 1.19; all P > 0.238) as well as on day 2, before memory testing (all t < 0.94, all P > 0.351). Linear regression models predicting the plausibility rating on day 1 with trial type and group revealed a main effect of trial type, with lower plausibility ratings for spatial compared to combined trials (t = −2.87, P = 0.004, β = −0.05) and, trend-wise, lower plausibility ratings for spatial compared to spatial-temporal trials (t = 1.92, P = 0.056, β = 0.03). There were no significant group differences in the plausibility rating (t = 0.17, P = 0.869, β < 0.01). We accounted for the trial type differences in plausibility rating by including it as a predictor in the (generalized) LMMs of behavioral and electrophysiological data.

Table 2.

Control variables

Discussion

Binding spatially and temporally separated information into a unified representation is critical for episodic memory formation. Here, we aimed (i) to elucidate the neural dynamics involved in successful bridging across temporal and spatial gaps and (ii) to assess how acute stress affects the spatial-temporal integration in episodic memory as well as the involved neural processes. Our results show that early neural processing, reflected in the P300 and LPC, was modulated primarily by the spatial discontinuity of the encoded information but less predictive for successful memory formation. Later processing, expressed as Slow Waves, in turn, was less affected by spatial and temporal gaps per se but directly linked to successful subsequent remembering. Furthermore, theta oscillations were linked to successful color memory. This binding-related theta effect was modulated by the spatial discontinuity, with a stronger theta decrease linked to higher binding demands. Stress per se had no effect on memory performance. However, the activity of the two major stress systems, reflected in blood pressure and cortisol levels, appeared to be differentially involved in mnemonic binding. Whereas blood pressure increases were specifically linked to later object memory, cortisol elevations were specifically associated with subsequent color memory. At the neural level, stress modulated the effects of spatial-temporal discontinuity on electrophysiological correlates of memory formation, with differences due to spatial discontinuity being more pronounced after stress.

On a behavioral level, we observed decreasing associative memory performance with increasing spatial-temporal discontinuity, while item memory was unaffected. This is consistent with the idea that such discontinuities increase the difficulty of memory binding and thus hamper associative memory. However, this result differs from the behavioral effects found in a previous study using the same paradigm (Staresina and Davachi 2009), which obtained no significant effect of trial type on any measure of memory performance. A likely explanation for the different results is the fact that the previous study implemented the memory test immediately after learning, while the present study had an interval of 24 h between encoding and testing. This interval most likely increased the difficulty of memory recall and thus increased the sensitivity of performance measures to effects of spatial-temporal discontinuity during memory formation. In addition, as the sample of the present study (n = 78) was significantly larger than the sample of the previous study (n = 18), we had a considerably higher power to detect an effect of spatial and temporal continuities on the mnemonic binding. In line with this view, the study by Staresina and Davachi (2009) showed a nonsignificant trend for impaired color memory for spatiotemporally discontinuous trials, when focusing on high-confidence responses.

The earliest relevant ERP component, the P300, reflecting the allocation of attentional resources (Polich and Kok 1995; Polich 2007; Olofsson et al. 2008) was markedly sensitive to spatial, but not additional temporal discontinuity. Interestingly, P300 amplitude was decreased for spatially discontinuous trials, although these trials placed increased demands on active memory integration. The P300 response to the stimulus in the present study might thus reflect attentional processes less relevant for mnemonic integration, which are even inhibited when integration demands increase. Comparable P300 amplitude decreases with increasing effort have been observed before in dual-tasking conditions (Kramer et al. 1985). Later, during the LPC, reflecting item-level processing (Mangels et al. 2001; Kim et al. 2009; Weymar et al. 2012), an additional distinction between spatial and spatial-temporal trials emerged: LPC amplitude was lowest for spatial trials compared to both combined and spatial-temporal trials. This suggests that the integration of spatial and spatial-temporal discontinuities is not mirrored by linear changes in neural activity, but rather requires qualitatively different operations. These operations might be relevant for later associative memory performance, as suggested by the trend-level subsequent memory effects for color memory performance.

For successful subsequent remembering, later neural processing, reflected in Slow Wave amplitudes and oscillatory power, appeared to be particularly relevant. The Slow Wave, observable in both anterior and posterior electrode sites, is typically sensitive to variations of encoding depth (Mangels et al. 2001; Schott et al. 2002; Guo et al. 2004). Hence, while the P300 and LPC presumably reflected early attentional processes not central to later memory, encoding depth as reflected by the Slow Waves was critical to later integrative memory. The decreased amplitude particularly for spatial trials already found during the P300 and LPC time window continued during the Slow Wave, suggesting that spatial discontinuity poses specific demands on all levels of processing from early attentional processes to later in-depth encoding. This special role of spatial discontinuities compared to both combined and spatial-temporal trials can be explained by the temporal trial structure: for spatial-temporal trials the color was presented before the object, creating temporal discontinuity. While this increased the effort necessary for integrating the color with the presented object (as demonstrated by the impaired color memory performance in spatial-temporal trials), it also provided a “head start” for color processing in spatial-temporal trials. For spatial trials, however, color and object were presented together and needed to be processed simultaneously, which may have resulted in the observed effects on neural correlates of memory formation.

In addition to the subsequent memory effects observed in the ERPs, oscillatory power, specifically in the theta band, reflected the integration of color and object memory. This specific role of theta oscillations dovetails with earlier findings implicating a central role for theta in memory binding (Lega et al. 2012; Backus et al. 2016; Clarke et al. 2018). We observed theta decreases for successful binding of color to object in memory, which tended to be more pronounced for spatially discontinuous trials compared to the combined presentation of object and color. However, additional temporal discontinuity did not modulate the involvement of theta in subsequent memory beyond the effect of spatial discontinuity alone. This result suggests that decreased theta power is critical in particular for spatial integration. Theta power decreases have been linked to successful memory formation (Burke et al. 2013; Long et al. 2014) and can cooccur with increased hemodynamic activity in the medial temporal lobe (Fellner et al. 2016). Thus, the theta power decrease observed in the present study might be an indicator of increased hippocampal recruitment, which is in line with earlier results reporting increased hippocampal activation for memory formation over spatial-temporal discontinuities (Staresina and Davachi 2009).

Beyond investigating the neural dynamics involved in spatial-temporal integration in episodic memory, we aimed to determine, for the first time, the impact of stress on this mnemonic integration. Previous studies suggested that stress shortly before encoding may enhance memory formation (Nater et al. 2007; Smeets et al. 2007; Schwabe et al. 2008a; Zoladz et al. 2011; Vogel and Schwabe 2018, but see also: Elzinga et al. 2005; Schwabe et al. 2010; Shields et al. 2017), although whether stress has an enhancing or impairing effect on memory formation appears to depend on several factors, such as the specific learning task used and the cognitive function tested or the cortisol levels during learning, with moderate (as opposed to very high or low doses) leading to enhanced memory (Abercrombie et al. 2003; Diamond et al. 2007; Salehi et al. 2010). Here, we obtained no significant effect of stress per se on the mnemonic binding. It should be noted, however, that we used in the present study emotionally neutral material and another recent study suggested that stress may affect associative memory for emotionally arousing material (Goldfarb et al. 2019), which is known to be more sensitive to stress effects (Schwabe et al. 2010). Although we did not obtain an effect of stress per se, apart from a nonsignificant trend for enhanced memory confidence in stressed participants, the activation of two major stress systems, the hypothalamic-pituitary-adrenal (HPA) axis and the autonomic nervous system (ANS), was associated with enhanced memory. Most interestingly, systolic blood pressure and salivary cortisol appeared to be differentially involved in object and color memory. Whereas blood pressure increases were selectively associated with enhanced object memory, cortisol increases were selectively linked to enhanced color memory. Distinct roles of cortisol and noradrenergic arousal, well known to synergistically impact memory processes (Roozendaal et al. 2004; Joëls et al. 2011), in memory have been reported before (Schwabe et al. 2008a; Schönfeld et al. 2014). The obtained association between cortisol and enhanced color memory, indicative for successful mnemonic binding, is further in line with previous studies reporting a positive link between cortisol and associative memory (van Ast et al. 2013, 2014), although this relation might depend on whether rapid, nongenomic or delayed, genomic actions of cortisol prevail (van Ast et al. 2014). Given attempts to translate the basic findings on stress and stress mediators on memory into clinical interventions (de Quervain et al. 2017), further pharmacological studies are needed to systematically dissect the roles of glucocorticoids and noradrenaline in memory formation.

These beneficial effects of stress mediators on memory formation were mirrored in the electrophysiological data: In line with recent findings (Kamp et al. 2018), the observed subsequent memory effects in the posterior Slow Waves were modulated by stress. Interestingly, this stress-induced modulation of neural subsequent memory effects appeared to be specific to color memory, suggesting that stress specifically promotes the neural processes critical for successful binding of information, but not for general recognition (see Supplemental Material). In line with this interpretation, stress also decreased oscillatory power in the theta and alpha band specifically when information had to be integrated across time and space (compared to spatial discontinuity only). Because power decreases in the alpha-theta range were also related to subsequent memory, it is tempting to speculate that the observed power decrease after stress might reflect the recruitment of additional resources for spatial-temporal memory integration.

In sum, our data showed that temporal and spatial gaps between event elements hamper episodic memory formation and that effort is required to bridge these gaps. Early neural processing is recruited by these gaps, in particular by spatial discontinuities, presumably providing required attentional resources. Later processes, including theta oscillations critically involved in mnemonic binding, were predictive of whether mnemonic integration and memory formation were successful or not. Major stress mediators, elevated briefly before encoding, appeared to modulate these processes, which might contribute to the superior memory for stressful episodes (Sandi et al. 1997; Vogel and Schwabe 2016). These findings provide novel insights into a key process of successful episodic memory formation, the mnemonic binding of episode elements across spatial and temporal gaps.

Materials and Methods

Participants

Eighty-two healthy volunteers participated in this experiment. Exclusion criteria comprised a history of any mental or neurological disorder, current medication intake, drug- or tobacco-use as well as, in women, pregnancy or intake of hormonal contraceptives, which can affect the cortisol response to stress (Kirschbaum et al. 1995; Lovallo et al. 2019). Menstrual cycle phase in women did not differ between stress and control group (stress: 11 in follicular phase, 8 in luteal phase; control: 8 in follicular phase, 11 in luteal phase; χ2(1, n = 38) = 0.95; P = 0.330). Four participants had to be excluded from the analysis because of missing EEG data (N = 2), insufficient memory performance on day 2 (N = 1, outlier criterion: mean ± 1.5 × interquartile range) or because they failed to appear on day 2 (N = 1), thus leaving a final sample size of 78 subjects (38 women, age: mean = 24.94, SD = 3.78). All participants provided written informed consent and received a monetary compensation for participation (35€). The study was approved by the local ethics committee of the University of Hamburg and conducted in accordance with the Declaration of Helsinki.

Task and procedure

Testing took place on two consecutive days between 8:30 and 12:00.

Day 1

After their arrival in the laboratory, participants were prepared for the EEG measurement. Next, participants gave subjective mood ratings (German version of the Positive and Negative Affect Schedule, PANAS, Krohne et al. 1996), a saliva sample using Salivette collection devices (Sarstedt), and baseline measurements of blood pressure were taken using a Critikon Dinamap system.

Day 1: Stress and control manipulation

Participants then underwent the Socially Evaluated Cold Pressor Test (SECPT, Schwabe et al. 2008b), a standardized stress protocol that is known to elicit robust stress responses (Schwabe and Schächinger 2018), or a nonstressful control manipulation. During the SECPT, participants immersed their left hand for 3 min into ice water (0°C–2°C) while being videotaped and evaluated by a cold, unresponsive experimenter. In the control condition, participants immersed their hand into warm water (35°C–37°C) and were neither videotaped nor evaluated. In order to assess the effectiveness of the stress manipulation, we took blood pressure measurements, subjective stress ratings, and saliva samples at several time points across the experiment. Blood pressure was measured during the SECPT/control manipulation (as well as at baseline and 25, 50, and 70 min after treatment onset). Immediately after the SECPT/control manipulation, participants rated how challenging, stressful, painful, and unpleasant they had experienced the situation on a scale from 0 (“not at all”) to 100 (“very much”). Twenty-five minutes after manipulation onset, we measured subjective mood again and collected another saliva sample. From saliva, we analyzed, at the end of the study, the free fraction of cortisol using a luminescence assay (IBL).

Day 1: Memory encoding task probing temporal-spatial binding

Twenty-five minutes after the onset of the SECPT or control manipulation, when the peak in stress-induced cortisol was expected, participants completed, while EEG was recorded, a memory encoding task that has been introduced before (Staresina and Davachi 2009). The stimulus material for this encoding task consisted of 450 grayscale photos of everyday objects. For each participant, 300 of these pictures were randomly determined as learning material, the remaining 150 served as lures for the test phase on day 2. In each encoding trial, participants saw one picture for 2 sec at the center of a computer screen. Participants were instructed to imagine the object in the presented color and subsequently rate the plausibility of that color for the specific object (high, middle, or low plausibility). Alternatively, participants could also indicate that they either did not recognize the depicted object or were not able to imagine the object in the presented color. In this latter case, the respective trials were excluded from subsequent analysis (on average 6.91 trials). The encoding task lasted about 45 min in total. In order to test the neural underpinnings of spatial-temporal integration and its potential modulation by stress, three different trial types were implemented (Fig. 6): (i) in combined trials the object was digitally colored, that is, it was presented in a specific color and therefore required only minimal integrative effort; (ii) in spatial trials the object was presented in grayscale at the center of the screen and the object color was presented in a frame around the object, thus requiring participants to integrate object and object color spatially; and (iii) in spatial-temporal trials the object was again presented in grayscale at the center of the screen but this time the frame indicating the color of the object was presented, for 500 msec, 1000 msec before the object itself was presented in grayscale, thus requiring participants to integrate object and object color across time and space. To control for effects of mere color presentation (especially on neural activity), additional visual control trials were added, in which only a colored frame was presented, without a subsequent presentation of an object. A total of 100 trials was presented for each trial type (combined, spatial, spatial-temporal, and visual control), with trial type order being fully randomized. The color of the presented object was determined randomly out of four possible colors (red, blue, green, or brown). There were three breaks during the task, with a duration determined by the participant (average duration: 30 sec). During the second break (∼50 min after stressor onset) as well as after the task (∼70 min after stressor onset) blood pressure and subjective mood were assessed again and further saliva samples were collected.

Figure 6.

Time-frequency power analysis. (A) Cluster-based permutation tests identified two time-frequency clusters distinguishing trials with subsequent color memory from trials with mere object recognition. (B) Theta power revealing decreases due to subsequent color memory for spatial trials especially in the stress group. (C) Power in the alpha-theta band was decreased for both recognition and color memory. Error bars represent SEM. (*) P < 0.05 (**) P < 0.01.

Day 2: Memory testing

About 24 h after memory encoding, participants returned to the laboratory, gave another mood rating and saliva sample and their blood pressure was measured again. Afterwards, they completed a memory test, in which they saw one after another 450 pictures of objects, including the 300 pictures they had seen on day 1 as well as 150 lures. During this memory test, all objects were presented in grayscale and participants were first required to indicate whether the object was encountered during encoding on day 1 or not (“old” and “new,” respectively; Fig. 1). If participants selected “old,” they were further asked to indicate the color that was associated with the object during encoding on day 1, followed by a confidence rating for the color judgement (“certain,” “uncertain,” or “guessed”). If they selected “new,” the next object was presented. For both judgements, participants had no specific time limits, yet they were asked to respond quickly. Old and new pictures were presented in a randomized order without intertrial interval.

In order to control for potential group differences in chronic stress, depressive mood, and anxiety, participants completed the Trier inventory for the assessment of chronic stress (TICS, Schulz and Schlotz 1999), the Beck depression inventory (BDI, Beck et al. 1961), and the state-trait anxiety inventory (STAI, Spielberger and Sydeman 1994) on the first experimental day.

Behavioral data analysis

Hits (vs. misses) and correct (vs. incorrect) color judgements were analyzed on single-trial level by means of binomial generalized linear mixed models (GLMMs) using the lme4 R-package (Bates et al. 2014). In order to isolate the binding of color information from mere object memory, we included only correctly as “old” classified trials in the analysis of color memory, thereby contrasting successful binding against mere recognition. In a separate model, we also analyzed object recognition memory by contrasting forgotten trials and trials with recognition memory. For each trial, object recognition memory and color memory were modeled separately using group (stress vs. control), trial type (combined vs. spatial vs. spatial-temporal) and their interaction as fixed effects. Additionally, we included the plausibility rating and for the color memory model also the confidence rating as fixed effects of no interest. The models also included random intercepts of participants and stimuli as well as random slopes of the predictors group and item type. The factorial predictors were contrast coded using sliding difference contrasts, comparing the neighboring factor levels (e.g., combined vs. spatial and spatial vs. spatial-temporal for the trial type effect). In additional models, we also included the physiological stress response as a predictor instead of group. For this, we used the systolic blood pressure measurement during the manipulation and the salivary cortisol level 25 min after the manipulation as predictors, conducting a separate model for each predictor. We chose these specific time points to capture the peaks of sympathetic and glucocorticoid activity, based on the known temporal profile of action of these physiological stress response systems (Joëls and Baram 2009). In order to verify the effectiveness of the stress manipulation, we analyzed the physiological and subjective stress response by means of multilevel linear models using the nlme R-package (Pinheiro et al. 2012). Control variables were analyzed by means of t-tests. All statistical analyses were carried out using R-Studio (Version 3.5.2; R Core Team 2018) or SPSS (IBM). All reported P-values are two-tailed.

EEG recording and analysis

During memory encoding on day 1, EEG data was recorded using a 64-channel BioSemi Active Two system with a sampling rate of 2048 Hz. The electrodes were arranged according to the international 10/20 system. Eye movements and blinks were recorded using additional electrodes placed on the outer canthi of both eyes as well as above and below the right eye.

EEG data were preprocessed using BrainVision Analyzer software (BrainProducts) and custom-written MATLAB (The MathWorks) scripts (Processing scripts adapted from: Cohen 2014; Frömer et al. 2018). First, the data were band-pass filtered between 0.1 and 80 Hz and resampled to 512 Hz. Blinks were removed using independent component analysis (ICA). Afterwards, bad channels were replaced using topographic interpolation and the data were rereferenced to the average reference of all scalp electrodes.

ERP analysis

For the analysis of event-related-potentials (ERPs), the data was segmented from −400 to 1500 msec relative to picture onset. Segments with voltage steps >50 mV, overall voltage differences >200 mV, or a signal <0.1 mV were rejected. On average, 20 Trials were rejected per participant (range: 1–97). The groups did not differ regarding the number of rejected trials (t(75.08) = −1.28; P = 0.204). The remaining trials were baseline-corrected relative to the 400 msec preceding picture onset. Next, the ERP components of interest were extracted from every trial. For this, we defined two clusters of electrodes: an anterior (F1, Fz, F2, FC1, FCz, FC2) and a posterior (P1, Pz, P2, PO3, POz, PO4) cluster. We focused on the P300, the LPC, and Slow Waves, which have been linked to successful memory formation before (Wagner et al. 1999; Otten and Donchin 2000). The ERP components of interest were defined in accordance with prior studies (Spencer and Polich 1999; Kim et al. 2009; Chen et al. 2014). In particular, the P300 was defined as the average amplitude over 300–400 msec of the posterior cluster relative to picture onset. The LPC was defined as the average amplitude over 400–700 msec of the posterior cluster and the Slow Wave was extracted both from the anterior and the posterior cluster over a time window of 1000–1500 msec.

In line with our behavioral data analysis, the ERP amplitudes for each trial were modeled using LMMs. For each ERP component, we fitted models containing the fixed effects group (stress vs. control), trial type (combined vs. spatial vs. spatial-temporal), subsequent associative memory (only object remembered vs. object and color remembered) as well as all possible interactions. Additionally, the plausibility rating as well as confidence rating were included as nuisance variables. The model also included random intercepts for participants and stimuli. For the factors item type and group, the same contrasts were applied as for the behavioral analysis. Each ERP component was additionally modeled with the physiological stress response (salivary cortisol 25 min after the manipulation and systolic blood pressure during the manipulation) instead of group, building a separate model for each physiological measure. These models test the additional predictive value of the two major stress response systems, sympathetic and glucocorticoid activity, beyond the effects of group. For the sake of completeness, we set up an additional model including recognition memory (object subsequently forgotten vs. only object remembered) as a predictor. The results of this analysis are presented in Supplemental Material S1.

Time-frequency power analysis

For the analysis of spectral power, the same data segment was extracted as for the ERP analysis. Additionally, a time window from −600 to −100 msec relative to the first frame onset served as a baseline for decibel normalization. Both segments were subjected to the same artifact rejection procedure as used for the ERPs. Because of the decibel transformation, data was averaged over trials for each condition prior to baseline normalization, which precluded an analysis on trial level as done for the behavioral and ERP data. Thus, power values were analyzed with LMMs including only subjects as random intercepts. The fixed effects were the same as included in the LMMs of the ERP analysis. In order to minimize the influence of conditions with few trials on the average, participants with either >30% rejected trials in either item type condition or <6 trials in either subsequent memory condition were excluded from the analysis of spectral power. Consequently, 67 participants were included in the power analysis (stress group: 19 women, 14 men; control group: 16 women, 18 men; mean number of trials: 377.18; SD: 23.54).

Time-frequency-decomposition was performed for 30 logarithmically spaced frequencies between 2 and 80 Hz using 5-cycle morlet wavelets. The baseline-normalized power values were then averaged over the same predefined electrode clusters used for the ERP analyses. In order to identify time-frequency windows of interest, we contrasted the trials with correct color judgements against trials with only correct object memory using cluster-based permutation tests. For this, t-values of this contrast were computed for each frequency × time point combination. These values were tested against a distribution of t-values obtained from randomly changing labels of the contrasted conditions over 2000 permutations (significance threshold: P < 0.05). The significant t-values were then grouped into coherent clusters and corrected against the cluster distribution derived from the permutation (cluster statistic: a sum of t-values, threshold: P = 0.05). Power values were extracted from the clusters and averaged for each condition. Then, the condition averages were baseline normalized and further analyzed as described above.

All models (behavioral and EEG data) were also conducted including sex as a predictor of no interest, which did not change the effects of the other predictors.