Computational design of enhanced learning protocols.

Learning and memory are influenced by the temporal pattern of training stimuli. However, the mechanisms that determine the effectiveness of a particular training protocol are not well understood. We tested the hypothesis that the efficacy of a protocol is determined in part by interactions among biochemical cascades that underlie learning and memory. Previous findings suggest that the protein kinase A (PKA) and extracellular signal-regulated kinase (ERK) cascades are necessary to induce long-term synaptic facilitation (LTF) in Aplysia, a neuronal correlate of memory. We developed a computational model of the PKA and ERK cascades and used it to identify a training protocol that maximized PKA and ERK interactions. In vitro studies confirmed that the protocol enhanced LTF. Moreover, the protocol enhanced the levels of phosphorylation of the transcription factor CREB1. Behavioral training confirmed that long-term memory also was enhanced by the protocol. These results illustrate the feasibility of using computational models to design training protocols that improve memory.

In the field of experimental psychology, virtually all learning paradigms used in animal and human studies were developed on an ad hoc, trial-and-error basis. For example, multiple training trials spaced over time are generally more effective in producing long-term memory (LTM) than are multiple trials massed together[1]. However, it is not known why one procedure is better than another, nor is the optimal spacing of trials known a priori. In principle, one way to enhance learning and memory is to design training protocols that are synchronized or in phase with the dynamics of biochemical cascades underlying the induction of LTM[2,3]. However, this task is challenging because of nonlinear interactions and multiple time scales within these cascades. One way of overcoming this challenge is to develop computational models of these nonlinear interactions and use the models to predict training protocols that maximize LTM.

Long-term sensitization (LTS) of withdrawal reflexes in the mollusk Aplysia is an example of LTM that is particularly well characterized[4] and therefore serves as an excellent system for testing the concept that computational approaches can help design training protocols that enhance learning and memory. Distinct phases of memory for sensitization are induced by different numbers of training trials[5-9]. At the cellular level, LTS is due, in part, to serotonin (5-HT)-induced LTF of synaptic connections between sensory neurons and motor neurons[5,10,11]. Two biochemical cascades required for LTF are those that mediate activation of protein kinase A (PKA)[12-15] and the MAP kinase isoform termed extracellular signal-regulated kinase (ERK)[16-18]. These kinases are both needed to phosphorylate transcription factors critical for LTF (e.g., the transcriptional activator cAMP responsive element binding protein 1, CREB1)[19,20]. These events, in turn, induce genes whose products play essential roles in LTF[21,22]. The requirement for both PKA and ERK suggests that the strength of LTF is likely to depend on synergism between these kinases. Following a single 5-HT stimulus, PKA is activated rapidly but transiently, with activity returning to near basal levels within 15 min[13]. In contrast, ERK is activated more slowly with maximal activation occurring about 45 min after a single trial[7]. These results indicate that a single stimulus will produce little overlap between PKA and ERK. However, they also suggest that multiple properly timed stimuli could increase the overlap and hence the synergism of the two cascades. The present study tested the hypothesis that a stimulation protocol maximizing the overlap would enhance learning and memory. To test this hypothesis, a computational model of the PKA and ERK cascades was developed, and simulations of the model were used to predict a protocol that could enhance learning and memory. Subsequent in vitro and in vivo empirical studies confirmed that LTF and LTS were enhanced by the protocol.

RESULTS

Computational model of PKA/ERK cascades

In the present study, a simplified mathematical model of the dynamic activation of PKA and ERK was developed (Fig. 1a). In the model, interactions between PKA and ERK cascades were represented by a term denoted ‘inducer’. Because activation of ERK and PKA are both required for LTF[15,16], inducer was proportional to the product of PKA and ERK activities. The peak levels of inducer, which corresponded to the peak synergistic interaction between PKA and ERK, were hypothesized to predict the efficacy of stimulus protocols.

Figure 1

Computational model of PKA and ERK pathways

(a) Schematic of the model. 5-HT activates PKA via cAMP and activates ERK via Raf-MEK. PKA and ERK interact, at least in part, via phosphorylation of transcription factors, to induce LTF. The variable inducer represents the PKA/ERK interaction. (b–c) Simulated time courses of activated PKA (PKAC, red traces), activated ERK (ERKpp, yellow traces), and inducer (violet traces) in response to five, 5-min pulses of 5-HT (green traces). The Standard protocol (b) represents the protocol generally used in studies of LTF in vitro. The Enhanced protocol (c) produced the largest peak in the concentration of inducer. The patterns of 5-HT pulses are illustrated in each panel. The Standard protocol had uniform ISIs of 20 min, whereas the Enhanced protocol had non-uniform ISIs of 10, 10, 5 and 30 min. Units of concentration of all variables are µM.

As a point of reference, the simulated peak level of inducer produced by five, 5-min pulses of 5-HT with uniform 20-min interstimulus intervals (ISIs) was selected (Fig. 1b). This 'Standard' protocol has been widely used in experimental studies since its introduction in 1986[23]. To identify a protocol that produced the highest peak level of inducer, 9,999 alternative protocols were simulated. Each protocol included five, 5-min 5-HT stimuli, but the ISIs were chosen as multiples of 5 min, in the range from 5–50 min (Supplementary Movie 1). Thus, each of the four ISIs had 10 possible values (i.e., 104 total possible permutations, one of which represented the Standard protocol). Simulation of these protocols revealed considerable variability in the peak level of inducer (Supplementary Fig. 1).

The protocol that maximized the peak level of inducer (the 'Enhanced' protocol) consisted of non-uniformly spaced 5-HT applications with ISIs of 10, 10, 5, and 30 min. The peak level was ~50% higher than the in Standard protocol (Fig. 1c). To the extent that the model captured the salient features of the dynamics of PKA and ERK and the extent to which the synergism between PKA and ERK activity is essential for LTF, the Enhanced protocol was predicted to induce more LTF than the Standard protocol.

Enhancement of long-term synaptic facilitation (LTF)

Electrophysiological recordings from sensorimotor co-cultures were used to test our predictions. Three groups of preparations were examined, the Standard and Enhanced protocols (see Fig. 1b,c) and a vehicle (Control) group, which received five 5-min applications of solution without 5-HT (20-min ISIs). In each group, excitatory postsynaptic potentials (EPSPs) were measured prior to 5-HT treatment (pretest) and 1, 2, and 5 d after 5-HT treatment (post-tests). Long-term facilitation was defined as a significant increase in the EPSP amplitude in either the Standard or Enhanced groups as compared to EPSPs measured in the Control group. Sample recordings are illustrated in Figure 2a and summary data are illustrated in Figure 2b.

Figure 2

Enhanced protocol increased the magnitude and duration of LTF

Two 5-HT protocols (Enhanced, red traces; Standard, orange traces) were tested (protocol parameters as in Fig. 1). In addition, a vehicle (Control) protocol (black traces) was tested in which sensorimotor co-cultures were not exposed to 5-HT. (a) A single EPSP was elicited immediately prior to 5-HT treatment (Pretest). Individual EPSPs were also elicited 1 d, 2 d, and 5 d after treatment (post-tests). The dashed lines indicate the amplitude of the Pretest EPSP. (b) In each sensorimotor pair, the EPSPs that were measured at 1, 2, and 5 days were normalized to the Pretest EPSP. A value of 1 (dashed line) represented no change in EPSP amplitude. The Enhanced protocol produced greater and longer-lasting LTF. In this figure and subsequent figures, summary data are represented as means ± s.e.m.

Statistical analyses (two-way ANOVA) indicated significant overall differences in the amplitude of EPSPs among the three treatment groups (F = 22.06, p < 0.001) and among the three time points (F = 7.28, p < 0.002). Post hoc, pair-wise comparisons (Student-Newman-Keuls tests, SNK) indicated that 1 day after 5-HT treatment both the Standard protocol (q = 4.49, n = 10, p < 0.003) and Enhanced protocol (q = 7.57, n = 11, p < 0.001) induced LTF as compared to vehicle controls (n = 8). The LTF that was induced by the Standard protocol, however, did not persist beyond the first day (day 2: q = 2.54, n = 10; day 5: q = 0.21, n = 9) as compared to controls on day 2 (n = 8) and on day 5 (n = 8). In contrast, the LTF that was induced by the Enhanced protocol persisted for up to five days (day 2: q = 5.21, n = 9, p <0.002; day 5: q = 3.41, n = 9, p < 0.02) as compared to vehicle controls. In addition, post hoc analyses indicated that the Enhanced protocol induced significantly greater LTF at all time points as compared to the Standard protocol (day 1: q = 3.18, p <0.03; day 2: q = 2.89, p <0.05; day 5: q = 3.73, p < 0.03).

Increased levels of CREB1 phosphorylation

The induction and consolidation of LTF in vitro depends on the activity of several transcription factors[24-27]. For example, increased levels of CREB1 phosphorylation are associated with the induction and consolidation of LTF at sensorimotor synapses[2,25,28]. Because the Enhanced protocol induced greater and longer-lasting LTF, the Enhanced protocol was hypothesized to be associated with a greater level of CREB1 phosphorylation. This hypothesis was tested in sensory neuron cultures (Fig. 3).

Figure 3

Enhanced protocol increased the levels of phosphorylated CREB1

(a) Immunofluorescence staining of phosphorylated CREB1 in sensory neuron cultures. One group of cultures received the Standard protocol (five, 5-min pulses of 5-HT with uniform ISIs of 20 min), a second group the Enhanced protocol (five, 5-min pulses of 5-HT with ISIs of 10, 10, 5, and 30 min) and a third group of cultures served as vehicle controls (five, 5-min pulses of vehicle with uniform ISIs of 20 min). Nuclear staining of phosphorylated CREB1 was measured in the cell bodies of sensory neurons immediately and 18 h after 5-HT treatment. Scale bar, 20 µm. (b) Summary data from the Standard (orange bars, S) and Enhanced (red bars, E) protocols. The Enhanced protocol induced significantly greater levels of phosphorylated CREB1 at both time points.

Three groups of sensory neurons cultures were examined, one group was treated with the Standard protocol, the second group was treated with the Enhanced protocol identical to that used in the electrophysiological experiments of Figure 2, and the third group was a vehicle control. In each group, the level of CREB1 phosphorylation was measured either immediately (0 h) or 18 h after treatment, and the values in the Standard and Enhanced groups were normalized to vehicle controls. Statistical analyses (two-way ANOVA) indicated overall significant differences between the two 5-HT protocols (F = 30.12, p < 0.001). Pair-wise comparisons (SNK tests) indicated that compared to the Standard protocol, the Enhanced protocol induced significantly higher levels of CREB1 phosphorylation immediately (q = 6.94, n = 4, p < 0.001), and 18 h after treatment (q = 3.69, n = 7, p < 0.02).

Enhancement of long-term sensitization (LTS)

Both long-term synaptic plasticity and modifications in CREB1 are thought to be key processes underlying long-term memory. Having demonstrated that the Enhanced protocol improved LTF and phosphorylation of CREB1, the Enhanced protocol was hypothesized to produce stronger learning than the Standard protocol in behaving animals. Long-term sensitization training was used to test this hypothesis. For each animal, LTS was tested 1 d and 5 d after training (Fig. 4).

Figure 4

Enhanced protocol induced LTS that persisted for at least 5 d

For these behavioral experiments, one group of animals was trained using the Standard protocol (orange bars, S) and a second group was trained using the Enhanced protocol (red bars, E). Training stimuli were applied to only one side of each animal (i.e., the ipsilateral side). The contralateral side of each animal served as a control (black bars, C). Both the Standard and Enhanced protocols induced LTS, as indicated by the significant increase in responses (i.e., the duration of the tail-elicited siphon withdrawal) 1 d after training as compared to control. At 1 d, the magnitude of LTS was not significantly different between the Standard and Enhanced protocols (q = 0.76). However, only the Enhanced protocol induced LTS that persisted for at least 5 d. For the Enhanced protocol, the levels of LTS at 1 d and 5 d were not significantly different (q = 2.63).

Statistical analyses (two-way ANOVA) indicated significant differences among the three training groups (F = 11.77, p < 0.001). Pair-wise comparisons (SNK tests) indicated that both the Standard protocol (q = 3.12, n = 12, p < 0.04) and Enhanced protocol (q = 4.13, n = 14, p < 0.02) induced LTS that lasted 1 d. The LTS that was induced by the Standard protocol, however, did not persist for five days (q2 = 0.38, n = 14). In contrast, the LTS that was induced by the Enhanced protocol persisted for at least five days (q = 5.48, n = 9, p < 0.001).

DISCUSSION

The results indicated that computational approaches could be used to design a stimulus protocol that increases long-term synaptic plasticity, transcriptional activation, and long-term sensitization. Interestingly, the non-uniform distribution of ISIs in the Enhanced protocol was in marked contrast to the fixed intervals used in previous protocols for behavioral training in Aplysia[6,7,10]. The use of non-uniform ISIs is not unprecedented, however. For example, a form of training with expanding ISIs appears to lead to enhanced performance in verbal recall tasks[1]. The mechanistic underpinnings of this form of training are not well understood, and therefore provide little guidance for designing protocols that maximize retention[1]. The present results suggested that modeling the dynamics of biochemical cascades could play an essential role in understanding why specific training protocols are more effective than others, and thereby, help guide the development of better learning paradigms.

A goal of the present study was to develop training protocols that enhanced learning and memory. However, the model described in Fig. 1a was also compatible with previous empirical results in which a massed training protocol (25 min continuous 5-HT) produced significantly less LTF than a spaced protocol (five, 5-min pulses of 5-HT)[29]. Simulations of this massed protocol produced a peak level of inducer of only 0.00016 µM, which was at the low end of the distribution illustrated in Supplementary Fig. 1. Thus, the model was able to predict submaximal performance, which helped to increase confidence in its predictive power.

As indicated in Supplementary Fig. 1, many training protocols were predicted to outperform the Standard protocol. These results indicate that the improvement found with the Enhanced protocol did not represent a narrow range of model parameter values, but occurred for various combinations of values. Such behavior is expected for robust models of physiological processes, in which optima should not be sharply peaked at very specific parameter values.

In humans, a number of cognitive models have been proposed to explain the superiority of spaced training or practice[1,30], but no consensus has emerged as to their usefulness or generality[31]. These models include variations in processing, encoding, and consolidation. It is likely that these models have had limited predictive ability, at least in part, because they are not based on known neurobiological processes. In contrast, the present model was based on well characterized biochemical cascades in individual neurons that participate in the induction and expression of LTM.

Although the present model was limited to a description of the dynamics of biochemical cascades associated with the initial induction of LTM and to relatively brief training sessions (e.g., 1.5 h), the model successfully predicted a protocol that enhanced synaptic strength and memory for at least five days after stimulation. These results confirmed that important aspects of the induction of LTM occur during the initial training phase. The model is undoubtedly incomplete, however. For example, the slow dynamics of additional biochemical cascades contribute to LTM (e.g., changes in the synthesis of transcription factors such as CREB1 and CREB2[2,32]). However, these slower processes were not considered in the present model. In addition, the model did not include descriptions of postsynaptic protein kinase C[33], upregulation of local protein synthesis in the vicinity of the synapse[34] or synthesis of the peptide sensorin in response to retrograde signals[35], which are involved in LTF. Finally, the model did not account for the differences in the time course and magnitude of LTF (Fig. 2) from that of LTS (Fig. 4). Several factors may contribute to this discrepancy, including a nonlinear transformation of LTF into motor neuron activity and plasticity at other loci in the neural circuit that contribute to the behavioral enhancement[36,37]. An expanded model that includes the dynamics of additional biochemical cascades and neural circuit elements would presumably have greater predictive capability.

The present results indicated the feasibility of using computational methods to assist in the design of training procedures that enhance learning. It will be important to determine the extent to which the computational approaches can be applied to other memory systems, and ultimately, applied to training procedures used to improve human cognition.

METHODS

A detailed description of the methods is available in the online version of the paper at http://www.nature.com/natureneuroscience/.

Note: Supplementary information is available on the Nature Neuroscience website.