Molecular mechanisms underlying formation of long-term reward memories and extinction memories in the honeybee (Apis mellifera).

The honeybee (Apis mellifera) has long served as an invertebrate model organism for reward learning and memory research. Its capacity for learning and memory formation is rooted in the ecological need to efficiently collect nectar and pollen during summer to ensure survival of the hive during winter. Foraging bees learn to associate a flower's characteristic features with a reward in a way that resembles olfactory appetitive classical conditioning, a learning paradigm that is used to study mechanisms underlying learning and memory formation in the honeybee. Due to a plethora of studies on appetitive classical conditioning and phenomena related to it, the honeybee is one of the best characterized invertebrate model organisms from a learning psychological point of view. Moreover, classical conditioning and associated behavioral phenomena are surprisingly similar in honeybees and vertebrates, suggesting a convergence of underlying neuronal processes, including the molecular mechanisms that contribute to them. Here I review current thinking on the molecular mechanisms underlying long-term memory (LTM) formation in honeybees following classical conditioning and extinction, demonstrating that an in-depth analysis of the molecular mechanisms of classical conditioning in honeybees might add to our understanding of associative learning in honeybees and vertebrates.

Reward memories in honeybees

The honeybee (Apis mellifera) has long served as an invertebrate model organism for reward learning and memory research (Menzel 2012). Its value as a model organism in this area is rooted in its impressive capacity for learning and memory formation. Honeybees are social insects that exhibit a prounounced divison of labor. During the summer season a honeybee colony consists of a single, reproductive queen, a few hundred drones, and tens of thousands of sterile worker bees. Depending on their age, worker bees either work inside the hive and fulfill hive duties such as brood rearing, or forage outside the hive collecting nectar or pollen to nourish the queen, workers, and brood. During the winter, long-living worker bees stay inside the hive, clustering around the queen to keep her temperature constant and ensure her survival. Survival during the winter season is only possible when sufficient nectar and pollen has been collected during the summer and is stored in the hive (Seeley and Visscher 1985; Winston 1991; Seeley 1995.

Honeybees visit one particular flower species during a foraging trip and collect nectar and pollen from the same food source as long as it is productive (Free 1963; Greggers and Menzel 1993; Greggers and Mauelshagen 1997; Gruter et al. 2011). Accordingly, honeybees learn the properties of a food source including its location and sensory characteristics such as odor, color, and shape (von Frisch 1967; for recent reviews, see Sandoz 2011; Avargues-Weber and Giurfa 2013; Hempel de Ibarra et al. 2014). They also learn the quantity or quality of the food source and the time of day at which it displays its highest total sugar concentration (Menzel 1968; Menzel and Erber 1972; Buchanan and Bitterman 1988, 1989; Lee and Bitterman 1990; Couvillon et al. 1991, 1994; Loo and Bitterman 1992; Couvillon and Bitterman 1993; Hoban et al. 1996; Moore 2001; Gil et al. 2007; Gil and De Marco 2009; Moore and Doherty 2009; Gil 2010; Edge et al. 2012).

Classical olfactory conditioning resembles associative learning during foraging

Learning an association between the characteristic features of a food source and the reward of food resembles Pavlovian classical conditioning, in which an animal learns that a conditioned stimulus (CS) predicts the occurrence of a meaningful unconditioned stimulus (US). Accordingly, the mechanisms that underlie honeybee reward learning are studied in individual honeybees that have been classically conditioned according to an appetitive Pavlovian conditioning paradigm based on the olfactory conditioning of the proboscis extension response (PER) (Takeda 1961; Bitterman et al. 1983; Felsenberg et al. 2011; Giurfa and Sandoz 2012; Matsumoto et al. 2012). In this paradigm, honeybees learn the association between an initially neutral olfactory stimulus, the conditioned stimulus (CS), and a sucrose stimulus, the unconditioned stimulus (US). The presentation of the US alone elicits an unconditioned response (UR), i.e., the extension of the proboscis when the antennae or the proboscis of a honeybee come into contact with sucrose solution. This response is termed the proboscis extension response (PER). Once the association between the olfactory CS and the rewarding US has been learned, the CS alone elicits the PER. When a PER is elicited by the CS alone following classical conditioning, this response resembles the conditioned response (CR). Accordingly, during appetitive classical conditioning the CS is learned as the predictor of the rewarding US, acquiring excitatory properties that elicit the CR.

In experiments on PER conditioning in honeybees, a group of insects is conditioned each day and the conditioned response is tested during memory retrieval. The PER to the presentation of the CS during conditioning and memory retrieval is recorded for each individual and the percentage of bees that respond with the PER is analyzed (Felsenberg et al. 2011; Matsumoto et al. 2012). A significant increase in the percentage of animals responding with a PER during conditioning is interpreted as learning. A high percentage of animals responding to the CS during memory retrieval is interpreted as an indication of the formation of a strong memory, whereas a low percentage of animals is interpreted as a weak memory. Accordingly, the group behavior of animals is seen as an indication of the probability that learning and memory formation will occur in an individual animal. However, Pamir et al. (2011) report a step-like, stable change of behavior of individual honeybees during classical conditioning that contrasts with the gradually increasing learning curve observed in group behavior: Once a bee responds to the CS during conditioning, the probability that this insect will again respond to the CS during conditioning and memory retrieval is higher than the probability that it will not respond. This step-like change in the behavioral response is most likely attributed to the dichotomous nature of the behavior recorded (proboscis extension: yes or no) and is not necessarily an indication that the underlying neuronal and molecular processes also proceed in a step-wise manner. This step-like behavior does, however, provide an opportunity to differentiate between the learning behavior of individual bees (“learners” vs. “nonlearners”) and to examine the neuronal and molecular processes that underlie it (Matsumoto et al. 2012).

Common principles underlying classical conditioning in honeybees and vertebrates

Appetitive classical conditioning in the honeybee has been characterized with an impressive depth, making the honeybee one of the best-characterized invertebrate models from the point of view of psychological learning theory. Several learning phenomena known from studies on classical conditioning in vertebrates have been demonstrated in honeybees, including trace conditioning (Szyszka et al. 2011), backward conditioning (Bitterman et al. 1983; Hellstern et al. 1998; Dacher and Smith 2008; Felsenberg et al. 2014a), second-order conditioning (Hussaini et al. 2007), extinction learning (Sandoz and Pham-Delegue 2004; Stollhoff et al. 2005; Eisenhardt and Menzel 2007; Stollhoff and Eisenhardt 2009; Plath et al. 2012), sensory preconditioning (Müller et al. 2000), and blocking (Guerrieri et al. 2005; for a contrasting review, see Gerber and Ullrich 1999), latent inhibition (Chandra et al. 2010), and occasion setting (Mota et al. 2011). These phenomena are surprisingly similar to vertebrates, in which they were initially described (Bouton 1993; Fanselow 1998; Schmajuk et al. 1998; Myers and Davis 2002; Gewirtz and Davis 2006; Denniston and Miller 2007; Raybuck and Lattal 2014). This suggests that the underlying neuronal mechanisms are also likely based on common principles. A resemblance between behavioral performance does not necessarily imply a similar principle at every level of the nervous system and one should not expect, given gross differences between brain anatomy in honeybees and vertebrates, that similarities in classical conditioning are due to common overall brain organization. However, common principles might be found at the level of neuronal circuits and through connectivity within defined circuits, i.e., similar “connectomes,” or similar types of neurons that “serve as a basis of a common vocabulary,” and may be based on closely related molecular mechanisms (Bargmann and Marder 2013).

Indeed, several lines of evidence suggest that common principles might be found at the molecular level. The honeybee genome project revealed a plethora of honeybee proteins homologous to vertebrate neuronal proteins (Honeybee Genome Sequencing Consortium 2006). In addition, converging work on the molecular mechanisms of synaptic plasticity in invertebrates and vertebrates suggests similar principles regarding the neuronal basis of learning and memory formation (Glanzman 2010). Furthermore, growing evidence in vertebrates suggest that the evolution of neuronal proteins has contributed to the evolution of complex learning behavior (Nithianantharajah et al. 2013; Ryan et al. 2013), suggesting that less complex forms of learning in invertebrates might be based on molecules and molecular mechanisms that are similar to those of vertebrates. Given the surprising similarities between honeybees and vertebrates with respect to classical conditioning and its related behavioral phenomena, I thus hypothesize that these molecular mechanisms might rest on a common principle that makes the honeybee a particularly interesting invertebrate model organism for learning and memory research. To illustrate this hypothesis, I will here review the molecular mechanisms that underlie the formation of long-term memories following classical conditioning and extinction in honeybees.

Two long-term reward memories are distinguishable in honeybees

The molecular mechanisms of honeybee appetitive Pavlovian conditioning and memory formation have been examined in depth over the last 20 yr. Depending on the time points of pharmacological interference and memory retrieval, molecular mechanisms that underlie the formation of short-term memory (STM), mid-term memory (MTM), early long-term memory (eLTM), and late long-term memory (lLTM) are distinguished (Menzel 1999, 2001; Müller 2002, 2013; Schwärzel and Müller 2006). Here, I will focus solely on the mechanisms of long-term memory formation since the mechanisms of coincidence detection, STM and MTM formation during and after classical conditioning have been reviewed and discussed elsewhere (e.g., Menzel 1999, 2001; Müller 2002, 2013; Eisenhardt 2006; Schwärzel and Müller 2006).

Long-term memories are particularly stable memories. Their formation depends on the synthesis of new proteins that contribute to structural changes and thus to the stability of specific memory traces. Accordingly, long-term memories undergo a stabilization process during which translation and transcription take place. This process is termed memory consolidation, and the subsequent long-term memories are therefore defined as consolidated memories (Dudai 2004).

Two different consolidated memories which form following classical conditioning with three CS–US pairings, i.e., conditioning trials, can be dissociated due to their differential sensitivity to translational and transcriptional inhibitors (for review, see Müller 2013). The formation of eLTM is sensitive to the translational inhibitor emetine (Friedrich et al. 2004; Stollhoff et al. 2005) but not to anisomycin (Wustenberg et al. 1998; Felsenberg et al. 2011), whereas lLTM depends on anisomycin-sensitive translation as well as actinomycin D-sensitive transcription and is visible 3 d after three-trial conditioning (Wustenberg et al. 1998; Friedrich et al. 2004; Felsenberg et al. 2011; Lefer et al. 2012; Matsumoto et al. 2014). It is noteworthy that these two different memories can be distinguished through the use of different inhibitors for translation (eLTM: emetine, lLTM: anisomycin). Why these two inhibitors of protein synthesis differentially target eLTM and lLTM is not yet understood. Their different modes of action could be one explanation for the sensitivity of different LTMs to these two protein synthesis inhibitors: Emetine binds the small ribosomal 40S subunit inhibiting aminoacyl tRNAs transfer but does not prevent the transpeptidation or translocation of peptidyl-tRNA from acceptor to donor sites (Grollman 1968; Gupta and Siminovitch 1977; Boersma et al. 1979; Rhoads and Roufa 1985; Wong et al. 2014). Anisomycin, on the other hand, binds to the large ribosomal 60S subunit, inhibiting its peptidyltransferase activity and thus the formation of peptide bonds (Hansen et al. 2003). Additionally, anisomycin not only inhibits translation, but seems to enhance the expression of immediate early genes, a process termed gene superinduction (for reviews, see Alberini 2008; Radulovic and Tronson 2008). Thus it is possible that anisomycin and emetine cause differences in LTM formation because of these additional effects of anisomycin.

Besides its sensitivity to anisomycin, a second unique feature of lLTM is its sensitivity to the transcriptional inhibitor actinomycin D (Grunbaum and Müller 1998; Wustenberg et al. 1998; Friedrich et al. 2004). Transcription has therefore been assumed to be the sole prerequisite for lLTM formation. Meanwhile, an involvement of epigenetic mechanisms in honeybee LTM formation has also been reported. Epigenetic processes such as histone acetylation, histone methylation, and DNA methylation regulate gene expression (Puckett and Lubin 2011). Merschbaecher et al. (2012) demonstrated that the acetylation of histones is crucial for eLTM formation. Acetylation by the histone acetyltransferase (HAT) results in chromatin opening, thereby permitting transcription, whereas histone deacetylation by histone deacetylases (HDACs) has a repressive effect on transcription (Cho and Cavalli 2014). In honeybees an enhancement of histone acetylation after classical conditioning is observed when comparing classically conditioned animals with animals that received an unpaired presentation of CS and US. In line with this, the inhibition of HAT inhibits eLTM formation, whereas HDAC enhances it (Merschbaecher et al. 2012). Moreover, the DNA methyltransferase 3 (dnmt3) gene is up-regulated 30 min after classical conditioning in honeybees (Lockett et al. 2010), and the ability of honeybees to differentiate between a learned odor and a novel odor at 1 and 3 d after conditioning is affected by the inhibition of DNA methylation (Biergans et al. 2012). This is particularly interesting because there are indications that dnmt3 might regulate alternative splicing in honeybees (Li-Byarlay et al. 2013).

These results are in contrast to the notion that transcription is a process that mediates only lLTM formation. One reason for this contradiction could be the “qualitatively different effects” that epigenetic mechanisms and actinomycin D have on transcription (Merschbaecher et al. 2012). HADC and HAT, for example, regulate the binding of histones to DNA and hence also the degree of DNA condensation as a precondition for transcription of a specific set of genes. In contrast, actinomycin D intercalates in the DNA double helix, preferentially blocking transcription in so-called transcription bubbles that have already been unwound by DNA helicases (Sobell 1985; Puckett and Lubin 2011; Paramanathan et al. 2012). Clearly, the learning-dependent regulation of transcription is still poorly understood and must be clarified in future experiments in honeybees. However, the results of studies on epigenetic processes indicate that transcription-dependent processes are not a unique feature of lLTM formation, but play a role in eLTM formation as well.

The fact that translation, transcription, and epigenetic processes are required for long-term memory formation suggests that de novo synthesized proteins are involved in this process in honeybees, although exactly which proteins are synthesized de novo after learning remains unclear. The antennal lobes (AL) and mushroom bodies (MB) are neuropiles of the honeybee brain where odor and reward pathways converge and thus where memory traces might be formed (Hammer and Menzel 1995). In both neuropiles, a structural reorganization is observed in areas of high synaptic density 3 d after conditioning (Hourcade et al. 2009, 2010). In the MB this requires transcription and thus most likely indicates de novo protein synthesis (Hourcade et al. 2010). These studies suggest that at least some of the proteins synthesized de novo following classical conditioning are synaptic proteins. Indeed in mice, where an analysis of the proteome following associative learning has been performed, a quantitative increase was found in 23% of proteins of the synaptic proteome after associative learning (Kahne et al. 2012).

In the same study, quantitative decreases were observed in more than twice as many synaptic proteins (59%) (Kahne et al. 2012). So not surprisingly, recent studies have demonstrated that protein degradation plays a role in long-term memory formation. In particular, the ubiquitin proteasome system (UPS) seems to be involved in LTM formation in invertebrate and vertebrate model organisms (for review, see Jarome and Helmstetter 2013). The UPS system is a network of ubiquitin ligases and proteasomal structures that label proteins for subsequent degradation by the proteasome (Sommer and Wolf 2014). In the honeybee, the proteasome inhibitors Z-Leu-Leu-Leu-CHO (MG132) and clasto-lactacystein β-lactone enhance eLTM and lLTM following conditioning with three CS–US trials (Felsenberg et al. 2012, 2014b). Accordingly, protein degradation in honeybees modulates LTM strength rather than being a prerequisite for LTM formation. This finding is in contrast to studies in other invertebrate and vertebrate model organisms, in which the inhibition of protein degradation with proteasome inhibitors during memory formation blocks LTM formation. Most of these experiments were performed in aversive learning paradigms. It remains to be shown whether this contradiction can be attributed to differences in LTM formation between honeybees and other model organisms or to differences in the mechanisms of appetitive and aversive LTM formation (Felsenberg et al. 2014b).

In summary, the formation of both eLTM and lLTM in honeybees depends on the three main processes regulating the composition of the proteome: transcription, translation, and protein degradation. The reported differences between eLTM and lLTM formation might be due to a different regulation of these processes.

Cyclic AMP-, Ca2+-, and IP3-dependent processes play a role in eLTM and lLTM formation

Candidate mechanisms for regulating transcription, translation, and protein degradation in honeybees are the signaling cascades involved in the formation of eLTM and lLTM.

Protein kinase A (PKA) plays a central role in eLTM and lLTM formation, as has been shown through experiments that systematically interfere with PKA activity and antisense oligonucleotide inhibition of a PKA regulatory subunit in the honeybee brain (Fiala et al. 1999; Müller 2000). The AL is the the first-order olfactory neuropile of the honeybee brain receiving direct input from olfactory receptors and reward relaying neurons (Sandoz 2011). PKA activity in the AL underlies eLTM and lLTM formation and is regulated by cGMP and cAMP (Müller 2000; Leboulle and Müller 2004). Cyclic GMP is subsequently synthesized by the NO-dependent guanylyl cyclase (GC) during conditioning and shortly thereafter (Müller 1996, 2000). In contrast to processes restricted to the AL, systemic inhibition of adenylyl cyclases (AC) blocks lLTM, whereas eLTM is left intact (Matsumoto et al. 2014). This result suggests that different ACs are involved in the formation of eLTM and lLTM shortly after conditioning, a notion that is supported by the cloning and characterization of different AC genes from the honeybee brain (Wachten et al. 2006; Fuss et al. 2010; Balfanz et al. 2012). So, in addition to PKA, other cAMP target proteins such as cyclic-nucleotide gated ion channels (CGN) and exchange proteins (EPAC), might play a role in eLTM and lLTM formation. In line with this, Matsumoto et al. (2014) demonstrate that the pharmacological inhibition of CGNs blocks lLTM, whereas it leaves eLTM intact.

Ca2+ is another second messenger that plays a crucial role in honeybee LTM formation. lLTM formation but not eLTM formation depends on intracellular Ca2+, Ca2+-binding protein Calmodulin (CaM) and Ca2+/Calmodulin-dependent kinase (CaM kinase) (Perisse et al. 2009; Matsumoto et al. 2014). Two modes of action that increase intracellular Ca2+ following conditioning are conceivable: First, an influx of extracellular Ca2+ through CGNs, NMDA receptors or voltage-dependent Ca2+ channels (Kloppenburg et al. 1999; Grunewald 2003; Cens et al. 2013); second, a release of intracellular Ca2+ from intracellular stores, such as the endoplasmic reticulum, mediated by the second messenger 1,4,5-trisphosphate (IP3) (Kamikouchi et al. 1998; Uno et al. 2013). Both mechanisms might play a role in LTM formation. First, reports have demonstrated that lLTM depends on CGNs (see above) and that an NMDA receptor is involved (Mussig et al. 2010), although RNAi inhibition of a honeybee NMDA receptor subunit blocked eLTM formation but not lLTM (Mussig et al. 2010). This contradiction to the dependency of lLTM formation on Ca2+ (Perisse et al. 2009) remains to be clarified. Second, the activity of protein kinase C, which is activated by diacylglycerol (DAG), increases up to 4 d after classical conditioning (Grunbaum and Müller 1998). Because DAG and IP3 result from a phospholipase C-dependent cleavage of phosphatidylinositol 4,5-bisphosphate (PIP2) (Berridge 2009), a learning-dependent increase of PKC suggests that IP3 increases after conditioning.

Taken together, these findings suggest that candidate signaling cascades regulating transcription, translation, and protein degradation in the honeybee following classical conditioning are the cAMP-dependent signaling cascade, and IP3-, and Ca2+-dependent processes.

In fact, recent reports in vertebrates have demonstrated an involvement of these three signaling molecules in regulating transcription, translation, and protein degradation by the UPS.

Transcription is initiated when transcription factors bind to their target genes; such a factor is the cAMP-response element binding protein (CREB) (Alberini 2009), which is activated by learning. In vertebrates, CREB-dependent transcription is activated by several kinases, including CaMkinase II and PKA at the kinase inducible domain (KID). Upon phosphorylation CREB binds to the transcriptional apparatus, thereby activating the transcription of target genes (for review, see Alberini 2009). AmCREB, a CREB homolog containing the KID domain and the highly conserved DNA binding and dimerization domain, the basic region-leucin zipper (bZIP), has been isolated from honeybee brain. This suggests that AmCREB is a possible target of learning-dependent alterations of cAMP and Ca2+ (Eisenhardt et al. 2003, 2006).

Learning-dependent translation in vertebrates is regulated by initiation factors and elongation factors (for review, see Gal-Ben-Ari et al. 2012). Phosphorylation of the initiation factor eIF2a generally inhibits protein synthesis but promotes the synthesis of a few transcripts including ATF-4, a repressor of CREB, and plays a critical role in LTM formation (for review, see Costa-Mattioli et al. 2009; Jiang et al. 2010). Mice with reduced levels of phosphorylated eIF2a show enhanced LTM performance (for review, see Costa-Mattioli et al. 2009). Protein synthesis of a gene encoding the transcriptional regulator CHOP that is translated although eIF2 is increased is regulated through open reading frames (uORFS) in its 5′ untranslated region (5′ UTRs) (Palam et al. 2011). This is particularly interesting, because different AmCREB splice variants isolated from the honeybee central brain contain different 5′UTRs, suggesting a differential regulation of their translation by a similar mechanism (Eisenhardt et al. 2003, 2006). Thus, AmCREB is a potential target gene in the learning-dependent regulation of translation.

A second factor that regulates protein synthesis in vertebrates and is implicated in LTM formation is the elongation factor eEF2. The phosphorylation of eEF2 generally inhibits protein synthesis while at the same time enhancing the translation of a small subset of transcripts (for review, see Taha et al. 2013). eEF2 is phosphorylated and inactivated by elongation factor-2 kinase (eEF2K), whose activity is dependent on Ca2+ and calmodulin (CaM) (Kenney et al. 2014). Moreover, this kinase is phosphorylated by PKA in vitro (Redpath and Proud 1993; Diggle et al. 2001), suggesting that PKA and other Ca2+- and cAMP-dependent processes might be involved in regulating translational elongation.

Ca2+-dependent processes might be involved in regulating the activity of the proteasome as well. Jarome and Helmstetter (2013) propose a critical involvement of the NMDA and CaMKinase II in a proteasome-dependent mechanism of LTM formation after classical conditioning. They demonstrated a Ca2+/ CaMkinase II–dependent regulation of the proteasomes’ activity in vertebrates following fear conditioning (Jarome et al. 2013), and there is accumulating evidence that protein degradation by the UPS in neurons is regulated through an NMDA-dependent mechanism (for review, see Jarome and Helmstetter 2013).

Taken together, mounting evidence from studies in vertebrates indicates that cAMP via PKA and Ca2+ and CaM-dependent processes take part in regulating transcription, translation, and protein degradation. This makes it plausible that in honeybees, cAMP- and Ca2+-dependent processes that have been demonstrated to be involved in eLTM and lLTM formation may also play a crucial role in regulating transcription, translation, and protein degradation. However, a learning-induced regulation of transcription, translation, and protein degradation by cAMP- and Ca2+-dependent processes remains to be shown in future studies in honeybees. Its demonstration would provide additional evidence for the convergence of molecular processes underlying honeybee and vertebrate associative learning.

Parameters of classical conditioning regulate the molecular mechanisms underlying LTM formation

How are the molecular processes underlying long-term memory formation in honeybees induced by learning? In honeybees, two main parameters of learning are crucial: the number of trials performed and the inter-trial interval (ITI). Experiments with foraging honeybees have shown that the stability of reward memories correlates with the number of learning trials: one-trial learning leads to a transient memory that is stable for up to ∼1 d, whereas multiple learning trials lead to strong memories lasting up to 10 d after learning (Menzel 1968). Comparable with LTM formation in free-flying honeybees LTM strength and stability, in harnessed honeybees depends on the number of conditioning trials: one trial, i.e., one CS–US pairing, leads to a weak memory whereas three conditioning trials lead to a strong memory 1 d following conditioning (Menzel 1990; Müller 1996). Early LTM (eLTM) formation depends on PKA and NO-synthase and lLTM formation depends on Ca2+ after three-trial conditioning but not after one-trial conditioning (Müller 1996; Perisse et al. 2009). In the AL, PKA activity is prolonged following three-trial conditioning when compared with conditioning with one trial (Müller 2000) and PKC activity is significantly enhanced over several days after three-trial conditioning compared with one-trial conditioning (Grunbaum and Müller 1998). Moreover, the sensitivity of LTM formation to inhibitors of translation, transcription, and protein degradation depends on the number of conditioning trials: strong LTMs formed after three-trial conditioning are sensitive to inhibitors of these three processes whereas weak LTMs formed after one-trial conditioning are not (Friedrich et al. 2004; Felsenberg et al. 2012). When honeybees are conditioned with three trials, a long-lasting enhancement of histone acetylation is observed, whereas one conditioning trial leads to a transient enhancement of histone acetylation (Merschbaecher et al. 2012). Moreover, three-trial conditioning affects the sensitivity of 1 and 2 d eLTM to drugs that interfere with histone acetylation. One trial affects the sensitivity of eLTM at 1 d but not at 2 d (Merschbaecher et al. 2012).

In most of studies where bees were conditioned with three conditioning trials, an ITI of 2 or 10 min was applied, but the different effects of these two ITIs on learning and LTM formation have not been examined. However, an effect of the ITI on LTM formation has been demonstrated in conditioning with five trials. Conditioning with an ITI of 30 sec and 3 min resulted in a significantly weaker lLTM (4 d LTM) than eLTM (1 d LTM), whereas eLTM and lLTM of similar strength were formed when using an ITI of 1 min and 10 min (Gerber et al. 1998).

Late LTM (lLTM) formation following conditioning with five trials is sensitive to the transcriptional inhibitor actinomycin D, depending on the time point of inhibitor injection (Lefer et al. 2012). Menzel et al. (2001) reported a different sensitivity of LTMs to the transcriptional inhibitor actinomycin D, depending on the ITI between five conditioning trials: Actinomycin D inhibits eLTM and lLTM formation following conditioning with an ITI of 10 min (spaced training), whereas it blocks lLTM but not eLTM formation after conditioning with an ITI of 30 sec (massed training) (Menzel et al. 2001). These studies demonstrate that the number of trials and the ITI between trials regulate molecular mechanisms of LTM formation in honeybees.

Opposing reward memories influence the behavior of foraging honeybees

Above it is suggested that classical conditioning resembles the way free-flying bees learn to associate the features of a food source with its reward. Interestingly, when the productivity of a food source declines, honeybees decrease their foraging behavior toward this particular food source and shift to a more efficient one (Menzel 1968; Couvillon and Bitterman 1980, 1984; Greggers and Menzel 1993; Greggers and Mauelshagen 1997). But honeybees will return to food sources previously experienced as being productive for several days (Moore et al. 2011; Wagner et al. 2013). Thus, instead of overwriting their memory of a food scource that is no longer rewarded, honeybees form a memory regarding both currently and previously rewarded food sources. The number of training trials and the reward duration affect the honeybee's decline in foraging behavior toward this nonrewarded food source (Menzel 1968; Couvillon and Bitterman 1980, 1984), indicating an interplay between opposing learning processes: learning about the reward provided by a food source and learning about its failure.

The decline in a bee's behavior toward an unrewarded food source resembles the behavioral phenomenon of extinction after withdrawal of the US in classical conditioning of vertebrates, in which animals learn about the failure of a previously experienced US and form an extinction memory concerning this experience (Pavlov 1927; Myers and Davis 2002).

Molecular mechanisms of long-term extinction memory are regulated by the parameters of classical conditioning

Extinction is observed in harnessed honeybees when the CS alone is presented multiple times after the animals have been successfully conditioned. Several studies examine the extinction of short-term and long-term memories (Bitterman et al. 1983; Sandoz and Pham-Delegue 2004; Stollhoff et al. 2005; Stollhoff and Eisenhardt 2009; Plath et al. 2012). Three behavioral phenomena—renewal, reinstatement, and spontaneous recovery—demonstrate a time- and context-dependent reappearance of the CR after extinction (Bouton 2002; Myers and Davis 2002). These phenomena have first been described in vertebrates and have been interpreted as an indication that the original, extinguished memory is transiently suppressed, but not erased, following extinction learning (Myers and Davis 2002; Bouton and Moody 2004). Two of these behavioral phenomena, reinstatement and spontaneous recovery, have been described in honeybees, demonstrating a time- and context-dependent reappearance of the CR after extinction. This suggests that in honeybees the original, extinguished memory is also surpressed by the extinction memory but not erased following extinction (Sandoz and Pham-Delegue 2004; Stollhoff et al. 2005; Plath et al. 2012).

Stollhoff et al. (2005) studied extinction of eLTM 1 d after conditioning in harnessed honeybees. They demonstrated that extinction depends on the number of extinction trials: the more extinction trials are applied, the more extinction can be observed (Stollhoff et al. 2005). After two extinction trials a long-term extinction memory is formed and is behaviorally visible 1 d after extinction (Stollhoff et al. 2005). The formation of this long-term extinction memory is susceptible to an inhibition of protein synthesis (Stollhoff et al. 2005; Stollhoff and Eisenhardt 2009) and to sleep deprivation (Hussaini et al. 2009). Both a susceptibility to protein synthesis inhibition and sleep deprivation indicate that the formation of a long-term extinction memory undergoes a progressive stabilization of the memory trace, i.e., consolidation.

Stollhoff and Eisenhardt (2009) revealed a direct impact of a parameter of classical conditioning, i.e., US duration, on the molecular mechanism of extinction memory formation: Protein synthesis-dependent long-term extinction memory is only observed when the presentation of the US during classical conditioning exceeds 2 sec (Stollhoff and Eisenhardt 2009).

A second study suggests an interrelation between classical conditioning and extinction memory formation and indicates that this interplay is mediated by underlying molecular mechanisms. Felsenberg et al. (2012) extinguished eLTM after classical conditioning and tested the impact of the UPS inhibitor MG132 on the formation of a long-term extinction memory. They reported an inhibition of long-term extinction memory by MG132. Because an enhancement of LTM formation after classical conditioning by the UPS inhibitor was observed in this study (see above), the authors concluded that extinction 1 d after classical conditioning results in strengthening the original memory (re)activated upon extinction. This strengthened original memory controls behavior despite the fact that extinction takes place. Accordingly, protein degradation might constrain the original memory following extinction, a mechanism that permits the behavioral expression of extinction memory (Felsenberg et al. 2012). The effect of MG132 on long-term extinction memory formation depends on the number of conditioning trials performed during classical conditioning, indicating that molecular mechanisms such as protein degradation during long-term extinction memory formation crucially depend on the parameters of classical conditioning.

Additionally, a third study on the role of epigenetic mechanisms suggests an interrelation between molecular mechanisms of LTM formation following classical conditioning and extinction: Lockett et al. (2010) demonstrated that the application of a DNA methyltransferase 3 inhibitor 24 h before and immediately after conditioning enhances extinction of a 1 d LTM. This suggests that methylation before and during conditioning prevents rapid extinction 24 h later and accordingly increases the LTM's resistance to extinction. In contrast, application of the same inhibitor 2 h before extinction inhibits it, suggesting that methylation 22 h after acquisition enhances extinction (Lockett et al. 2010). Thus, gene methylation during conditioning might be the prerequisite for the formation of a stable memory, counteracting the control of behavior by extinction memories. Methylation at later time points after conditioning might lower the resistance to extinction. These findings suggest that DNA methylation has an impact on the balance between the original memory and the extinction memory.

How can these results be reconciled? Two studies reviewed above show that the parameters of learning have an impact on LTM strength and its underlying molecular mechanisms in honeybees. According to Rescorla and Wagner (1972), the cause for extinction and extinction learning is a mismatch between the predicted occurrence of the US following classical conditioning and the experienced failure of the US during extinction. The more conditioning trials an animal experienced, the more association strength, i.e., predictive power, the CS acquired. Thus, the extent of a mismatch between what is predicted by the CS and what is experienced is smaller when animals have been conditioned with a short US than with a long US (see Stollhoff et al. 2009) or when the animals have been conditioned with one trial rather than with many trials (see Felsenberg et al. 2012). Accordingly, both studies suggest that the extent of the mismatch has to exceed a certain threshold to trigger molecular mechanisms of long-term extinction memory formation. These mechanisms might ensure that LTMs are formed only when meaningful changes in US magnitude (i.e., its duration) or US reliability (i.e., the number of trials during conditioning) occur. Such findings suggest that during learning, long-lasting memories are compared with current experiences and that the outcome of this comparison is crucial for the molecular mechanisms of LTM formation. Because studies on extinction in honeybees indicate the formation of a parallel extinction memory, the comparison of current experiences with previous memories triggers the formation of a new long-term memory instead of overwriting the initially formed memory. Thus two opposing memories, the initial memory formed and the extinction memory, together contribute to behavior when the animals encounter the respective stimuli and most likely enable honeybees, depending on the weight of each memory, to return to food sources previously experienced as being productive for several days (Moore et al. 2011; Wagner et al. 2013).

Conclusion

As described above, the honeybee is one of the best characterized invertebrate organisms with regards to classical conditioning phenomena. Our studies on the molecular mechanisms of long-term memory formation in honeybees indicate that the parameters of classical conditioning induce long-term memory formation following classical conditioning and extinction. These findings suggest that in honeybees, LTM formation depends on a bee's previous experiences in addition to its current learning experience. Moreover, these studies demonstrate how an in-depth analysis of the molecular mechanisms of learning behavior can add to our understanding of associative learning in honeybees. Therefore, future studies on the molecular mechanisms underlying phenomena related to classical conditioning in honeybees, such as backward conditioning (Felsenberg et al. 2014a) or trace conditioning (Szyszka et al. 2011) will lead to a greater understanding of the connection between molecular and behavioral processes.

The honeybee genome was sequenced in 2006 (Honeybee Genome Sequencing Consortium 2006), and assembly and annotation of the genome have been upgraded in 2014. This has improved our insights into genomic structure through an improved gene set and a more comprehensive knowledge of protein functions (Elsik et al. 2014). This growing wealth of information is crucial for an efficient analysis of the molecular mechanisms underlying learning and memory in honeybees. It permits a comparision of genes and proteins involved in associative learning in honeybees and vertebrates, the development of tools for the analysis of molecular processes, including antibodies, predictions about the effects of existing pharmacological agents in honeybee behavioral pharmacology, and choosing appropriate oligonucleotide or RNAi sequences to interfere with gene functions during learning and memory formation (Fiala et al. 1999; Mussig et al. 2010; El Hassani et al. 2012; Leboulle et al. 2013). Moreover, information about gene sequences and the potential functions of the proteins they encode, combined with emerging techniques aimed at generating transgenic bees (Schulte et al. 2014), will further the molecular analysis of honeybee learning and memory formation. Thus this molecular information on the honeybee, combined with existing knowledge on the molecular mechanisms that underlie honeybee memory formation and the high convergence of classical conditioning phenomena in honeybees and vertebrates, will advance future in-depth analysis of the mechanisms of honeybee classical conditioning and should equally contribute to our understanding of associative learning in vertebrates.