The roles of dopamine and related compounds in reward-seeking behavior across animal phyla.

Motile animals actively seek out and gather resources they find rewarding, and this is an extremely powerful organizer and motivator of animal behavior. Mammalian studies have revealed interconnected neurobiological systems for reward learning, reward assessment, reinforcement and reward-seeking; all involving the biogenic amine dopamine. The neurobiology of reward-seeking behavioral systems is less well understood in invertebrates, but in many diverse invertebrate groups, reward learning and responses to food rewards also involve dopamine. The obvious exceptions are the arthropods in which the chemically related biogenic amine octopamine has a greater effect on reward learning and reinforcement than dopamine. Here we review the functions of these biogenic amines in behavioral responses to rewards in different animal groups, and discuss these findings in an evolutionary context.

Introduction

Most motile animals show some form of active foraging behavior to locate resources they need in their environment, and will actively avoid stimuli that are harmful to them. These basic behavioral responses have been used to provide a simple operational definition of whether stimuli are rewarding or punishing to an animal. In the field of animal behavior research, rewarding and punishing stimuli are often defined simply by the nature of the responses they elicit: rewards elicit approach behavior, whereas punishing stimuli elicit avoidance behavior (Skinner, 1938). Rewards also have a reinforcing property, in that almost all motile animals studied will learn to repeat actions that bring about (or bring closer) a rewarding outcome. Reinforcement in this context describes the process of “stamping in” actions that result in attaining the reward (Wise and Rompre, 1989; Wise, 2004). Given this operational definition of reward, it can be said that reward-seeking and reward learning are fundamental aspects of animal behavior. These behavioral responses appear to be universal across animal groups. Reward-seeking was recognized as fundamental to even the earliest explanatory models of behavior: Sherrington (1906), Tinbergen (1951), and Lorenz (1965) all developed behavioral models that incorporated an assumed innate “drive” to seek rewards.

Exploring the neurobiology of reward responses and reward processing has long been a major focus of neuroscience research. Understandably the vast majority of studies have considered humans and other mammals, while more recently, mechanisms of reward processing have been studied in a wider range of animal model systems. Mammalian research has clearly established that the circuit connecting midbrain dopaminergic neurons to the ventral striatum and prefrontal cortex is central to mammalian brain reward systems (Wise and Rompre, 1989; Koob and Le Moal, 1997; Schultz, 2000, 2007; Watson and Platt, 2008). Dopamine is a key modulator of this circuit, and of behavioral responses to rewards (Berridge and Robinson, 1998; Roitman et al., 2004; Wise, 2004; Berridge et al., 2009).

More recent comparative studies with diverse species scattered across a divergent range of animal phyla have also repeatedly demonstrated roles for the biogenic amines, and especially dopamine, in reward-seeking and reward learning. This raises the question of whether the function of dopamine in reward-seeking and learning has been broadly conserved across animal phyla, or whether this is a case of convergent evolution. In this review we discuss the evidence that dopamine affects reward responses across the Nematoda, Vertebrata, Platyhelminthes and Mollusca, representing highly divergent phyla. We then discuss how phylum Arthropoda appear to be an exception in that most studies have shown reward responses and reward learning to be affected by octopamine rather than dopamine. Finally we place this information in an evolutionary context.

Dopamine and Reward Responses in Nematoda, Platyhelminthes, Mollusca, and Vertebrata

Dopamine has been shown to affect responses to reward in extremely diverse animal groups, but the documented effects of dopamine vary. Most studies with invertebrates have used food stimuli as rewards since these tend to elicit the most robust behavioral responses and learning.

Nematodes (e.g., Caenorhabditis elegans) possess one of the simplest animal nervous systems, but even in this animal, dopamine affects the expression of a form of food-seeking behavior. C. elegans possesses dopaminergic mechanosensory neurons that release dopamine in the presence of bacterial food (Sawin et al., 2000; Rivard et al., 2010). Dopamine release from these neurons reduced crawling speed (Sawin et al., 2000; Rivard et al., 2010), and dopamine modulation of the locomotor circuit also increased turning behavior (Hills et al., 2004). Such behavioral responses to food are greater in starved worms than fed worms (Sawin et al., 2000). These changes in locomotor behavior in response to food are a very simple form of reward-seeking behavior. The effects of dopamine on locomotion result in an “exploratory” area-restricted search pattern of locomotion, which ensures that the animal explores and dwells longer in an area containing food (Hills et al., 2004), and the outcome of their change in crawling behavior is that they are more likely to locate food. It is striking that even this elementary form of reward-seeking behavior is modulated by dopamine.

Caenorhabditis elegans learned to locate food faster in a simple T-maze after repeated training trials with the apparatus (Qin and Wheeler, 2007). Dopamine deficient cat-2 mutant worms, which lack the enzyme tyrosine-hydroxylase necessary for the biosynthesis of dopamine (Figure 1), learned the maze task less well, and the memory did not persist as long as in wild-type animals (Qin and Wheeler, 2007). This suggests that dopamine affects memory systems involved in food-seeking behavior in nematodes.

Figure 1

Chemical relationships and biosynthetic pathways linking dopamine, tyramine, octopamine, and norepinephrine (enzymes in italics). Dopamine, tyramine, octopamine, and norepinephrine are all derived from tyrosine. In order to synthesize dopamine, tyrosine is first converted to DOPA by tyrosine-hydroxylase, which is then decarboxylated by DOPA-decarboxylase to yield dopamine. Tyramine is either produced directly from tyrosine by tyrosine-decarboxylase, or (more rarely) dehydroxylated from dopamine by dopamine-dehydroxylase. This figure summarizes the most common synthesis pathways, but there are variations among the phyla. In some groups, octopamine is a trace amine and synthesized from tyramine by dopamine-β-hydroxylase, while in other phyla, norepinephrine is physiologically irrelevant and not present at any biologically meaningful level.

The Platyhelminthes (flatworms) are also simple animals. They have no body cavities and a simple nervous system, but they do have rudimentary cephalization, and they can learn both classical and operant conditioning tasks (Best and Rubinstein, 1962; Shafer and Corman, 1963). Dopamine seems to be involved in the mechanism of reinforcement in the flatworm Dugesia japonica. Kusayama and Watanabe (2000) developed a conditioned-place-preference assay for D. japonica, and were able to induce in the flatworms a preference for an environment in which they had been treated with methamphetamine (which increases extracellular biogenic amine levels). This preference could be eliminated by treatment with three different antagonists characterized in mammals as active against dopamine receptors (Kusayama and Watanabe, 2000), suggesting that dopamine is involved in reinforcement in planarians.

Far more is known about the role of dopamine in reinforcement in the mollusk Aplysia (Lechner et al., 2000a,2000b). In Aplysia, the ingestion of seaweed involves rhythmic and coordinated movements of the foregut and peri-oral structures. Lechner et al. (2000a) developed a training protocol for the classical conditioning of biting in Aplysia so that animals could be trained to associate a light touch to the lips with a paintbrush with the presentation of seaweed reward. After conditioning, the number of biting responses to the lip tickle was increased (Lechner et al., 2000a,2000b). Lesioning the esophageal nerve blocked this form of conditioning, showing that the esophageal nerve mediates the reinforcing property of the unconditioned food reward during classical conditioning (Lechner et al., 2000b). Brembs et al. (2002) were able to train the biting response in a slightly different operant paradigm that paired biting with direct electrical stimulation of the esophageal nerve, providing more evidence that the esophageal nerve conveys the reinforcement signal. The esophageal nerve contains many processes that are dopaminergic (Kabotyanski et al., 2000), and Brembs et al. (2002) were able to replace electrical stimulation of the esophageal nerve with iontophoretic application of dopamine onto selected post-synaptic neurons to achieve effective training. This showed conclusively that dopamine is the neurochemical mediator of the reinforcement signal in operant conditioning. In the classical conditioning paradigm, association of a tactile stimulus with food could also be blocked by treatment with the dopamine receptor antagonist, methylergonovine (Reyes et al., 2005). Therefore evidence from both classical and operant conditioning studies in Aplysia suggest a role for dopamine in reinforcement and reward learning.

The Nematoda, Platyhelminthes and Mollusca are representatives of three different superphyla within the protostomes (Figure 2). The Chordata are deuterostomes, and a separate lineage from the protostome groups (Figure 2). Despite all this taxonomic diversity dopamine affects behavioral responses to reward and reinforcement in a similar manner. The affects of dopamine on mammalian reward responses have been well reviewed recently (Schultz, 2007), and hence here we consider only the main findings as relevant to this comparative review.

Figure 2

Phylogeny of groups discussed in this paper, based on nearly complete ribosomal RNA gene analyses (Mallatt et al., . Discussed phyla represent examples of protostome, deuterostome, and diploblastic groups, yet dopamine is a modulator in the motor systems of all these diverse phyla, and affects responses to rewarding or punishing stimuli in mollusks, platyhelminths, arthropods, nematodes, and chordates.

Dopamine has long been identified with motor function in mammals, but the first evidence linking dopamine to reward-seeking came from the observation that moderate dopamine receptor antagonist treatments attenuated the motivation to respond to a food reward before compromising the ability to respond (Wise and Schwartz, 1981; Wise, 2004). This effect appeared to be caused in part by dopamine receptor antagonists eliminating the reinforcing properties of rewards (Wise and Schwartz, 1981). Conversely, stimulation of the midbrain dopamine system is strongly reinforcing. Animals will work at lever-press and other tasks for electrical stimulation of dopaminergic midbrain regions such as the ventral tegmental area and lateral hypothalamus (Routtenberg and Lindy, 1965; Carlezon and Chartoff, 2007; Watson and Platt, 2008). Such stimulation can be chosen in preference over food or water reward (Routtenberg and Lindy, 1965).

Dopamine regulates learning of stimuli associated with reward: it is essential for both the establishment and expression of conditioned reinforcement via associative learning (Wise, 2004). Dopamine-selective lesions of the nucleus accumbens can block responding to reward-associated stimuli (Taylor and Robbins, 1986), whereas amphetamine injection into the nucleus accumbens, to elevate extracellular dopamine in this area, enhances responding (Taylor and Robbins, 1984).

Recordings from midbrain dopamine neurons in mammals have shown strong responses to both primary rewards (such as food and water), and also conditioned stimuli associated with rewards (Schultz, 1998; Schultz, 2001). Most midbrain dopaminergic neurons projecting to the nucleus accumbens and frontal cortex are tonically active (meaning a relatively constant “baseline” level of firing), but show phasic activation (bursts of enhanced firing) following primary food rewards, or stimuli learned to be predictive of reward (Schultz, 2000, 2007). These reward responses are not unconditional; rather the intensity of the phasic activation is modulated by reward predictability (Hollerman and Schultz, 1998; Schultz, 1998). Most midbrain dopaminergic neurons have a tonic firing rate that is strongly enhanced by unexpected rewards far more than expected rewards, while the neuronal firing rate drops below baseline in response to expected rewards that do not appear (Schultz, 2001, 2007). This pattern of activity appears to represent the reward prediction error: that being the difference between predicted and obtained rewards. Reward prediction error is central to reward-driven learning according to the Rescorla–Wagner model of learning (Rescorla and Wagner, 1972; Schultz, 2000; Pessiglione et al., 2006).

Subsecond changes in the amount of dopamine released into the nucleus accumbens appear to directly modulate reward-seeking behavior (Roitman et al., 2004). Short pulses of dopamine released into the nucleus accumbens were recorded in rats trained to lever-press for sucrose in response to stimuli signaling the start of a lever-pressing session. Lever-presses were coincident with the peaks of the dopamine surge (Roitman et al., 2004). The taste of sugar evoked a similar short pulse of dopamine release into the nucleus accumbens, whereas quinine (an aversive taste) suppressed dopamine release (Roitman et al., 2008). Together, these findings show that the phasic responses of dopamine neurons signal an assessment of the current value of reward stimuli, and that these dopamine signals directly modulate behavioral responses to rewards.

Roles of Dopamine and Octopamine in Reward Responses in Arthropoda

So far, we have discussed examples from four phyla of highly diverse animals in which dopamine dominates reward learning and the reinforcing properties of rewards, but the Arthropoda do not seem to fit this pattern. The Arthropoda are ecdysozoan protostomes most closely related to Nematoda (Figure 2), but within this group evidence from both insects and crustaceans has shown that octopamine affects reward learning and behavioral responses to rewards (Hammer, 1997; Hammer and Menzel, 1998; Schwaerzel et al., 2003; Unoki et al., 2005; Vergoz et al., 2007; Kaczer and Maldonado, 2009; Selcho et al., 2009).

Among the vertebrates, octopamine (chemically similar to both dopamine and noradrenaline, Figure 1) is a trace amine whose physiological importance is presently not well established (Burchett and Hicks, 2006). By contrast, in the arthropods, octopamine is a major regulator of behavior and physiology (Roeder et al., 2003; Roeder, 2005). The similarities between the octopamine receptor subtypes in protostomes and adrenergic receptor subtypes in vertebrates suggest these two systems may have diverged from a common evolutionary origin (Evans and Maqueira, 2003; Maqueira et al., 2005; Pfluger and Stevenson, 2005). As detailed below, pharmacological studies with Crustacea and Insecta have shown that octopamine affects reward learning and reward responses more strongly than dopamine.

Kaczer and Maldonado (2009) showed that in the crab Chasmagnathus granulates, octopamine treatments influenced expression of a learned exploratory response triggered by experiencing food in a novel environment (Kaczer and Maldonado, 2009). Octopamine injection enhanced the exploratory response to food, whereas injection of two octopamine receptor antagonists reduced this response (Kaczer and Maldonado, 2009).

Similar regulation of food reward by octopamine has also been demonstrated in insects. Diverse studies with honey bees (Apis mellifera, Hymenoptera) have shown that octopamine treatment affects behavioral responses to sucrose reward (Mercer and Menzel, 1982; Hammer and Menzel, 1998; Scheiner et al., 2002; Schulz et al., 2002; Barron et al., 2007). A robust and widely used assay for appetitive conditioning in honey bees is proboscis extension response conditioning where bees learn to extend their proboscis in response to a novel odor paired with the presentation of sucrose reward (Kuwabara, 1957; Bitterman et al., 1983). Dopamine microinjection into the brain reduced performance in appetitive conditioning of proboscis extension (Mercer and Menzel, 1982), whereas microinjection of octopamine into either the mushroom bodies or antennal lobe could substitute for sucrose presentation in training (Hammer and Menzel, 1998). Some of the VUM (Ventral unpaired median)neurons respond to sucrose (Hammer, 1993; Schroeter et al., 2007), and one of these (VUMmx1) has been shown to mediate sucrose reinforcement (Hammer, 1993). This neuron is believed to be octopaminergic (Menzel, 2001).

It is of interest to note that thoracic octopamine injection increased reflexive proboscis extension responsiveness to sucrose in an unconditioned paradigm in honey bees, whereas dopamine receptor agonist treatment reduced responsiveness (Scheiner et al., 2002). This suggests an opponent relationship between octopamine and dopamine systems in response to sucrose reward. More recent pharmacological studies with the cricket (Gryllus bimaculatus, Orthoptera) (Unoki et al., 2005, 2006; Mizunami et al., 2009) and honey bees (Farooqui et al., 2003; Vergoz et al., 2007) have shown that treatments with octopamine receptor antagonists and agonists affected performance in reward learning assays, but treatment with dopamine receptor antagonists and agonists affected performance in aversive learning assays. As a result of these studies a commonly held view is that for the arthropods, octopamine and dopamine modulate different motivational systems with octopamine modulating appetitive learning and dopamine modulating aversive learning (Beggs et al., 2007; Vergoz et al., 2007). However, new research with Drosophila (described below) suggests that this interpretation may be an oversimplification.

The arthropod studies described so far have relied heavily on pharmacological tools to manipulate biogenic amine systems. A difficulty with this approach is that the affinities of most of the available biogenic amine receptor agonists and antagonists to all the biogenic amine receptors in the different experimental insect species are incompletely known. Consequently, it is difficult to experimentally manipulate a single receptor system in isolation or to be completely confident that nominated agonists or antagonists do not affect more than one biogenic amine system. Currently, the only solution to this problem is to use several different antagonists or agonists against the same receptor system(s), and hopefully show the same behavioral effects (Unoki et al., 2005; Vergoz et al., 2007). Also, in many cases pharmacological treatments have been applied to the whole organism or to the whole brain, which has limited a circuit-level analysis of reinforcement systems in arthropods.

Genetic Analyses of the Function of the Biogenic Amines in Reward and Aversive Learning in Drosophila

The genetic tools available for Drosophila melanogaster (Diptera) have enabled very different approaches to investigate the roles of octopamine and dopamine in reward responses. In Drosophila, several studies have used different genetic tools to manipulate all (or most) dopaminergic or octopaminergic neurons in the fly brain. The conclusions of these studies are consistent in that they have shown that dopamine is required for aversive learning, but not reward learning, and octopamine is required for reward learning but not aversive learning (Schwaerzel et al., 2003; Schroll et al., 2006; Claridge-Chang et al., 2009; Honjo and Furukubo-Tokunaga, 2009). However, more recent studies have used more selective genetic manipulations to target specific dopamine receptors, or specific small groups of dopamine neurons. These have shown that some dopamine signals may also modulate reward responses in Drosophila (Kim et al., 2007; Krashes et al., 2009; Selcho et al., 2009). In this section we first review studies that have manipulated all dopaminergic or octopaminergic neurons in the fly brain, and then studies that have selectively targeted specific populations of dopamine neurons, or dopamine receptor systems. We then discuss how findings from genetic studies with Drosophila can be reconciled with pharmacological studies with other arthropods.

Schwaerzel et al. (2003) explored the role of octopamine and dopamine in appetitive and aversive conditioning using strains of Drosophila melanogaster in which the enzymes responsible for the synthesis of different biogenic amines were under the control of heat-shock sensitive promoters. Flies in which the tyramine-β-hydroxylase (Figure1) gene had been knocked out could not synthesize octopamine (Monastirioti et al., 1996). These flies performed normally in an aversive learning task associating electric shock with a novel odor, but did not learn to associate a sugar reward with an odor (Schwaerzel et al., 2003). This defect could be rescued by a transgene containing the wild-type tyramine-β-hydroxylase gene downstream of a heat-shock promoter, such that after heat-shock to activate the promoter and restore octopamine synthesis, flies performed normally in both the appetitive and aversive learning tasks (Schwaerzel et al., 2003).

To examine the role of dopamine signaling in the two learning assays Schwaerzel et al. (2003) used a sophisticated gene construct that enabled neurotransmitter release from dopaminergic neurons to be blocked by maintaining flies at an elevated temperature. At the restrictive temperature, flies performed poorly in the aversive learning paradigm, but normally in an appetitive learning paradigm (Schwaerzel et al., 2003). Similar findings have been reported for Drosophila larvae (Honjo and Furukubo-Tokunaga, 2009). The conclusion is that for adult and larval Drosophila, octopamine affects learning of rewarding stimuli and dopamine affects learning of aversive stimuli.

Relatively new genetic tools allow neuronal activity to be modulated by light pulses, which has allowed researchers to study the behavioral changes that result when octopaminergic or dopaminergic cell populations are activated in association with different environmental stimuli. To investigate the roles of octopamine and dopamine in learning by Drosophila larvae, Schroll et al. (2006) used channelrhodopsin gene constructs that allowed different neuronal populations to be activated by pulses of blue light. Larvae learned to avoid an odor that had been paired with light activation of dopaminergic neurons, but they became attracted to odors paired with light activation of octopaminergic and tyraminergic neurons (Schroll et al., 2006). The inference is that activity of dopaminergic neurons mediates punishment, whereas activity of octopaminergic or tyraminergic neuron populations mediates the reinforcing properties of reward (Schroll et al., 2006).

Claridge-Chang et al. (2009) were able to optically activate populations of dopaminergic neurons in transgenic adult flies with a burst of laser light, by driving the expression of ATP-gated P2X2 channels in dopaminergic neurons, and using laser light to trigger ATP release from a previously microinjected caged precursor (Claridge-Chang et al., 2009). When laser-activation of dopaminergic neurons was associated with a specific odor cue, flies learned aversion to the odor.

The consistent message from the Drosophila studies discussed so far is that for both larval and adult flies, octopamine is necessary for the learning of food reward, and dopamine is necessary for aversive learning. This is in agreement with the main findings from pharmacological studies performed with other arthropods (Unoki et al., 2005; Vergoz et al., 2007). However, more targeted genetic manipulations of specific dopamine signals in Drosophila suggest that this understanding of dopamine's role in insects is an oversimplification.

In the insect brain, the mushroom bodies are a protocerebral higher brain center known for their roles in olfactory processing and learning and memory (Farris, 2008). In Drosophila, the mushroom bodies are necessary for associative learning (de Belle and Heisenberg, 1994; Heisenberg, 1998; Schwaerzel et al., 2003; Margulies et al., 2005; Krashes et al., 2007). The dopamine receptor dDA1 (a D1-like dopamine receptor that activates adenylyl cyclase) is highly expressed in adult Drosophila mushroom bodies, and also other regions of the brain. Kim et al. (2003) identified two mutants dumb and dumb that eliminated expression of dDA1 in the adult mushroom bodies and central complex. Both dumb mutants completely failed to learn the association of an odor stimulus with electric shock, and also showed partial impairment of learning of an odor associated with sucrose reward (Kim et al., 2007). These defects could be rescued by restoring dDA1 expression in the mushroom bodies (Kim et al., 2007). A study with larval Drosophila also reported that dumb and dumb mutants were defective in both aversive and appetitive learning assays (Selcho et al., 2009), supporting the conclusion that signaling via the dDA1 receptor in the mushroom bodies modulates learning of both rewarding and punishing stimuli.

Krashes and Waddell (2008) have shown that the level of satiation of Drosophila influences the performance of flies in assays of appetitive memory. In fed flies, appetitive memory performance is low because mushroom body neurons are inhibited by tonic dopamine release from a population of dopaminergic neurons innervating the medial lobe and pedunculus of the mushroom body (the MB-MP neurons Krashes et al., 2009). Stimulation of neurons expressing neuropeptide F (dDPF) promoted appetitive memory performance in flies, mimicking the performance levels seen in hungry flies (Krashes et al., 2009). dNPF is an ortholog of mammalian neuropeptide Y that regulates food-seeking in mammals (Tatemoto et al., 1982; Kaira, 1997). One action of dNPF is to suppress the inhibitory MB-MP neurons, which then enables the expression of food-associated conditioned responses (Krashes et al., 2009). Therefore, a specific dopamine signal inhibits mushroom body neurons, and reduces the expression of appetitive memory.

To conclude the discussion of the arthropods; both pharmacological treatments and genetic manipulations of brain octopamine and dopamine systems have suggested different behavioral roles for octopamine and dopamine. Octopamine affects reward responses and dopamine affects punishment responses (Schwaerzel et al., 2003; Unoki et al., 2005; Vergoz et al., 2007). But new genetic studies that have manipulated specific dopamine signals have shown that in Drosophila, different dopamine signals affect expression of learned responses to both rewarding and punishing stimuli (Kim et al., 2007; Krashes et al., 2009; Selcho et al., 2009). It would seem that pharmacological or genetic manipulations of the whole brain might not have been selective or precise enough to reveal all the behavioral effects of different dopamine signals.

The arthropods are different from the other phyla discussed so far in that octopamine has been shown by most studies to play a dominant role in mediating reward responses and reward learning, but it now seems likely that in Drosophila different dopamine signals affect expression of learned responses to both rewarding and punishing stimuli. This complexity parallels what is known of the many different behavioral roles of dopamine in mammals. In the mammalian brain, dopamine is most well known for its important role in the reward systems, but distinct mesolimbic dopamine signals mediate behavioral responses to aversive events and stress also (Ikemoto and Panksepp, 1999; Pruessner et al., 2004; Alcaro et al., 2007; Schultz, 2007; Fadok et al., 2009; Diaconescu et al., 2010).

The Evolution of Brain Reward Systems: Inferences from Comparative Neurochemistry

From a phylogenetic perspective, the link between dopamine and behavioral responses to reward is extremely broad (Figure 2). In this section we consider the implications of the similar behavioral roles of dopamine in various different phyla for the evolution of brain reward systems.

A consideration of the general behavioral functions of the biogenic amines across animal phyla suggests that dopamine could have been predisposed to evolve functions in reward processing from an ancestral role as a signaling molecule modulating motor circuits in response to salient environmental stimuli. In one of the simplest metazoans, the nematode C. elegans, dopamine functions to modulate motor output and locomotor behavior, and is released in response to environmental stimuli that signal the local abundance of food (Hills et al., 2004). As far as we know, dopamine modulation of motor circuits has been reported for every animal phylum in which it has been investigated: Nematoda (Sawin et al., 2000; Rivard et al., 2010), Platyhelminthes (Buttarelli et al., 2000, 2008; Raffa et al., 2001), Annelida (Esch and Kristan, 2001, 2002; Friesen and Kristan, 2007), Mollusca (Pavlova, 2001), Arthropoda (Burrows, 1996), chordata (Grillner et al., 1995; Jordan et al., 2008) and also diploblastic Cnidaria (Chung and Spencer, 1996; Kass-Simon and Pierobon, 2007).

The Cnidaria represent perhaps the simplest animal nervous systems, and molecular and morphological evidence places the Cnidaria as basal among metazoans (Mallatt et al., 2010). In the cnidarian Hydra japonica, dopamine affects the extent of mouth opening in response to food stimuli (Hanai and Kitajima, 1984). It seems likely that modulation of motor circuits in response to environmental stimuli could be one of the ancestral functions of dopamine as a signaling molecule in simple nervous systems. From this proposed ancestral role, different biogenic amine systems could have evolved progressively more specialized functions in behavioral responses to rewarding or aversive stimuli as increasing levels of behavioral complexity evolved along with the evolution of more complex nervous systems (Hills, 2006). This hypothesis would explain why dopamine and other biogenic amines have roles in aversive responses (Schwaerzel et al., 2003; Schroll et al., 2006; Alcaro et al., 2007; Schultz, 2007; Claridge-Chang et al., 2009) and in setting the general level of arousal (Andretic et al., 2005; Kume et al., 2005; Monti and Monti, 2007; Krashes et al., 2009), as well as in reward responses across many phyla.

There are now several examples of genes, gene pathways or signaling molecules that appear to have “conserved” behavioral roles across vertebrates and invertebrates. As examples, cyclic AMP-dependent protein kinase-related proteins are involved in learning and memory across diverse vertebrate and invertebrate groups (Dubnau et al., 2003; Kandel, 2006). Cyclic GMP-protein kinases affect various form of foraging behavior across nematodes and arthropods (Fitzpatrick and Sokolowski, 2004; Toth and Robinson, 2007) and serotonin has a role in aggression across vertebrates and invertebrates (Kravitz, 2000). The roles of dopamine in reward responses across phyla is another example of what seems to be a general behavioral mechanism, but is this the product of conservation or convergent evolution?

When considering traits shared across phyla distinguishing between conservation and convergence is not simple. The difficulty is illustrated by considering the case of eye evolution (Fernald, 2006). Based on morphological evidence it was thought that vertebrate and invertebrate eyes evolved independently, and their similarities were the result of convergent evolution. But detailed molecular genetic analyses of the process of eye development have shown that eye development in vertebrate and invertebrates is organized by homologous gene families (Pichaud and Desplan, 2002). This has renewed debate over whether eyes have evolved repeatedly, or once from an ancestral light-sensitive structure.

In the field of Evolutionary Development there is now the concept of a basic genetic “toolkit” for development that is broadly conserved across diverse taxa (Carroll, 2005). The “toolkit” concept recognizes that common genomic elements can be involved in the development of different structures across phyla, even if the way the tools are used and the structures formed are very different between groups (Carroll, 2005). Similarly Toth and Robinson (2007) have argued that the “toolkit” concept can be extended to aid in understanding the evolution of different forms of behavior. A core “toolkit” of genes and signaling molecules could have been adapted and used in different ways as various complex forms of behavior evolved (Toth and Robinson, 2007).

The toolkit concept can perhaps explain dopamine's role in reward responses across phyla. Ancestrally biogenic amines may have functioned as signaling molecules in nervous systems released in response to environmental stimuli, but these simple behavioral elements have been adapted and modified in various ways as new and more complex behavioral responses to reward and punishment evolved. Developmental and molecular evidence indicates that higher brain centers have evolved independently in different phyla (Farris, 2008). Here, higher brain centers are defined as multimodal areas that gather and integrate information from lower unimodal regions for integration of sensory information/associations, behavioral flexibility and “cognitive” behavior (Farris, 2008). Reward systems in arthropods and vertebrates both extensively involve higher brain centers (Hammer, 1993; Hammer and Menzel, 1998; Schwaerzel et al., 2003; Roitman et al., 2004; Wise, 2004). Since the vertebrate cortex and insect mushroom bodies are structures that have evolved independently (Farris, 2008) brain reward systems almost certainly evolved independently in these groups. But in both cases the evolutionary process may have made use of a common molecular toolkit, which included the biogenic amines as signaling molecules.

Conclusion

Even the simplest motile animals change their behavior in response to the perception of stimuli they need to survive or reproduce, and most animals display active reward-seeking behavior. This is a major organizer and driver of animal behavior (Tinbergen, 1951), and research with mammals has emphasized dopamine as a key neurochemical that modulates reinforcement, reward-seeking and reward learning (Wise and Rompre, 1989; Schultz et al., 1993; Berridge and Robinson, 1998; Schultz, 2007; Berridge et al., 2009).

Effects of biogenic amines, especially dopamine, on behavioral responses to reward have been reported across diverse animal phyla, but the reported functions of the biogenic amines do differ between groups. In nematodes, dopamine is a modulator of motor neurons. By changing locomotor behavior in response to food stimuli dopamine can trigger an elementary form of food searching behavior (Sawin et al., 2000; Hills, 2006; Rivard et al., 2010), and learned changes in locomotor behavior in response to food (Qin and Wheeler, 2007). In mollusks, dopamine not only modulates the motor neurons involved in feeding behavior, but also plays a role in reinforcement and reward learning (Lechner et al., 2000a,2000b; Brembs et al., 2002). In several different insects, octopamine has been shown to be necessary for reward learning (Hammer and Menzel, 1998; Schwaerzel et al., 2003; Unoki et al., 2005), but dopamine signals also affect reward responses (Kim et al., 2007; Krashes et al., 2009; Selcho et al., 2009).

While the link between the biogenic amines and reward responses is clearly strong across diverse phyla, it is unlikely that this indicates a true homology of brain reward systems. It is possible that an ancestral role for the biogenic amines as modulators of motor circuits in response to environmental stimuli meant that these neurochemical systems were predisposed to be adapted in the course of evolution for more specialized functions in reward-seeking behavior and reward learning as higher levels of brain complexity evolved.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.