Neural basis of reward and craving--a homeostatic point of view.

Here, it is argued that the interoceptive system, which provides information about the subject's internal state and is integrated in the insular cortex, and not the subcortical ventral striatum, is the critical neural substrate for reward-related processes. Understanding the internal state of the individual, which is processed via this system, makes it possible to develop new interventions that are aimed at treating reward-dysfunction disorders, ie, substance and alcohol dependence. Although the ventral striatum is important for signaling the degree to which rewarding stimuli are predicted to occur, this system alone cannot account for the complex affective, cognitive, and behavioral phenomena that occur when individuals come into contact with potentially rewarding stimuli. On the other hand, the interoceptive system is able to make connections between all cortical, subcortical, and limbic systems to orchestrate a complex set of responses. Craving and urges are among the most notable responses, and may have important functions to preserve homeostasis.

Overview

Reward is a complex construct that entails a feeling and an action. Components of reward include the hedonic aspects, ie, the degree to which a stimulus is associated with pleasure, and the incentive motivational aspects, ie, the degree to which a stimulus induces an action towards obtaining it.[1] Typically, the feeling is described as “pleasurable” or “positive” and the actions comprise behavior aimed at approaching the stimulus that is associated with reward. However, importantly, both feeling and action are highly dependent on the homeostatic state of the individual[2] That is, the degree to which a stimulus elicits a reward-consistent response depends in turn on the internal state of the subject. Therefore, to understand the neurobiology of reward, one needs to examine the neural substrates that process the feeling, and action associated with a stimulus as it relates to the internal state of the individual. As a consequence, treatments of disorders of reward systems need to be focused on modulating the interoceptive system and its underlying neural substrates, instead of altering the hedonic or incentive properties of the stimulus associated with the reward, or the underlying neural systems that process these associations. To this end, experiments will need to be conducted that examine how modulating the interoceptive state using C-fiber modulation will affect reward processing. This review provides an overview of the integration of the hedonic and incentive motivational view of reward with that of the homeostatic perspective of reward, and is focused on the neural substrates that underlie these processes.

The hedonic aspects of rewards-pleasure

The subjective experience of pleasure is at the heart of rewardrelated processing. This component of rewardrelated processing, ie, the hedonic or pleasurable component associated with the experience, is critical for understanding why individuals approach reward-related internal representations, external stimuli, or environments. Moreover, it is this complex set of features that is associated with the use of substances. Pleasure is fundamentally an experiential state, which combines a sensation as well as an emotion or feeling associated with it.[3] Thus, it is not surprising that visceral factors profoundly affect the hedonic impact and therefore directly alter the degree of relative desirability of different stimuli.[4] Fundamentally, the pleasurable state relates to changes in perceived body state that are likely processed via ascending slowconducting primary afférents.[2] As pointed out in ref 5, unmyelinated primary afferent fibers, designated as C-fibers when of cutaneous origin or as group IV when of muscular origin, have been traditionally linked to pain processing. More recently, however, the function of these fibers has been widely expanded to include a range of sensations such as pain,[6] temperature,[7] itch,[8] tickle,[9] sensual touch,[10,11] muscle tension,[5] air hunger,[12] stomach pH,[13] and intestinal tension,[14] which provide an integrated sense of the physiological condition of the entire body.[2] These afférents are processed in a distinct neural pathway that includes the lateral spinothalamic tract, midbrain homeostatic nuclei, the ventromedial thalamus, and the posterior insular cortex. Finally, these topographic and modality-specific organized pathways are integrated in the anterior insular cortex.[15] The anterior insular cortex in turn is integrally connected with subcortical,[16] limbic,[17] and executive control brain systems.[18] Within the anterior insular cortex, a multidimensional representation and integration of the current and possibly the predicted[19] body state provides the individual with a temporal representation of a “global moment in time” (Craig AD, personal communication). Importantly, this interoceptive network processes information in a homeostatic manner, ie, the valence of the information fundamentally depends on the nature of the individual's current state. For example, the same ternperature of an air-conditioned room is pleasantly experienced in the heat of the summer but is experienced aversively on a cold winter day It has been suggested that this network is fundamentally important for the generation of different feeling states,[2] and is closely linked to our overall awareness of ourselves.[20]

Based on this brief outline, it should be clear that the hedonic aspect of a stimulus is a property that emerges from the interplay between the stimulus characteristics and the individual state. Not surprisingly, the hedonic value of a stimulus is substantially influenced by its context. For example, in a decisionmaking situation, unexpected outcomes have greater hedonic impact than expected ones, and any given outcome is perceived as less pleasant if an unobtained outcome is perceived as being better.[21] That is, surprise, which strongly activates the ventral striatum,[22] and comparison with nonexperienced alternatives, contribute strongly to the experience of pleasure. Similarly, anticipation of pleasure has a profound influence on decisionmaking, and can explain why individuals make risky choices.[23] For example, people feel displeasure when the outcomes of selected actions fall short of the counterfactual alternative, and increased pleasure when their outcomes exceed the counterfactual alternative.[24] Moreover, predictions of future hedonic reactions result from a complex interplay between the current state of the individual and the changes that occur as the individual is getting closer in time to experiencing the stimulus. Specifically, initially the hedonic experience is based on the atemporal imagination of the stimulus, which is subsequently corrected with information about the time at which the event will actually occur.[25] The experience of the hedonic aspects of a rewarding stimulus itself has profound consequences of subsequent behaviors. In many instances individuals show deteriorating performance when they are anticipating the hedonic quality of a future experience.[26,27] Thus, to speak of the pleasurable property of a stimulus without referring to the contextual and individual state is to fundamentally misunderstand the way the brain processes hedonic aspects of reward.

Animal experiments have shown that an area within the medial caudal subregion of the nucleus accumbens shell, as well as rostral ventral pallidum, are necessary to process the hedonic reward properties of food.[28,29] Moreover, it appears that the ventral pallidum, an area adjacent to and connected with the insular cortex, [17] is a key structure in brain mesocorticolimbic reward circuits that mediate “liking” or hedonic reactions. Specifically, firing patterns of neurons within this structure selectively track the hedonic values of tastes, even across hedonic reversals caused by changing the homeostatic state of the animal[30] One possible way to examine the brain structures necessary to process the hedonic aspects of reward is to study individuals who are unable to experience pleasure due to an underlying psychiatric condition, ie, depressed subjects with profound anhedonia. In humans, neuroimaging investigations with depressed individuals have shown altered activation in midline cortical structures as well as putamen and thalamus that were directly related to the degree of anhedonia.[31] This was also found in another study, which showed that anhedonia was positively and negatively correlated with ventromedial prefrontal cortex and amygdala as well as ventral striatal activity.[32] Therefore, one top-down modulatory area, which is important for the assessment of hedonic valence is the midline cortical mantle, which includes medial prefrontal cortex as well as parts of the anterior cingulate, which has been referred to as limbic motor cortex.[2]

Examining other intrinsically hedonic stimuli and how these stimuli are processed in the brain provides a complementary approach to better understanding of the neural basis of hedonic processing. For example, food intake is an essential human activity regulated by homeostatic and hedonic systems. Recent neuroimaging experiments have identified that the orbitofrontal cortex is perhaps the strongest candidate for linking food and other kinds of reward to hedonic experience,[33] which has prompted some to suggest that this part of the brain may mediate the hedonic experience.[34] Similarly, cerebral blood flow changes during intensely pleasant emotional responses due to music have been observed in ventral striatum, midbrain, amygdala, orbitofrontal cortex, and ventral medial prefrontal cortex.[35] Others have suggested that cortical asymmetry contributes to the degree of hedonic experience. For example, greater left than right superior frontal activation was associated with higher levels of both forms of well-being. Appropriately engaging sources of appetitive motivation, characteristic of higher left than right baseline levels of prefrontal activation, may encourage the experience of well-being.[36]

Taken together, these observations make it clear that hedonic processing occurs on multiple levels in the brain and involves different brain structures that are important for contributing to stimulus-dependent, context-dependent, and homeostasis-related processing of the hedonic value. Common to these neural substrates that have been implicated in this process, ie, ventral pallidum, medial prefrontal and orbitofrontal cortex, is the fact that these brain areas are closely connected to the interoceptive systern as outlined above.

The incentive motivational aspects of reward-urge craving

Turning to the incentive motivational aspect of rewardrelated processing, it is important to also integrate these aspects within the homeostatic perspective. Surprisingly, there has been a burgeoning literature on bodily urges that has not been associated with the traditional drug addiction notion of incentive motivational processing, but can be linked easily, generating a broader perspective and enabling us to develop a neurologic formulation of drug addiction.

Urges can be conceived of as feeling states which are associated with strong incentive motivational properties to act, eg, pursue drug use. Some investigators have proposed that there may be two types of urge networks: (i) a “positive-affect” network, which is activated by appetitive stimuli, especially appetitive drug actions that activate “go” incentive motivational systems; and (ii) a “negative-affect” network, activated by aversive stimuli or consequences and by withdrawal and signals of withdrawal. The activation of this network is characterized by withdrawal symptoms and signs, negative affect, and drug-seeking.[37] Similarly, craving involves an intense feeling state associated with stimuli predictive of, or reminding the subject of, drugs. Nevertheless, the definition of craving is much less clear and is mostly described as an emotional-motivational state.[38] Thus, despite this wide use, there is little consensus on what craving means, the best way to measure it, or what mechanism accounts for the urge to use a drug. Some have proposed that there is no single model or theory of craving; this could account for the wide variation in experimental findings of craving-related phenomena.[39] Other investigators have identified several craving-related dimensions, which include specificity, strength, positive outcomes, behavioral intention, thoughts, physical symptoms, affect, and cues.[40] Taken together, cravings and urges are important but complex components of the incentive motivational aspect of reward processing, and are often targets of clinical interventions for individuals with substance use and dependence.

Several cognitive models have been put forth to explain the concept of craving. These include cognitive labeling, outcome expectancy, dual-affect, and cognitive processing conceptualizations.[41] Another way to conceptualize these states is to view them as metacognitions, ie, statements about other cognitions. Therefore, an individual who craves is experiencing a cognitive event, eg, a thought or feeling that is aversive or unpleasant,[42] which in turn creates an increased state of awareness about this event. The degree of self-reported urge for drugs has important implications for abstinence. Relapse to drug use has been closely linked to exposure to conditioned stimuli that frequently induce craving, and a wide variety of such stimuli, many of which were unique to individuals, have been reported.[43] Specifically, those individuals who report losing urges had significant higher abstinence rates than those reporting still having the urge to use.[44]

Others found a significant relationship between craving and total proportion of cocaine-positive urines.[45] Similarly, craving has emerged as a predictive factor for continued use in methamphetamine-dependent individuals.[46] However, some investigators have called into question that subjective cravings are invariably associated with drug use.[41] Moreover, there is even some evidence that cravings may actually protect some drinkers against further drinking.[47] This has led some to question the assumption that craving is the underlying basis of addiction and represents the most appropriate target for treatment.[48] Therefore, one cannot take craving in isolation, but has to consider the phenomenon of urge and craving as part of a homeostatic system, which aims to maintain an individual at some steady state-level.

Thus, urges do not occur in isolation, but are immediately incorporated into an existing homeostatic cognitive and affective system of the individual. For example, self-efficacy, ie, the confidence in being able to resist the urge, can profoundly modulate drug use behavior.[49] Moreover, temptation, ie, the contextual characteristics that are aimed to increase desire, leads to stronger urges to drink alcohol, greater difficulty controlling urges, and increased alcohol consumption, even when controlling for alcohol consumption in the past month.[50] Finally, social stress frequently occurs before, and may contribute to the degree of, cravings.[51] Substance -using individuals who perceive an opportunity to consume their drug of choice report higher urges than those who do not anticipate being able to use the drug.[52] It has been argued that the degree of urge modulates the threshold for triggering an action.[53]

Therefore, craving and urges are important component processes of decision-making in the presence of ambivalence or conflict.[54] Thus, similarly to the hedonic properties of a reward processing, the incentive motivational aspects are an emerging property based on the stimulus characteristics and the individuals homeostatic state.

One way to study the neural substrates underlying urges is to examine frequently observed behaviors that are often attributed to urge-related processing. Here, four examples of urge-related behaviors are reviewed that can shed new light on the neurobiology of these metacognitive states. First, in a functional positron emission tomography (PET) study to investigate the neural substrates underlying itch and the motor intention of the urge to scratch, investigators found activation of the anterior cingulate cortex, supplementary motor area, premotor area, and inferior parietal lobule.[55] Others have observed that increases in regional cerebral blood flow in orbitofrontal cortex, neostriatum, global pallidus, and thalamus were related to urges to perform compulsive movements.[56] A functional magnetic resonance imaging (fMRI) study of intense itch and urge to scratch showed significant activity in the genual anterior cingulate, striatum, and thalamus as well as orbitofrontal, supplementary motor, posterior parietal areas, and bilateral insula.[57]

Second, air hunger, ie, the uncomfortable urge to breathe, is another urge-related phenomenon, which can be used to study the neural systems underlying urge and craving. Several neuroimaging studies have found activation of limbic and paralimbic regions during air hunger, which are often found to modulate homeostatic imbalance such as pain, thirst, and hunger for food. A recent fMRI study found that anterior cingulate, operculum, cerebellum, amygdala, thalamus, and basal ganglia were activated during air hunger. Most of all, there was a consistent activation of anterior insular cortex, which suggests that this structure acts within a network of limbic and paralimbic neural substrates to mediate urges.[58] Third, the urge to void is a frequently experienced behavioral state, and generally increases with bladder distention in a complex manner. For example, at moderate bladder filling, urge to void appears to be under cognitive control and leads to a fluctuation of the conscious urge sensation. A recent fMRI study found significant brain activity associated with an increased urge to void in the insular cortex, frontal opercula, supplementary motor area, cingulate motor area, right posterior parietal cortex, left prefrontal cortex, and cerebellum.[59] Fourth, anorectal continence is another urge-driven behavior that is under complex cerebral control A recent neuroimaging study showed that subjective sensation of discomfort increased during repeated rectal distension was associated with activation in the anterior cingulate gyrus, insula, thalamus, and secondary somatosensory cortex. Moreover, voluntary contraction of the anal sphincter in response to anal distention was associated with activation of motor cortex and increased activity in supplementary motor as well as insular cortex.[60] Thus, these neuroimaging studies have in common the involvement of the interoceptive system in the expression of diverse urge-related behaviors.

Imagery-based techniques are frequently used to elicit memory of drug-related craving experiences,[61] and some have even argued that stress imagery testing procedures may function as provocative tests for stress-induced drug craving.[62] Several brain systems have been implicated in modulating the degree of drug-induced cravings. For example, the degree of drug-related craving by means of administration of presentation of conditioned stimuli has been related to activity in striatum,[63] thalamus,[64] anterior cingulate,[65] inferior frontal cortex,[66,67] and orbitofrontal cortex,[68,70] but also with insula,[71,72] amygdala,[73] and cerebellum.[74] For example, when viewing videos that display cocaine-related stimuli users experience craving, which is associated with increases in amygdala and anterior cingulate cerebral blood flow relative to their responses to a nondrug video.[75] Similarly, imagery-induced drug craving has been associated with bilateral activation of amygdala, insula, and anterior cingulate gyrus as well as the nucleus accumbens area.[76] In alcohol-dependent individuals, cue-induced craving has been associated with activation in amygdala and hippocampal area as well as the cerebellum,[77] but also visual and other limbic areas.[78] Smoking-induced craving was associated with increased activation in left inferior frontal gyrus, left ventral anterior cingulate, and bilateral middle frontal gyrus.[79] Using fMRI, Garavan and colleagues[80] identified regions involved in craving that showed substance-user specificity as well as content specificity in medial and middle frontal gyri, bilateral inferior frontal gyrus, bilateral inferior parietal lobule, insula, and anterior as well as posterior cingulate gyrus. The neural substrates are not limited to drug-induced cravings. For example, food craving-related changes in fMRI studies have been identified in hippocampus, insula, and caudate.[81] However, there may be some gender differences with respect to the degree to which these areas are recruited during craving experiences.[82] For example, female subjects show more activation than males in the anterior cingulate and posterior cingulate cortices, related to craving.[83]

The four examples of physiological urges described above, and the vast literature on drug- or alcohol-induced craving, clearly point toward a core neural system, which overlaps significantly with the interoceptive system. In particular, the anterior cingulate (limbic motor cortex) and the anterior insula (limbic sensory cortex) are key neural substrates modulating the urge and cravingrelated aspects of reward. First, the anterior cingulate cortex forms a large region around the rostrum of the corpus callosum that is termed the anterior executive region.[84,85] This brain structure is part of what has been called the limbic motor cortex.[86] The affect division of anterior cingulate cortex modulates autonomic activity and internal emotional responses, while the cognition division is engaged in response selection associated with skeletomotor activity and responses to noxious stimuli.[87]

Thus, the anterior cingulate cortex plays a crucial role in linking the hedonic experience to the incentive motivational components of reward.[88] This area has been shown to be activated in addicted subjects during intoxication, craving, and bingeing, and they are deactivated during withdrawal (for review see ref 89). Some investigators have proposed that cue-induced activation of the anterior cingulate may play a role in the attribution of incentive salience to alcohol-associated stimuli.[90]

Second, the insula (for review see refs 91,92) is one of the paralimbic structures and constitutes the invaginated portion of the cerebral cortex, forming the base of the sylvian fissure. The insular cortex has been considered to be limbic sensory cortex by some investigators.[86] A central insular sulcus divides the insula into two portions, the anterior and posterior insula. The anterior insula is composed of three principal short insular gyri (anterior, middle, and posterior) as well as the accessory and transverse insular gyri. All five gyri converge at the insular apex. The posterior insula is composed of the anterior and posterior long insular gyri and the postcentral insular sulcus, which separates them. The anterior insula is strongly connected to different parts of the frontal lobe, whereas the posterior insula is connected to both the parietal and temporal lobes.[93] The columnar organization of the insular cortex shows a highly organized anterior inferior to posterior superior gradient (for example see ref 94). Specifically, whereas posterior insula is characterized by a granular cortical architecture, the anterior inferior insula has an agranular columnar organization, ie, lacks layer 4 granular cells. This type of transition is found in other parts of the brain whenever cortical rerepresentations are based on modulatory or selective feedback circuits.[95] Finally, the discovery of spindle cells within the anterior insular-orbitofrontal transition region[96] has provided a cellular substrate underlying the possibility of widespread cortical integration. The insular cortex has been implicated in a wide variety of processes, which includes pain,[97] interoceptive,[20] emotion-related,[98] cognitive,[99] and social processes.[100] A recent study with brain-lesioned individuals showed that those who had insular damage were more likely to experience a disruption of cigarette addiction, including abolition of the urge to smoke.[101] Relevant to reward-related processes, the insular cortex is important for subjective feeling states and interoceptive awareness,[2,20] and has been identified as taking part in inhibitory processing, together with the middle and inferior frontal gyri, frontal limbic areas, and the inferior parietal lobe.[102]

Given the fact that this area receives integrated input from ascending primary afférents and is closely connected to all parts of the cortical mantle and limbic motor cortex, it is obvious that the insula is ideally suited to orchestrate craving-related processing. For a conceptual summary, see . Although it is not clear at this point whether this is primarily related to the sensation of urge or the motivational component associated with it, the close connection between this structure and the anterior cingulate suggests that it may be the integrity of both that is needed to modulate urge-related behaviors.

Conclusions

Reward-related processing is an important aspect of understanding drug addiction. Nevertheless, surprisingly little insight has been gained into how pleasure and urge are integrated in the brain and how this process is modulated as part of the homeostatic dynamic state of the individual. It has been suggested that, from an evolutionary perspective, drugs that affect the hedonic systems can have profoundly adverse consequences because they bypass adaptive information processing systems and act directly on ancient brain mechanisms that control emotion and behavior.[103] For example, drugs that induce positive emotions give a false signal of a fitness benefit. In comparison, drugs that block negative emotions can impair useful defenses. Koob and LeMoal have argued that sensitization and counteradaptation processes contribute to hedonic homeostatic dysregulation in substance-dependent individuals,[104] and that prolonged exposure to drug stimuli changes the hedonic setpoint[105] In comparison, others have argued that addictive drugs produce long-lasting adaptations in those neural systems, which are involved in the process of incentive motivation and reward such that these brain systems are hypersensitive to drugs and drug-associated stimuli, primarily to the subcomponent of reward termed incentive salience (drug “wanting”) but not to the pleasurable effects of drugs (drug “liking”).[106] By focusing on the underlying neural substrates, ie, the insular cortex as the limbic sensory cortex and the anterior cingulate as the limbic motor cortex, and its afferent inputs from ascending primary afférents, as well as the top-down modulation via different cortical areas, one can begin to delineate how one can devise novel interventions for drug addiction. Moreover, the homeostatic viewpoint also helps to understand why there is an enormous behavioral and neural substrate activation intra- and inter-subject variability when processing rewards. Finally, a key step in moving our understanding of reward-related processing forward will be to delineate the conditions under which limbic sensory processing (the experience of pleasure) can be decoupled from the limbic motor processing (the urge or craving for a pleasurable experience).