A transient dopamine signal encodes subjective value and causally influences demand in an economic context.

The mesolimbic dopamine system is strongly implicated in motivational processes. Currently accepted theories suggest that transient mesolimbic dopamine release events energize reward seeking and encode reward value. During the pursuit of reward, critical associations are formed between the reward and cues that predict its availability. Conditioned by these experiences, dopamine neurons begin to fire upon the earliest presentation of a cue, and again at the receipt of reward. The resulting dopamine concentration scales proportionally to the value of the reward. In this study, we used a behavioral economics approach to quantify how transient dopamine release events scale with price and causally alter price sensitivity. We presented sucrose to rats across a range of prices and modeled the resulting demand curves to estimate price sensitivity. Using fast-scan cyclic voltammetry, we determined that the concentration of accumbal dopamine time-locked to cue presentation decreased with price. These data confirm and extend the notion that dopamine release events originating in the ventral tegmental area encode subjective value. Using optogenetics to augment dopamine concentration, we found that enhancing dopamine release at cue made demand more sensitive to price and decreased dopamine concentration at reward delivery. From these observations, we infer that value is decreased because of a negative reward prediction error (i.e., the animal receives less than expected). Conversely, enhancing dopamine at reward made demand less sensitive to price. We attribute this finding to a positive reward prediction error, whereby the animal perceives they received a better value than anticipated.

Our understanding of the role dopamine plays in the motivation to act has evolved over several decades. It was first demonstrated that neurotoxic lesions of mesolimbic dopamine fibers (1) and pharmacological antagonism of dopamine receptors (2) impair reward-seeking actions without disrupting general motor activity (3). Then, in vivo electrophysiological recordings demonstrated that bursts of dopamine neural activity occur when animals are presented with an unexpected reward or a reward predictive stimulus, but are suppressed when the reward is withheld (4). This observation led to the development of the reward prediction error theory, which suggests that a transient dopamine signal encodes the reward prediction signaled by the cue. Electrochemical studies generally confirmed this theory by demonstrating that transient accumbal dopamine release events occur when animals are presented with rewards and their conditioned predictors (i.e., cues), but suppressed during reward omission (5, 6). More recently, optogenetic manipulations, which can be used to assess the causal relationship between patterns of neural activity and behavior, were used to demonstrate that augmenting dopamine release at reward delivery accelerates reward learning (7).

At present, these observations are being reconsidered within the context of economic theory. It has been proposed that the transient dopamine signal represents subjective value (8). This revision is supported by electrophysiological and electrochemical studies demonstrating that increases in reward magnitude augment both the phasic activation of midbrain dopamine neurons and transient accumbal dopamine concentrations (6, 8). Similar results have been reported when price is modified by manipulating effort (6), but they remain controversial (9). Furthermore, a subpopulation of accumbal neurons that receives dopaminergic input has been shown to represent effort-based costs (10). Recent optogenetics experiments have also confirmed a causal role for dopamine in valuation by demonstrating that transient dopamine manipulations alter the willingness to work for a reward (11).

In this study, we use a behavioral economics approach to investigate the relationship between dopamine and price. Behavioral economics has historically been used in the fields of behavioral analysis (12) and psychopharmacology (13, 14), and more recently to study the relationship between dopamine and valuation (8, 15–18). An elegant body of electrophysiology studies demonstrated that dopamine neurons respond to gambles and outcomes to guide economic decision making (19), and are able to integrate various factors that underlie value representations to influence economic choices (20). As various factors can contribute to subjective value, including risk (20), satiety (21, 22), and delay (23, 24), we started by focusing on the most obvious and easiest to measure: the unit price of a commodity—generally defined as the response requirement per unit reward (13). In addition to characterizing the relationship between price and dopamine concentration, our study builds upon the existing literature by assessing how optical manipulation of dopamine neurons causally influence price sensitivity using demand curves.

Price sensitivity can be experimentally determined using demand curves that plot the relationship between price and consumption (14). Typically, consumption is inversely related to price, resulting in demand curves with a negative gradient. The rate at which the demand curve decays is a measure of price sensitivity or, in economic terms, the elasticity of demand (14). If an animal’s demand for a commodity becomes more sensitive to price, the demand curve would decay at a faster rate. From this, we would conclude that the value the animal places on the commodity is diminished. If the revised value-based theory of reward prediction error is correct, then the transient dopamine response should be sensitive to price and modulating dopamine release should alter the rate at which demand curves decay.

We predicted that dopamine would scale in an inversely proportional manner to unit price, irrespective of the order in which unit prices are presented. We further predicted that augmenting dopamine release at cue would make demand more sensitive to price because of a negative reward prediction (i.e., the animal receives a worse value than expected). Conversely, we predicted that augmenting dopamine release at reward delivery would sustain demand at higher prices because of a positive reward prediction (i.e., the animal receives a better value than expected).

Results

Introduction to Behavioral Economics Task.

To investigate the role of dopamine in valuation, we employed a sucrose-seeking task designed for behavioral economic analysis. Access to sucrose was provided across 10 unit prices (defined as lever presses required per mg of sucrose) within a single session, with each price presented for a fixed epoch of time. A predictive cue light placed above the lever signaled sucrose availability and then dimmed as the lever retracted at sucrose delivery. Within a session, we altered unit price by either controlling the cost, that is, the number of lever presses required for a sucrose pellet, or by changing the reward, that is, the amount of sucrose delivered in response to a lever press. We refer to these two versions of the task as a cost-manipulation task and a reward-manipulation task, respectively (or simply “cost task” and “reward task”). These manipulations, particularly the cost task, also introduced an opportunity cost due to the delay between cue and reward delivery. We address this in detail in the next section.

As illustrated in , in the cost task, the response requirement to receive a 45-mg sucrose pellet increased across epochs that are 10 min long. By contrast, in the reward task, the response requirement remained fixed at 1 while the volume of a sucrose solution (300 mg/mL) delivered per response decreased across 5-min-long epochs (; see for additional details). For consistency, we chose a set of identical unit prices between the two tasks (rightmost columns of ). Each task elicited a similar pattern of behavior: the animal increases response output across epochs until a maximal price is reached, at which point the animal ceases sucrose consumption.

Delay Contributes to the Concept of Cost.

It is well accepted that temporally restricting reward alters dopaminergic representations of value (9, 23–25). Specifically, when cost is increased by requiring more lever presses, this also introduces a delay between cue presentation and reward delivery. Thus, we performed additional analyses to assess how delay confounds our original conception of cost. We found that mean elapsed time between cue presentation and reward delivery increased across the first eight price points, with delay being more evident in the cost task: 0.896, 1.977, 3.143, 8.173, 16.55, 43.77, 89.09, and 117.96 s (), compared with the reward task: 0.753, 1.918, 2.39, 4.873, 5.742, 6.078, 8.02, and 7.03 s ().

In light of these observations, we conceptually redefine cost as a combination of effort cost (the number of lever presses required) and an opportunity cost due to delay. Thus, while data were primarily analyzed in terms of effort cost, it is important to note that delay adds to the overall notion of cost.

Additional Considerations: Randomized Price Presentation, Satiety, and Pump Reliability.

We presented unit prices in a consistent order to avoid premature extinction of responding and to establish a range of equivalently spaced price points for fitting the data. Randomly presenting unit prices produced erratic cumulative response records (), increased within-subject variance in the rate of demand curve decay (), and resulted in less reliable curve fitting (). Satiety greatly influences dopamine value signals (21, 22) and the rate at which sucrose demand curves decay (); thus, we further compared sucrose consumption between animals responding to either increasing price within a session or a fixed low price (1 response/45 mg sugar pellet; unit price of 0.022) throughout the session. We determined that rats remain under their satiety threshold in the behavioral economics task, as they consume significantly more sucrose (approximately three times as much) when access is provided at the lowest unit price (0.022) for the entire session (). In the reward-manipulation task, it is important to assess pump reliability because price is controlled by small changes in pump speed. Thus, we confirmed that pumps delivered the proper amount of sucrose in the reward-manipulation task by comparing the volume delivered in each epoch with the volume predicted ().

Dopamine Concentration Is Inversely Related to Price.

To assess for changes in nucleus accumbens (NAcc) dopamine concentration, we conducted fast-scan cyclic voltammetry (FSCV) during the behavioral economics task. The concentration of accumbal core dopamine time-locked to cue presentation and sucrose delivery significantly decreased across the first five prices (Fig. 1 and ). Only the first five price points were included because 100% of animals used for the electrochemical study maintained consumption within this range. We focused on the core region of the NAcc because of previous evidence showing that the core is critically involved in modulating cue-driven behavioral responses (26). We found that dopamine concentration was inversely related to price in both the cost-manipulation task (Fig. 1) and the reward-manipulation task (Fig. 1). Dopamine concentration data were bucketed into 10- and 5-min bins for the cost and reward tasks, respectively.

Fig. 1.

Accumbal dopamine concentration decreases with price. (A and C) Dopamine concentration (mean ± SEM) decreases across the first five prices in the cost-manipulation (A) and reward-manipulation (C) versions of the task (also see ). (E) Dopamine concentration also decreases across the first five price points if they are presented in a descending order (). Similar trends are observed at cue (square) and reward (circle). (A, C, and E) Inset triangles show the direction of price presentation. Dopamine concentration data are bucketed into either 10-min (cost task) or 5-min (reward task) bins. These data are also illustrated after being bucketed into 1-min bins in . Binning data over longer periods of time reduces the SEM. (B, D, and F) Dopamine concentration traces from a single representative animal (Top) and corresponding color plots (Bottom) show the effects of price on cue- and reward-evoked dopamine release in both the cost-manipulation (B) and reward-manipulation (D) tasks, when prices are presented in ascending order. Similar effects are observed when prices are reversed (F). Average color plots topographically depict the voltammetric data with time on the x axis, applied scan potential (Eapp) on the y axis, and background-subtracted faradaic current shown on the z axis in pseudocolor. Dopamine can be identified by an oxidation peak (green) at +0.6 V and a smaller reduction peak (yellow) at −0.2 V. Resp, response.

We also assessed how dopamine responds to price within each epoch by measuring the dopamine concentration on a minute-by-minute basis (). We found that concentration decreases within each epoch, consistent with a rapid, adaptive revaluation process (27). Here, data are bucketed into 1-min bins.

The Inverse Relationship Between Dopamine and Price Is Not an Order Effect.

To test whether the inverse relationship between dopamine and price is an artifact of presenting prices in ascending order, we provided rats with access to sucrose across the first five unit prices of the cost task, but started at the fifth highest unit price and then decreased prices in a descending order (Fig. 1). Even when prices were presented in a descending order, we observed an inverse relationship between dopamine concentration and price (Fig. 1 and ). Data were again bucketed into 10-min bins, as this duration corresponded to the epoch of time each price was presented in the cost task.

Optogenetics Data Reveal Dopamine Manipulations Influence Price Sensitivity.

To assess the causal influence of dopamine on price sensitivity, we optically augmented release during the task. We selectively activated (10 pulses at 20 Hz, 0.5-s duration) channelrhodopsin-2 (ChR2)-expressing dopamine neurons within the ventral tegmentum [ventral tegmental area (VTA)] of Th-Cre+/− rats during either cue or sucrose presentation using a counter balanced design (). Food-restricted rats (90% age-adjusted body weight), initially trained to respond for sucrose under a fixed ratio 1 (FR1) schedule, received seven training sessions in the behavioral economics task before the onset of optogenetic stimulation. All animals were tested in both the cost-manipulation and reward-manipulation tasks. Within each version, an animal first received 3 baseline days (tethered with an inactive patch cable), 3 d with optical stimulation at cue (or sucrose delivery), 3 additional baseline days, and 3 d with optical stimulation at the complimentary event. Three groups were tested with optical stimulation: (i) a wild-type (WT) group, (ii) a VTA stimulation group, and (iii) a NAcc stimulation group. Representative cumulative response records, response–price curves, and corresponding demand curves from a single animal depict the resulting patterns of behavior and consumption across all stimulation conditions in both tasks (Fig. 2).

Fig. 2.

Optogenetic stimulation alters price sensitivity in a representative rat. (A and B) Cumulative records from one animal responding in the cost-manipulation task (Left) and reward-manipulation task (Right). Colors denoting stimulation condition are shown in the legend. (C and D) The same data are replotted to show responses as a function of unit price. (E and F) Demand curves show consumption as a function of unit price. Demand curves are fitted to the data to estimate α. α measures price sensitivity; a high α values indicate consumption decays rapidly with price. Resp, response.

Mathematical Modeling of Demand Curves to Estimate Price Sensitivity.

To quantify changes in the elasticity of demand (i.e., price sensitivity), we then mathematically fitted individual demand curves to a single-exponential decay model:where is the consumption at given unit price . The model is parametrized by , the maximal consumption (at zero price); , the asymptotic minimum consumption; and , the rate of decay. The rate of decay, α, measures the sensitivity of consumption to price. When demand is more sensitive to price, in other words more elastic, the demand curve decays at a faster rate, leading to a higher estimate of (e.g., see orange demand curves in Fig. 2 ). From this, we infer that the subjective value of the commodity is diminished. In contrast, when demand persists at higher prices, that is, is less elastic, the estimated value of is lower (Fig. 2 , purple demand curves). We interpret this as a manifestation of a greater subjective value of the commodity.

Optogenetically Augmenting Dopamine Release Alters Price Sensitivity.

Applying the single-exponential decay model to our data revealed the effects of VTA optical stimulation on price sensitivity. First, we found that this model provides excellent fits to the observed demand profiles with a median R2 value of 0.976 (). We observed that, in individuals expressing ChR2, laser stimulation at the cue increased the mean α, while stimulation at reward decreased it (Fig. 3 ). We also observed modest changes in Qmax with relative magnitudes that were less than 10% (Fig. 3). These results indicate that the primary effect of optical stimulation is to alter the elasticity of demand (α) rather than the maximal consumption (Qmax). Because of the observed changes in dopamine within the NAcc (Fig. 1), we further tested whether stimulating terminals within this region produced comparable changes in price sensitivity. We found similar trends as above in cost-manipulation task, but a weaker effect in reward-manipulation task (Fig. 3).

Fig. 3.

Optogenetically augmenting dopamine release alters mean price sensitivity. (A) Mean demand profiles and their fits to the exponential decay model []. Each panel shows the demand curves for a specific laser treatment in bold color vs. baseline values in faded color (Middle Top). The three vertical columns correspond to WT (Left), VTA Cre stimulation (Middle), and NAcc Cre stimulation (Right). The two horizontal columns correspond to the cost-manipulation (Top) and reward-manipulation (Bottom) tasks. (B and C) Summary of estimated α (B) and Qmax (C) from individual fits (mean ± SEM for independent replicates from a given day). The three values shown for each treatment are chronologically ordered from Left to Right. Resp, response.

Bayesian Analysis of Demand Profiles.

To resolve our effects in the face of experimental noise, we performed an additional Bayesian analysis of demand profiles. Fitting each demand curve independent of the others reveals the high variability in individual responses that is typical of behavioral studies (); however, the consistency of stimulation effects across sessions should be noted (Fig. 3). Furthermore, this analysis does not take into account the expected correlation between responses from the same individual on different days, or between different individuals that share a common genetic background. A traditional analysis in such situations would use ANOVA on the parameter estimates. Using a standard ANOVA revealed significant effects of optical stimulation on α in Th-Cre rats under all conditions except during bilateral accumbal stimulation in the reward-manipulation task (P < 0.01 vs. baseline; ). However, it is difficult to justify the assumptions of ANOVA for parameters estimated from nonlinear regression. We therefore used a Bayesian estimation procedure to model the demand profiles (28).

The observed variability in response arises due to multiple factors, such as the different treatments, measurement errors, and individual differences between animals. A practical goal of our analysis was to quantify variability in α due to the different optogenetic manipulations alone. We therefore constructed a multilevel model in which the parameter α explicitly depends on the treatment (29). The observed response is viewed as the model prediction plus some normally distributed error. The error term captures variability at the individual and replicate level. We used a Bayesian inference procedure to compute posterior probability distributions of model parameters that best explain the observed responses (see for details).

The estimated posterior distributions of α are shown in Fig. 4 (). We observed that, in WT animals, different optogenetic treatments do not noticeably alter α. In contrast, in animals expressing ChR2, laser stimulation at cue onset shifted α to higher values, while stimulation at reward delivery shifted α to lower values. These trends are consistent with the results of the nonlinear regression analysis (Fig. 3), but the posterior distributions of α from the Bayesian analysis are better resolved.

Fig. 4.

Bayesian analysis of price sensitivity. (A) Posterior probability density of α (metric of price sensitivity) for each treatment (α[trt] in the multilevel model; see for details and statistics in ). (B) Maximum a posteriori (map) estimates and 95% credible intervals (ci) for α derived from the posterior distributions shown in A (). The map estimate is the value of α at which the posterior distribution is peaked, that is, the mode. The 95% ci is the highest posterior density interval that contains 95% of the probability mass under each posterior. (C) The effect size of a treatment on α, defined as the relative change over the baseline α. The map estimate and a 95% ci are shown for each of the two laser stimulations (). For this analysis, we combined the two baseline distributions of α to generate a single baseline distribution. (D) Map estimates and a 95% ci for Qmax from the multilevel analysis. (E) Predicted normalized consumption and a 95% ci derived from the posterior distributions of α shown in A and B. The normalized consumption is defined as and it scales consumption between 1 (when Q = Qmax) and 0 (when Q = Q). This allows us to isolate the effect of changing α alone, by rescaling all demand curves to a common scale irrespective of variations in Qmax and Qmin (). Resp, response.

Estimating the Effect Size of Optogenetic Manipulations on Price Sensitivity.

To quantify the magnitude of change in α, we summarized the posterior distributions using maximum a posteriori (map) estimates and 95% credible intervals (Fig. 4 and ). We also calculated the effect size for the two laser stimulations, defined as the ratio of treatment α to baseline α (Fig. 4 and ). We found that VTA stimulation at cue increased α by ∼50%, while stimulation at reward decreased it by roughly 50%, in both the cost-manipulation and reward-manipulation tasks. Bilateral stimulation at NAcc recapitulated these trends for the cost-manipulation task, but the effects were much weaker in the reward-manipulation task. We also found that the posterior distributions of Qmax and Qmin were minimally altered between the different conditions (Fig. 4 and ; also ).

Bayesian Prediction of Demand Profiles.

An attractive feature of the Bayesian framework is the ability to generate model predictions. We used the posterior distributions of α to predict a normalized consumption, which is a rescaled value of consumption between 1 and 0 (Fig. 4). These predictions show how the demand curves would shift in response to the different optogenetic treatments relative to baseline, in the absence of any individual variability and experimental noise.

Manipulating Cost or Reward Leads to Consistent Changes in Price Sensitivity.

We also examined the joint posterior distributions of α in cost-manipulation vs. reward-manipulation tasks to determine whether the induced dopamine release elicits the same trends in these two tasks (Fig. 5). We found that the shifts in α were consistent (i.e., α increased in both tasks, or decreased in both tasks for a given treatment). Once again, the effects were stronger in the VTA stimulations but more modest in NAcc stimulations.

Fig. 5.

The joint posterior density of α from cost-manipulation and reward-manipulation tasks for a given treatment. The treatments are color-coded as in other figures. Moving outward from the center, the contour lines mark the 99th, 95th, 90th, 75th, and 50th percentile values of the posterior probability density.

Dopamine Assessments During Optogenetic-Induced Modification of Demand.

Next, we combined in vivo electrochemistry and optogenetics to characterize how dopamine concentrations change during optical stimulation. To assess the relationship between price and dopamine release, we activated ChR2-expressing neurons in the VTA at cue presentation, while simultaneously measuring changes in dopamine concentration in the NAcc using FSCV. The concentration of dopamine at cue presentation remained unchanged across the first five price points during optical stimulation (Fig. 6; not significant), whereas the concentration of dopamine at reward delivery significantly decreased across price (Fig. 6 and ). We next compared these dopamine concentration values to those obtained from animals in our original electrochemistry experiment (Fig. 1). We found that dopamine concentration following optical stimulation was significantly higher at cue presentation at price points 2–5 in both tasks, and significantly lower at reward delivery at price points 2–4 and 1–3 in cost-manipulation and reward-manipulation tasks, respectively (Fig. 7 and ). To determine whether dopamine concentration was similar between the cost- and reward-manipulation tasks, we further compared mean dopamine concentration at cue and reward delivery under all conditions. We predicted identical dopamine concentration values between the tasks, as prices were matched between the two (). As predicted, dopamine concentration at both cue and reward was statistically similar between the cost-manipulation and reward-manipulation tasks (Fig. 7 ). Across all conditions, dopamine concentration was higher at cue presentation than reward delivery.

Fig. 6.

Optical stimulation at cue presentation increases dopamine concentration at the cue but decreases dopamine at sucrose delivery. (A–D) Dopamine at cue (orange squares) remains unchanged across price; dopamine at reward (purple circle) declines across price in both the cost-manipulation (A and B) and reward-manipulation (C and D) tasks (). See Fig. 1 for description of representative voltammetric plots. Resp, response.

Fig. 7.

Optogenetic stimulation at cue increases dopamine release at cue but decreases dopamine release at reward delivery. Compared with baseline animals (Fig. 1), stimulating dopamine release at cue presentation increased dopamine at cue (A and B) but decreased dopamine at reward delivery (C and D). The two vertical columns correspond to the cost-manipulation (Left) and reward-manipulation (Right) tasks. The two horizontal columns correspond to the cue (A and B) and reward (C and D) dopamine observations (). Orange circles represent stimulation at cue, and inverted red triangles represent no stimulation at cue; inverted gray triangles represent stimulation at reward, and purple circles represent no stimulation at reward. (E and F) Mean dopamine concentration for all stimulation conditions in all tasks. Mean dopamine concentration (average of all observations across all price points) at cue and reward are statistically similar between the cost-manipulation and reward-manipulation tasks (). In all conditions, higher dopamine concentrations were observed at cue vs. reward. Resp, response. *P < 0.05.

Optical Stimulation Reduces Response Latency Across All Conditions.

Finally, we analyzed changes in response latency across all conditions. Only latencies occurring in the first three price points were included in our analysis because 100% of rats maintained sucrose consumption across this range—even those receiving optical stimulation at cue presentation. As illustrated in Fig. 8, optical stimulation significantly reduced latencies across all conditions (statistics in ).

Fig. 8.

Response latencies are reduced across all stimulation conditions. (A and B) Response latency (mean ± SEM) across all VTA stimulation conditions. (C and D) Response latency (mean ± SEM) across all NAcc stimulation conditions (). Resp, response. *P < 0.05.

Task-Specific Optical Stimulation Parameters Failed to Alter Horizontal Activity.

To assess the effect of optical stimulation VTA dopamine neurons on general locomotor activity, we assessed changes in horizontal movement in an open field while applying stimulation every 30 s—which corresponds to the maximal possible rate of stimulation in cost-manipulation task. Optical stimulation failed to alter horizontal activity compared with baseline values ().

Discussion

In the present study, we examined the role of dopamine in value assessments using a combination of behavioral economic theory, in vivo electrochemistry, optogenetics, and modeling. Two tasks were used: a cost-manipulation task and a reward-manipulation task. We found that dopamine concentration decreased as price increased in both behavioral economics-based tasks (Fig. 1). A divisive ratio unit price model (response requirement/milligrams of sucrose) was used as opposed to a subtractive model (response requirement – milligrams of sucrose) because the ratio price model provides the definition of unit price (cost/unit of good). The inverse relationship between dopamine concentration and price was observed regardless of the order of sequential price presentation. These data confirm and extend the notion that dopamine release events originating in the VTA encode subjective value (8). Our optogenetics data revealed that augmenting release at cue presentation increased sensitivity to price and decreased dopamine release at reward delivery. In contrast, augmenting release at reward delivery reduced sensitivity to price. Therefore, we further conclude that transient dopamine release events originating in the VTA causally modify valuation.

The mesocorticolimbic system projects to various terminal fields including the orbitofrontal cortex, dorsal striatum, NAcc, and olfactory tubercle—all of which have been implicated in either valuation or motivation (30–35). As we observed accumbal dopamine concentration decreases with price, we also investigated the effects of bilaterally stimulating terminal release in the core region of the NAcc. We observed trends that were consistent with VTA stimulation, but the accumbal stimulation effects were weaker in the reward-manipulation task—confirming a recent study by Saddoris et al. (36). These data suggest that the NAcc is involved in valuation, but likely not exclusively.

The current study builds upon a growing body of literature that is reconsidering the influential theory of reward prediction error in the context of economic utility. Reward prediction error theory suggests that a transient dopamine signal encodes the discrepancy between a reward and its predictive cue (4, 8). A series of choice studies demonstrate that transient dopamine release events encode value as positive or negative reward prediction errors (6, 9, 37). The data presented herein generally support the basis of reward prediction error and its role in economic utility. Dopamine concentration at cue presentation and reward delivery decreased as price increased. Furthermore, increasing dopamine release at cue presentation rendered animals more sensitive to price and decreased dopamine concentration at reward delivery, consistent with a negative reward prediction error. In this case, we infer that the subjective value of sucrose is decreased because the animal perceives that they received less than expected. Conversely, augmenting release at reward presentation rendered animals less sensitive to price, consistent with a positive reward prediction error. In this case, we infer that the subjective value of sucrose is increased because the animal perceives that it is a good bargain to receive more than expected. However, it should be noted that dopamine concentration at cue and reward never reached zero when action was maintained in the task. This observation may suggest that the positive and negative prediction errors encountered in this task center around subjective value rather than a zero baseline.

These results are also relevant in the context of surprisal error (38), a central component in hierarchical models of brain function and the free-energy principle. According to this principle, intelligent agents act to minimize surprise (39–41). In this context, the dopamine prediction error signal is a surprise signal, and the content of the surprise is subjective value. Thus, the dopamine signal reflects a prediction error in subjective value—or, more precisely, a utility prediction error.

We also found that increasing dopamine release at either cue presentation or reward delivery decreased response latency—consistent with its role in invigoration (5, 42). Regardless of how valuation changed, optically amplifying transient dopamine release events reduced response latencies. Taken together with our demand curve analyses, we conclude that this invigorating role is dissociable from the role of dopamine in valuation.

Another important advancement in this study is our use of mathematical modeling of demand curves coupled with a comprehensive statistical analysis to quantify the causal effect of dopamine in valuation. While an elegant body of electrophysiological studies have used behavioral economic theory to assess the role of dopamine neural firing in valuation (19, 20), the current study builds upon this work by providing a formal demand analysis during both electrochemical and optogenetic assessments of dopamine release. Demand curves are a common tool used by economists to measure price sensitivity.

Assessing the role of dopamine in valuation using demand curve analysis is a logical progression following a recent study demonstrating that dopamine manipulations alter the valuation of work (27). As in the study by Hamid et al. (27), we found that transient dopamine release events are involved in adaptive value assessments, and that augmenting this signal alters valuation. We also confirm that optical stimulation of dopamine neurons decreases response latency for reward (27), but reached a distinct conclusion following our demand analysis. Despite decreasing response latency, optically increasing dopamine release at cue made animals more sensitive to price. From these observations, we conclude that heightened release at cue both invigorates behavior and leads to a negative reward prediction error due to a mismatch between the value of reward predicted and the value of reward received. This negative mismatch contributes to a decrease in subjective value. We believe this dissociation lends credence to the notion that transient dopamine release events play multiple roles in motivated behavior rather than functioning as a single uniform motivational signal (43).

Several future directions and alternative interpretations should be considered. While the present study demonstrates that optical activation of dopamine neurons modifies valuation, it remains unknown whether inhibiting dopamine neurons produces diametrically opposite effects. It is possible that decreasing release fails to alter valuation, which would suggest that, while dopamine is sufficient to modify valuation, it may not be necessary for valuation to occur. Additional investigation into how optical manipulation of dopamine neurons alters valuation using simpler behavioral designs should also be performed and compared with the results from demand analysis. The role of contingency degradation is another important consideration. Previous experiments demonstrated that providing unsignaled rewards during an experimental session decreases reward seeking through a devaluation process that requires the core region of the NAcc (44, 45). In theory, optical stimulation of dopamine neurons could function similarly to the receipt of unsignaled sucrose—either could ultimately devalue the final reward the animals are working for. This issue would be particularly concerning with long and/or more frequent patterns of dopamine neuron stimulation than those used in the present study (10 pulses at 20 Hz, 0.5-s duration). Future studies are needed to determine how dopamine interacts with other neurotransmitters within a neural network to encode and control valuation. Using the current approach, we can also assess how dopamine release and economic demand for reward are altered in animal models of psychiatric disease.

In conclusion, our results suggest that a transient dopamine value signal encodes subjective value and is capable of modifying the worth an animal places on a desired commodity.

Methods

Subjects and Surgery.

Male Long–Evans rats supplied by Charles River Labs and transgenic rats [LE-Tg(TH-Cre)3.1Deis] expressing the Cre-recombinase protein under control of the tyrosine hydroxylase (TH) promoter (Th::Cre+/−) (46) supplied by Rat Resource and Research Center (300–350 g at time of surgery) were used as subjects. Surgery was conducted in a Kopf stereotaxic apparatus under isoflurane anesthesia (5% induction, 2% maintenance). Upon recovery, rats were food restricted to 90% of their free-feeding body weight. Access to water and crinkle paper enrichment were provided ad libitum in the home cage. Rats were singly housed under a 12:12 light/dark cycle with lights off at 10:00 AM. All experiments were conducted in the dark/active cycle. For FSCV surgery, rats were implanted with a microdialysis guide cannula (BAS) aimed at the NAcc core [+1.3 anteroposterior (AP), +1.4 mediolateral (ML)] and a contralateral Ag/AgCl reference electrode. For optogenetics surgery, rats received Cre-dependent virus [AAV-EF1a-DIO-hChR2(H134R)-EYFP; UNC vector core] and optical ferrule cannulae (preassembled; Thor Labs) aimed at the VTA. Viral infusion coordinates were −5.2 and −6.0 AP, ±0.5 ML, and −7.4 and −8.4 dorsoventral (DV); VTA optical ferrule cannulae coordinates were −5.6 AP, +0.5 L, and −7.9 DV; NAcc optical ferrule cannulae coordinates were +1.3 AP, ±1.4 ML, and −6.5 DV. We also prepared a WT control group. Here, WT Long–Evans rats were transfected with virus and implanted with optical ferrule cannulae similarly to transgenic rats prepared for VTA stimulation. These WT animals also received identical optical stimulation of the VTA during behavior. Behavioral training commenced >1 wk following surgery; sessions that included optical stimulation commenced >4 wk following surgery. The University of Colorado Denver Institutional Animal Care and Use Committee approved all experiments and procedures in advance.

Behavioral Economics Tasks.

Following acquisition, animals were provided daily (7 d/wk) access to sucrose in behavioral economic tasks. In these tasks, sucrose is provided to rats across 10 increasing unit prices (i.e., response requirement/milligrams of sucrose). We developed two iterations of the task to assess the role of both the numerator and denominator of the unit price ratio. As illustrated in the left table of , in the numerator assessment, the response requirement to receive a 45-mg sucrose pellet increases every 10 min with a 30-s time-out period occurring after reinforcement. Thus, the animal may receive a maximum of 900 mg of sucrose per epoch. In the denominator assessment (right table), the response requirement remains fixed at 1, while the milliliters of sucrose solution (300 mg/mL) delivered in each epoch decreases by manipulating pump duration, as previously described and validated during cocaine self-administration (47, 48). In this task, the time-out period decreases in each epoch (30, 10, 5, 3, 1.67, 0.97, 0.54, 0.3, 0.17, and 0.10 s) so that the animal may receive a maximum of 450 mg of sucrose per epoch. Importantly, in each task, unit prices are matched across epochs. Reward availability was indicated to the animal by illumination of a cue light and lever extension. After each reinforced response, the cue light dimmed and the lever retracted for the duration of an experimenter-imposed time-out.

Fitting Demand Curves.

We used the R statistical environment (49) for our analysis of demand profiles. We fitted individual demand profiles to the model equation (Eq. ) using nonlinear least-squares regression, as implemented in the “minpack.lm” R package (50). Our code and data are freely available at https://gitlab.com/oleson/schelp-pnas.

Multilevel Model and Bayesian Analysis.

For the multilevel analysis, we constructed the following multilevel version of our model:

where the index enumerates each observation (a single unit price–consumption pair) within an experimental block. The variable is the treatment associated with this observation. This model structure effectively associated all observations under a given treatment with a treatment level α. Each observation is treated as an independent realization of this model plus some normally distributed noise with an unknown variance :

or, equivalently:

where represents a normal distribution. In the Bayesian inference procedure, the model parameters are treated as random variables with unknown distributions. The goal is to infer these probability distributions such that they best explain the observed data. This is achieved by proposing a prior probability distribution and applying Bayes’ rule to compute a posterior probability distribution that is conditioned on the observed data:

where is the probability of an observation given some . This probability is usually called the likelihood function. It is high when there is a good match between an observed response and the model prediction , or equivalently, when the error term is small in magnitude. Thus, parameter values that predict a closer match with the observations are more heavily weighted in the posterior distributions. We used an efficient Hamiltonian Monte Carlo scheme implemented in the Stan modeling language to generate samples from the posterior distributions (51).

FSCV.

On sessions involving voltammetric recordings, glass-encased carbon fiber microelectrodes were introduced into the NAcc using micromanipulators and locked in place for behavioral testing. Dopamine was detected from fast-scan cyclic voltammograms collected at the carbon fiber electrode every 100 ms (initial waveform: −0.4 to 1.3 V, 400 V/s) (52). Principal-component regression was used as previously described to extract the dopamine component from the raw voltammetric data (53). Representative dopamine concentration traces, but not data used for quantification, were smoothed using the built-in Tarheel CV smoothing option (eight-point nearest-neighbor smoothing kernel). Due to concerns regarding the validity of using standardized calibration factors for dopamine assessments (54), we applied a recently developed computational model (55) designed to calculate calibration factors for individual electrodes using background current values observed during in vivo recordings. For additional detail, see . When relevant, microelectrode placement was determined by performing electrolytic lesions of recording sites before perfusion (). Additional representative histology is also depicted in . See legend, for histology methods.

Optogenetic Stimulation.

Intracranial light (473 nm) was delivered using a laser (opto-engine) under control of a custom arduino system. Stimulation parameters (10 pulses at 20 Hz, 0.5-s duration) were identical for all experiments. Laser output was adjusted according to the Stanford brain tissue light transmission calculator to produce a 1-mm cone irradiance of 1 mW/mm2 in brain tissue, with 15-mW output from optical ferrule cannulae tip. Stability of light retention over the course of experimentation was confirmed ().

Statistics and Data Analysis.

All data analysis, except the behavioral economics component of the study, was performed with SigmaPlot (version 11). ANOVA and Bonferroni post hoc tests were used to assess for changes in dopamine concentration.