Changing value through cued approach: an automatic mechanism of behavior change.

It is believed that choice behavior reveals the underlying value of goods. The subjective values of stimuli can be changed through reward-based learning mechanisms as well as by modifying the description of the decision problem, but it has yet to be shown that preferences can be manipulated by perturbing intrinsic values of individual items. Here we show that the value of food items can be modulated by the concurrent presentation of an irrelevant auditory cue to which subjects must make a simple motor response (i.e., cue-approach training). Follow-up tests showed that the effects of this pairing on choice lasted at least 2 months after prolonged training. Eye-tracking during choice confirmed that cue-approach training increased attention to the cued items. Neuroimaging revealed the neural signature of a value change in the form of amplified preference-related activity in ventromedial prefrontal cortex.

RCT Entities:   Population Interventions Outcomes

A basic tenet of economic theory is that choices are a reflection of the values that the decision maker places on the available options[1], but it is well known that these values can change with experience, time, and context. Research in psychology over the past century has focused on reinforcement as the main mechanism to influence value[2,3], whereas research in behavioral economics has focused on factors such as the description of the decision problem (e.g. framing[4-6]). These studies have shown that choices are influenced by how options are described, how preferences are elicited and by the context of other available options[5,7,8], but they have not attempted to directly perturb the underlying basic values of individual items. Other work has shown that choice itself influences preference[9-11] and that preferences can also be modulated by mere exposure to items[12,13] and by embodied approach/avoidance responses[14-16]. Here we demonstrated a novel mechanism by which value can be modulated in a lasting manner simply through cued approach to a stimulus using a combination of behavioral, eye tracking and functional neuroimaging methods.

The present approach was influenced by previous research demonstrating that, by modulating the time spent viewing items[17], manipulations of attention can influence choice[18,19]. It has likewise been shown that focusing attention at behaviorally relevant points in time, regardless of the attended item, can “boost” memory for visual scenes[20,21]. We examined whether association of a food item with an irrelevant cue and motor response can modulate subsequent choices to consume the item.

In each of the studies reported below, participants were asked to fast for four hours prior to visiting the laboratory. Upon arrival, they participated in an auction (the BDM procedure)[22,23] (Fig. 1a) that measured willingness-to-pay (WTP) for each of 60 snack food items; participants were informed that the auction would be played for real consumption at the end of the experiment. Items were sorted based on WTP and split into sets of higher and lower valued items to allow assessment of differential effects based on individual item preferences. Participants underwent cue-approach training in which they observed images of individual food items presented for 1 sec (Fig. 1b) and were instructed to press a button as fast as possible (before the image disappeared) only when they heard a tone (i.e. Go trials). The tone presentation delay was adjusted using a staircase procedure to ensure that the task remained difficult. Sixteen items (8 high-value and 8 low-value) were consistently associated with a tone. There was no feedback regarding the success of the button press within the allotted time window. The entire set of items was presented 8, 12 or 16 times across the different studies. Within each study each item was presented the same number of times, thus controlling for any potential mere-exposure effects[12,13]. In a probe phase that followed training, participants were presented with pairs of items that were matched for their initial utility (to allow isolation of the cue-approach effect, Fig. 1c) but differed by whether they had or had not been associated with a cue during training (i.e. one Go and one NoGo item). Participants were asked to choose one item per trial for a chance to consume that item at the end of the study. One trial was randomly selected and the choice on that trial was honored. If training had no effect, participants should be indifferent between the two items and choose them equally often.

Figure 1

Task procedure. a) Participants were endowed with three dollars and told that after making a series of auction-based choices, they would have an opportunity to use them to buy a snack. During the auction phase, participants were presented with a series of 60 items, one at a time on a computer screen. They bid by moving a mouse cursor along an analog scale that spanned from 0 to 3 at the bottom of the screen. The auction was self-paced and the next item was presented only after the participant placed their bid. b) During training, participants were instructed to press a button when they heard a tone (occurring after a variable delay based on a staircase) but before the image disappeared from the screen (one second after it appeared). Images appeared on the screen one at a time and ~25% of items were associated with a tone. Trials were separated by a jittered inter-trial interval with a mean duration of 3 seconds. c) During the probe, participants were instructed to choose one of two items that appeared on the screen to the right and left of a central fixation cross. Participants were told that a single trial would be selected and honored for real consumption, meaning they would receive the food item they chose on that particular trial. Participants had 1.5 seconds to make their choice and trials were separated by a variable inter-trial interval with a mean duration of 3 seconds. d) The auction described in panel a was repeated at the end of the experiment.

RESULTS

Cue-approach binary-choice effects

We varied the length of training from the initial study (which included 12 repetitions of all 60 snack food items) to test whether fewer (8 repetitions, Study 2) or more (16 repetitions, Study 4) repetitions would have a different effect on choice. In all 4 independent samples (studies 1 to 4, total N = 102, Fig. 2a and Supplementary Table 1), we found a consistent effect of cue-approach training on choices during the probe phase for high value item pairs: high value Go items were chosen on 60% – 65% of trials (P’s = 0.00006 to 0.008; 63% across samples 1–4, P < 0.0001). There were no differences in this effect between the different samples (all pairwise comparisons p’s > 0.6). This occurred even though participants had never been previously presented with these items in pairs, nor were they externally reinforced for pressing the button during training. We did not find an increase of Go item choices in low value pairs (except for a marginally significant effect in Study 1 [58%, P = 0.054] and a weak effect across studies 1–4 [55% P = 0.032]). The increase in choice for Go items was significantly higher for high value versus low value pairs (P < 0.001 across the cue-approach studies 1–4). The fact that the effect was not consistent across high and low value pair trials suggests that choosing Go items was not a general strategy adopted by participants at probe (e.g. due to demand characteristic), but rather that cue-approach training was selectively more effective for higher value items.

Figure 2

Behavioral results for cue-approach and cue-avoidance studies. a) Proportion of choices of the GO item in pairs of high value Go versus NoGo (dark grey) and low value Go versus NoGo (light grey) items for each of the four cue-approach studies as well as for Study 4 retest. Number of repetitions reflect number of individual stimulus presentations during training. Significance level reflects odds of choosing the Go versus NoGo item. b) Willingness to pay (WTP) pre- and post-training for Go (solid line) and NoGo (dashed line) separately for items in the probe high Go versus high NoGo (top) and low Go versus low NoGo (bottom) pairs. The sample includes all participants from studies 1 though 4 of cue-approach. Significance level reflects interaction for time by item type (Go or NoGo) in a repeated measures linear regression. c) Proportion of choices of the Go item in pairs of high value Go versus NoGo (dark grey) and low value Go versus NoGo (light grey) items for the two cue-avoidance studies. d) Proportion of total choice time during probe that gaze position was on the high Go (black) or high NoGo (white) item in a pair for trials when Go or NoGo items were chosen separately. The sample is a subset of Study 4. Eighteen participants had their eye positions recorded with an eye tracker while performing the cue-approach task. Significance levels reflect repeated measures linear regression. Effects for panel d are discussed in the text. Error bars for panels a and c reflect one standard error of the mean (SEM). Error bars for panels b and d reflect within subject SEM. All P’s reflect 2-sided significance levels. *** P < 0.0001, ** P < 0.001, * P < 0.01, + P < 0.05

We should emphasize that all items used were palatable junk food items. In an additional study (Study 7) where all items were below the median auction value (see Supplementary Fig. 3 for details), there was still a cue-approach effect (see Supplementary Fig. 4 for results) for the higher-valued items within this set, suggesting that the difference in cue-approach effects between high and low valued items may be relative to the overall distribution of value in the training set; alternatively, there may be a nominal value threshold below which the effect does not occur.

Cue-approach auction effects

Following the probe phase, the initial auction was repeated (Fig. 1d) to test whether the expressed monetary value of individual items changed following training (both auctions were played out together such that only one bid was chosen). We found evidence that non-reinforced training influenced the value of individual items. Because high and low valued item sets were selected based on extreme values in the initial auction, both showed overall regression to the mean (see Fig. 2b). Across samples 1–4 the high Go items retained their value better (i.e. showed less regression to the mean) than the high NoGo items (delta between high Go and NoGo = 12.2 cents, P = 0.0006 combined across samples 1–4). For the low items there was also regression to the mean for both Go and NoGo items (delta between low Go and NoGo = 4.9 cents less, P = 0.12). When both auctions were analyzed using repeated measures linear regression with Go/NoGo and value (High/Low) factors and a participant grouping factor, there was a main effect of Go/NoGo (P < 0.0001) and the interaction term of High/Low by Go/NoGo was marginally significant on the second auction bids when accounting for the first auction bids (P = 0.053, two-tailed).

This suggests that cue-approach training also influenced subjective values of Go versus NoGo items (as expressed by relative changes in WTP) differently for high and low value items. It should be noted that previous work has demonstrated effects of choice on preferences (e.g. Brehm 1956[9] and Sharot et al., 2009[10] for the underlying neural mechanisms), which extend for long periods of time[24]. Thus, it is likely that the binary choices influenced the auction. In an additional sample collected without a probe phase (Study 9) there was not a significant effect on the 2nd auction (see Supplementary Table 4). Direct comparison of the auction effect between the two samples that had this effect (Study 2 and Study 4) with the no-probe Study 9 showed a marginal difference for Study 4 versus Study 9 (P = 0.0592) and for Study 2 versus Study 9 (P = 0.0471). When combining samples 1 through 4 and comparing them to Study 9 the comparison is not significant (P = 0.1 two-tailed). This hints that pre-post auction differences might reflect the effects of choice during the probe phase. A fuller understanding of the factors that modulate these changes in value will require additional empirical work.

Cue-avoidance and cue without approach

To further test the mechanism underlying the cue-approach effect we ran two additional independent samples (Fig. 2c) on a cue-avoidance task, which is a functional mirror of the cue-approach manipulation. The entire procedure described in Fig. 1 was identical except for the training phase. In this phase, rather than pressing the button only when the sound was heard, the participants were asked to press the button every time they saw an image, unless they heard a sound. The timing of the sound was titrated using a ladder technique similar to a standard stop-signal task[25]. In this version the cue appears at exactly the same frequency as Studies 1 through 4 but serves as an inhibition/avoidance cue rather than an approach cue, similar to the “automated inhibition” version of the stop-signal task[26,27]. In a probe phase identical to that described above, we found no difference between choices of Go and NoGo items (for choice of high Go: 48% [P = 0.5] and 45% [P = 0.3] in studies 5 (with 12 repetitions of all items) and 6 (with 16 repetitions of all items) respectively; for choice of low Go: 52% [P = 0.7] and 53%, [P = 0.4]). Furthermore, in a direct comparison between the pooled samples of the cue-approach version (studies 1 to 4, N = 102) and the pooled sample of the cue-avoidance version (studies 5 and 6, N = 68), we found that the choices of the high Go items were significantly greater for the cue-approach task (P < 0.0001) but not for the low Go items (P = 0.28).

To test whether a motoric action is necessary for the cue-approach effect, we performed an additional study (Study 8) where we repeated the same procedure without any motor responses. Participants heard the cue at times yoked to Study 4 participants, but were not required to make any motor responses. We did not find any effects during the probe phase (see Supplementary Fig. 4) of Study 8. There was a significant difference for choices of high Go value items between Study 4 and Study 8 (P = 0.002) but not for low Go value items (P = 0.7). These results suggest that the motor response is crucial for the cue-approach effect.

Eye-gaze during training

We further examined whether the observed effects were due to greater observation times during training and thus modulation of the mere-exposure effect (see Yagi et al, 2009[28], Huang and Hsieh, 2013[29]). We tested the gaze time during training on part of the samples where sufficient eye-tracking data were collected: On twelve participants from the imaging sample (Study 3), on all participants in the last run of the longer training cue-avoidance study (Study 6) and on all participants for the study with cues but no approach (Study 8). We did not find any significant differences in gaze times across all of training nor on the last training run (see Supplementary Table 3) in any of these studies.

It should be noted that there were no significant behavioral choice effects in any of these studies (nor within the subsample of 12 participants from Study 3) and thus these gaze data cannot entirely rule out an effect of gaze differences. To examine this question further we performed a regression of gaze time for each item and subsequent choices for the corresponding item during probe for each of these studies. This was possible because there was variance in choices of individual items even though there was no mean difference in choices between Go and NoGo items. We found no significant effect of gaze time (see Table S3) either for Go items and/or for the interaction of choices of Go versus NoGo items in any of these samples. Furthermore, the fact that participants observed the items for almost their entire presentation times in all of these studies (and for similar periods of time at the end of training) suggests that it is unlikely that the observed effects are driven by differences in gaze duration. Additional work is necessary to conclusively test whether gaze differences are related to choices during probe.

Eye-gaze during probe

Recent work shows that participants spend more time fixating on an item prior to choosing it compared to unchosen alternatives[17] and that high-value items draw attention[30], reflecting the role of attentional mechanisms in value computation and choice behavior[31]. Eye tracking data collected on a subset of sample 4 (16 repetitions, N = 18) during the probe phase was consistent with this. Proportion time spent viewing a particular item was calculated as the total amount of time the gaze position was within the bounds of a food item on the screen, divided by the reaction time. In a repeated measures linear regression comparing proportion of time spent viewing an item against Go/NoGo and Chosen/Unchosen factors (with two grouping factors of item presentations and participants), we found a main effect of Chosen/Unchosen (two left bars versus two right bars in Fig. 2d, P < 0.0001) and Go/NoGo (black versus white bars in Fig. 2d, P = 0.002). Thus participants tended to spend a larger proportion of the total gaze time on the chosen versus non-chosen item during the choice window regardless of whether it was a Go item (proportion = 39% versus 28% [left black bar versus right white bar in Fig. 2d], P < 0.0001) or NoGo item (proportion = 36% versus 32% [left white bar versus right black bar in Fig. 2d], P = 0.01), consistent with the previous results of Krajbich et al, 2011[17].

More importantly, we found a significant difference in gaze time between non-chosen Go versus NoGo items (P = 0.04; black versus white bar on the right in Fig. 2d), showing that cue-approach training drove attention towards the Go items in the subsequent decision-making phase even when these items were not chosen. This may reflect increased salience of Go items[32]. Further analyses found a positive association between the pre-post training difference in bids and the proportion of time spent viewing an item during the choice phase (P = 0.029), linking the observed gaze bias[18] directly to changes in preference.

Long-term behavioral maintenance

To examine the long-term maintenance of the cue-approach effect we invited participants to return to the lab after their initial visit to perform a second probe phase. We found that, after an average of 66 days (range 41–87), participants in the longest training sample (16 presentations) showed continued preference for the high Go over NoGo items (N = 20, P = 0.04, proportion 60%). Long-term maintenance was also present in the eye tracking data; an analysis of eye tracking data during the follow-up probe showed a main effect of Go/NoGo (black versus white bars in Supplementary Fig. 2, P = 0.0001), though in this case the effect was only present for chosen items (Supplementary Fig. 2). This suggests that longer cue-approach training may have effects that last over months. No such effect was found in any of the other shorter training samples.

In an additional sample (Study 7) we repeated the probe after one week and one month (see Supplementary Fig. 4); the cue-approach effect on choices remained significant at both delays for the higher valued items (which were below the median in this sample). Further studies will be needed to verify how much additional training may be necessary to ensure long-term maintenance and for how long this effect lasts.

Imaging results during probe and training

The foregoing behavioral and eye-tracking results provide converging evidence that cue-approach training increased the subjective value of cue-associated Go items. To investigate the neural signature of this value change, training and probe were performed while participants were scanned with fMRI in Study 3 (N = 21, items presented for 12 repetitions). Previous studies[23,33-35] establish that the vmPFC is crucial for encoding decision values, and therefore we focused our analysis on this region during the probe phase. During the probe phase we tested whether the greater preference for the chosen Go items was reflected in BOLD signals. The pairs during probe were matched for their pre-training preference; the effectiveness of this procedure is highlighted by the lack of difference in choices of Go and NoGo items in the cue-avoidance (studies 5 and 6) and no-approach study (Study 8). We defined the post-training preference for each item as the number of times each item was chosen during the probe phase (out of 8 presentations), and used this value as a parametric regressor for the chosen item on each trial (Go or NoGo). Consistent with their common involvement in representation of subjective value, activity in vmPFC and ventral and medio-dorsal striatal regions was positively associated with number of times chosen for high Go items (Fig. 3a, for all activation peaks see Supplementary Table 2). Further, we found that the association between BOLD response and preference was stronger (more positive) in vmPFC for high Go items compared to high NoGo items during probe when limiting the analysis to an extensive area of the vmPFC (Fig. 3b). Thus, the enhanced preference for high Go items was also reflected in the neural signature of a stronger relation between vmPFC activity and preference. A previous study[36] involving real versus hypothetical choices shows that the signal in brain regions including vmPFC are amplified during real choices, which can be attributed to increased attention during real choices. We did not find any neural signature for a value change for the low Go (versus either baseline or low NoGo items), consistent with the lack of a behavioral effect.

Figure 3

Imaging results. a) The parametric effect of the number of times each high value Go item was chosen during probe. b) The difference between High Go and High NoGo items in parametric effect of the number of times each item was chosen during probe. This analysis was only run in an a-priori mask of mPFC that encompassed the medial PFC by combining Harvard-Oxford atlas regions (frontal pole, frontal medial cortex, paracingulate gyrus, and subcallosal cortex) falling between x = 14, x = −14 and z < 0. x/y/z values reported in standard MNI space. Heatmap color bar ranges from z-stat = 2.3 to 3.3. Maps in panels a, c and d were cluster corrected at a whole brain level. All maps P < 0.05.

We also examined training-related changes in fMRI signals. There were no differences in stimulus-driven activations when comparing the fMRI activity related to onset of Go items at the end of training versus the beginning of training, perhaps reflecting the fact that no choices were required (participants simply pressed a button when they heard a tone). Previous studies show value-related activations in the absence of choice[37] and therefore for all subsequent analyses we used as a parametric regressor the same preference indicator as in the probe phase, i.e. the number of times each item was chosen during probe. In the last run of training, vmPFC response (using the pre-hypothesized anatomical mask for the region) was associated with number of choices on the later probe for both Go and NoGo items (separately) (Fig. 4). However, there were no significant differences between the preference-related effect for Go and NoGo items during the last run, and no preference-related effects for either item type during the first run of training. This suggests that the mere exposure to the items for both Go and NoGo items increased their values in a relatively similar way.

Figure 4

Modulation of number of times each a) high value Go item was chosen and b) NoGo item was chosen during probe. This analysis was only run in an extensive mask of mPFC that encompassed the medial PFC by combining Harvard-Oxford regions (frontal pole, frontal medial cortex, paracingulate gyrus, and subcallosal cortex) falling between x = 14, x = −14 and z < 0.

DISCUSSION

Previous research has focused on the roles of experience, time, and context in modulating decision values[2-6]. The present work is the first to show that the subjective value of goods (measured both by choice and by willingness-to-pay) can be modified through simple cue-approach exposure without external reinforcement or any other explicit manipulation of value. Our converging results using behavioral and neuroimaging methods provide insight into an automatic mechanism that directly perturbs the values of items by driving attention towards those items. We suggest that the cue-approach manipulation enhanced the attention devoted to the processing of the items and thus amplified the value signals of the items. This is further exemplified by the fact we obtained this effect only for the higher valued items in the training set, even when these were below the median (in Study 7). We found that the auditory cue by itself was not sufficient for the behavior change and that the change is not due to viewing time differences during training. It is plausible that a motor approach signal was activated during the probe phase, but the effect of training on auction responses, which required an entirely different response, speak against this alternative. Results of the cue without approach and the lack of effects in the cue-avoidance studies all suggest that the cue-approach effect is not merely an attentional or salience effect, but rather that it reflects the combination of the cue with the motor approach response. Other work[38,39] has managed to train participants to avoid choices of food items, albeit in a different paradigm with close to 100% stopping rates. This suggests that the failed stop trials in our studies (which targeted 75% successful inhibition) may have obscured the effect. Additional studies will be needed to test this. Future studies will also need to test whether the choice effects reported here generalize from the specific trained pictures to other stimuli representing the same or similar items.

In conclusion our findings suggest that cue-approach training can potentially serve as a “nudge”[40] working at the level of enhancing attention to and memory for items in order to modify choices. This is especially important given the failures of many current interventions aimed at changing unhealthy habits based on effortful self-control[41].

Online Methods

Participants

Two hundred and fifty two healthy subjects participated in nine studies using two versions of a task (see Supplementary Table 1 for samples details). No statistical test was run to determine sample size a priori. The sample sizes we chose are similar to those used in previous publications, and our replications used similar sample sizes after our first successful result.

All participants had normal or corrected-to-normal vision, no history of psychiatric diagnoses, neurologic or metabolic illnesses, no history of eating disorders, had no food restrictions, and were not taking any medications that would interfere with the experiment. Additionally, participants who were scanned (Study 3) were free of any metal implants or any other contraindications for MRI. Participants were told that the goal of the experiment was to study food preferences and were asked to refrain from eating 4 hours prior to arrival to the laboratory[23]. All participants gave informed consent and the study was approved by the internal review board (IRB) at the University of Texas at Austin.

Experimental procedure

The general task procedure is presented in Fig. 1. Participants first underwent an auction (Fig. 1a), then a training task (Fig. 1b), then a probe (Fig. 1c) and finally a repeat of the auction (Fig. 1d). Stimulus presentation and behavioral data acquisition were implemented in Pygame[42] for the Auction and Matlab (The Mathworks Inc.) with Psychotoolbox[43-45] for the training and probe phases.

Auction

First, participants took part in an auction[22,23] (Fig. 1a) in which photographs of 60 appetitive junk food items[23] were presented in random order. We followed the BDM procedure[22,23] where participants were endowed with three dollars and told that they would have an opportunity to use them to buy a snack at the end of the session. During the auction, participants were presented with one item at a time on a computer screen. They placed their bid by moving the mouse cursor along an analog scale that spanned from 0 to 3 at the bottom of the screen. The auction was self-paced and the next item was presented only after the participant placed their bid. This procedure has been shown to reliably obtain a measure of willingness to pay per item (WTP)[23]. Two participants from cue approach Study 1, one from Study 3 and two from Study 4 were excluded because there was not enough range in bids for those participants to properly categorize the stimuli using the method explained in the next section. In the cue-avoidance study, three participants were excluded from Study 5 and two from Study 6 because they bid less than $0.25 on more than 40 items; in studies 7 and 8 we also excluded one participant in each sample due to auction exclusion criteria. These exclusions ensured a sufficient number of highly valued items that were different enough from lower valued items.

Item selection

Items were ranked based on WTP where item #1 had the highest WTP and so forth until item #60, which had the lowest WTP. We then chose 8 items as higher-valued (ranked 8–15) and 8 items as lower-valued (ranked 46–53). Out of each of these 8 items, 4 were associated with an auditory cue to later serve as Go items and 4 were not associated with the cue to later serve as NoGo items (see Supplementary Fig. 1). This selection procedure ensured pairing of high Go with high NoGo items and low Go with low NoGo items. These pairs made up of two items of similar WTP later presented at probe (see below) such that participants should a priori be indifferent in a choice between them based on initially stated values. To maintain ~25% cue frequency (8 out of 30 high and 8 out of 30 low) as is usually done in stop-signal tasks[25], we chose an additional 4 high Value items (out of the items ranked 16–23) and an additional 4 low value items (out of the items ranked 38–45) that were later used in high Go versus low Go comparisons and high NoGo versus low NoGo comparisons during probe (results not reported).

For Study 7 (N = 26) we only used the lower 30 items for training and selected items ranked 38:45 as the higher-value items (see Supplementary Fig. 3).

Cue-approach training

The task is functionally opposite to the stop-signal task[25]. On each trial, images of the food items were presented on the screen for 1 second followed by an inter-trial interval of an average 3 seconds (range 1–12 seconds, Fig. 1b). Item presentation was randomized within a block of 60 trials. Participants were instructed to press a button on the keyboard as fast as they could, but only when they heard a tone. Their task was to press the button before the items disappeared from the screen. The items that were chosen as Go items were consistently associated with the tone. In Study 1 we initiated the tone at 650 milliseconds after the item was presented on the screen (Go-signal-delay, GSD) and then updated the GSD using a ladder technique; we increased the GSD by 17 milliseconds if participants pressed the button before the item disappeared, whereas the GSD was reduced by 50 milliseconds if the participants pressed the button after the item disappeared (always after 1 second). We chose this 3:1 ladder titration ratio to ensure a 75% success rate in button presses. In study 1 (N = 29) and 3 (N = 21) all 60 food items were presented 12 times each during training. In Study 2 they were presented 8 times and in Study 4 (N = 27) they were presented 16 times. The order of presentation within the sixty items was randomized and all items were presented before the next set started. After two presentations of all items the participants received a short self paced break before continuing to the next run. In studies 2, 3 and 4 we initiated the ladders at 750 milliseconds as the results from Study 1 showed a convergence around that number.

Cue-avoidance training

Item selection was identical to the procedure for cue-approach training. Task: this task is highly similar to the trained-inhibition version of the stop-signal task[26,27]. All components and timings of this task were identical to the cue-approach training except for two details: 1) In this task, participants were instructed to press the button as fast as they could every time they saw an item, unless they heard a tone. 2) Since this task was designed to optimize the difficulty of stopping, the stop-signal-delay was initiated at 250 msec. We used a similar 3:1 ratio for ladders titration in this task; we decreased the ladder by 50 milliseconds if participants didn’t stop on time and increased it by 17 milliseconds if they did manage to stop. In Study 5 (N = 42) all items were presented 12 times and in Study 6 (N = 26) all items were presented 16 times.

Cue without approach

For Study 8 we followed the same procedure as in the cue-approach training but did not ask the 31 participants to press the button when they heard the tone. We used yoked ladders from Study 4.

Probe

Following the completion of training, participants filled in a computer-adapted version of the Barratt Impulsiveness Scale (BIS)-11 questionnaire[46]. They were then told that they would next perform a new task (Fig. 1c) where they would be presented pairs of items. They were told that a single trial would be drawn at random at the end of the session and their choice on that trial would be honored (i.e. they would receive the item that they had chosen on that trial at the end of the experiment and will stay to consume it in the lab).

Pairing procedure

We presented unique pairs of items during the probe phase. The main goal of our analysis was to test how the cue-approach or avoidance training affected participants’ preferences between items that had similar initial value. We presented items from the same value category (high or low with similar WTP rankings) such that one item was associated with the cue during training and the other was not (see Supplementary Fig. 1). Participants were presented with 16 unique pairs (each of the 4 Go items were paired with each of the 4 NoGo items) for each value category. If the manipulation did not affect participants’ valuation of the items then they should be indifferent between them. Participants were also presented with 2 additional pair types made up of high Go versus low Go and high NoGo versus low NoGo items. These pairs served as “sanity checks” to ensure that the initial WTPs truly represented participants’ values (the results of these comparisons are not reported, but in all cases the expected differences between choice of high and low valued items were observed).

No Probe version

In Study 9 we did not include the probe phase for 25 participants.

Trial timing

At trial onset, the two items in a pair were presented directly to the right and left of a fixation cross (See Fig. 1c). Participants had 1.5 seconds to respond with either of two buttons on the keyboard corresponding to the left or right locations on the screen. The chosen item was highlighted with a green rectangle around it. The choice confirmation remained on the screen for 500 milliseconds until a fixation cross appeared during the inter-trial interval for an average of 3 seconds (range 1–12). If the participants did not choose within the allotted time, a message appeared on the screen asking them to please choose faster followed by the inter-trial fixation cross and the next trial. Each of the 64 unique pairs was presented twice across the two probe runs. The order and left-right locations of the items on the screen were randomized across participants and across the two runs.

Questionnaires

As mentioned above, the BIS-11[46] questionnaire was administered between training and probe (no significant correlations were found with the proportion of choices of high Go items, P’s > 0.3). At the end of the session, when participants remained in the lab to consume the food item they received several additional questionnaires which are beyond the scope of this manuscript.

Statistical Analyses

Probe behavior

To test whether cue-approach (or cue-inhibition) training induced a preference change, we performed a repeated measures logistic regression using the lmer function from the lme4 library in R[47] to compare the odds of choosing the Go to NoGo items against equal odds for the high value and low value pairs separately. We also performed a repeated measures linear regression to test for differences in reaction time (RT) for choices of Go and NoGo items for the high value and low value pairs separately.

Probe eye-tracking

Eye-tracking data were acquired using an Eyelink-1000 SR-Research (Mississauga, Ontario, Canada) eye-tracker. Usable data was obtained on 18 participants from cue-approach Study 4. Gaze position was categorized as being either within the x-axis boundaries of the fixation cross, within the x-axis boundaries of the stimulus on the right of the fixation cross, or on the left of the fixation cross. The proportion of the trial time spent looking at the right or left items on each trial was calculated. We examined the difference in the proportion of total trial time spent looking at the Go item versus the NoGo item, when the participant chose the high Go or the high NoGo item separately using repeated measures linear regression. We also examined the difference in the proportion of time spent looking at the Go versus the NoGo item when that item was not chosen using repeated measures linear regression. We also looked at the main effect of Go/NoGo item assignment as well as the main effect of Chosen/Unchosen on the proportion of choice time spent viewing an item during probe phase using repeated measures linear regression including the two factors Go/NoGo and Chosen/Unchosen with two grouping factors (for item presentation and participant).

Auction

Repeated measures linear regression was used to test the two-way interaction between Time (pre-training auction and post-training auction) and Condition (Go and NoGo) within each Value category separately. This interaction tests if the change in WTP over time is different for Go and NoGo items. P values for the effects in the mixed models were calculated using the Kenward-Rogers approximation for degrees-of-freedom[48]. In order to better account for regression to the mean, we looked at the main effect of factor Go/NoGo item assignment as well as its interaction with value (high/low value items) on WTP at the second, post-training auction while accounting for WTP on the first, pre-training auction using repeated measures linear regression with a grouping factor for participant.

We also investigated the influence of the change in WTP from pre- to post-training auctions on the proportion of choice time spent viewing a particular item during the probe phase accounting for Go/NoGo item assignment using repeated measures linear regression with a grouping factor for participant.

Retest

We re-contacted all participants and requested that they return to the lab. Participants were requested to fast for 4 hours similar to the original experiment. In this follow-up session, 20 participants performed the auction and then probe phase with the same pairings as the one they had originally performed on their first visit to the lab.

Imaging version

We performed Study 3 of the cue approach paradigm while 21 participants were scanned with fMRI. In this version participants used an MRI-compatible response pad to enter their response. They filled in the computer-adapted version of the BIS-11[46] using the MRI-compatible button box prior to the probe phase while inside the scanner.

fMRI acquisition and analysis

Imaging data were acquired on a 3T Skyra MRI scanner (Siemens, Erlangen, Germany) with a thirty-two channel head coil. Functional data were acquired using a T2*-weighted echo planar imaging sequence (repetition time [TR] = 2000 ms, echo time TE = 30 ms, flip angle [FA] = 60°, field of view [FOV] = 256, acquisition matrix of 128×128. Forty eight oblique axial slices with a 2 mm inplane resolution were positioned 30° off the anterior commissure-posterior commissure line to reduce the frontal signal dropout[49] and spaced 2 mm with a 0.5 mm gap to achieve full brain coverage). Slices were acquired using the multiband sequence[50] (acceleration factor = 2, parallel imaging factor iPAT = 2) in an interleaved fashion. Higher order shimming was used to reduce susceptibility artifacts. Each of the training runs consisted of 194 volumes and each of the probe runs consisted of 164 volumes. In addition to functional data, a single 3D high-resolution full brain image acquired using a magnetization prepared rapid gradient echo (MPRAGE) pulse sequence (TR = 1900 milliseconds, TI = 900 milliseconds, TE = 2.43 milliseconds, FA = 9°, FOV = 25 cm^2) was acquired for brain masking and image registration.

Raw imaging data in DICOM format were converted to NIFTI format and preprocessed through a standard preprocessing pipeline using the FSL package[51] version 5. Functional image time series were first aligned using the MCFLIRT tool to obtain six motion parameters that correspond to the x/y/z translation and rotation of the brain over time. Second, the skull was removed from the T2* images using the brain extraction tool (BET) and from the high-resolution T1 images using Freesurfer[52,53]. Spatial smoothing was performed using a Gaussian kernel with a FWHM of 5 mm. Data and design matrix were high-pass filtered using a Gaussian-weighted least-squares straight line fit with a cutoff period of 100 seconds. Grand-mean intensity normalization of each run’s entire 4D dataset by a single multiplicative factor was also performed. The functional volumes for each participant and run were registered to the high resolution T1-weighted structural volume using a boundary-based registration method[54] implemented in FSL5 (BBR). The T1-weighted image was then registered to the MNI152 2mm template using a linear registration implemented in FLIRT (12 DOF). These two registration steps were concatenated to obtain a functional-to-standard space registration matrix.

Imaging Analysis

Probe

We focused our analysis on the probe phase to examine the neural signature of value change. We used a general linear model (GLM) for the probe phase that included 7 regressors for each of the four trial types: For high Go versus high NoGo: 1) Onsets of trials when high Go items were chosen with fixed duration of 0.87 seconds which was the average RT across all trials across all subjects; 2) To explore the preference for each item (Fig. 3a) we used the demeaned total number of choices (on all probe trials where this item appeared) of the chosen item as a parametric modulator of the above onset regressor (the same average RT was used as duration as above). 3) To account for the difference in pre-training WTP between the items in each pair we added the WTP difference as a parametric modulator with the same onsets and durations as regressor #1. All of the above 3 regressors were added for the trials when participants chose the NoGo item in a pair. To account for RT differences between choices of the Go and NoGo items we added a regressor with the onsets of all high Go and NoGo trials but as the modulator we added the demeaned RT across all these trials. We defined the same 7 regressors for the probe trials that compared low Go to low NoGo and also high Go versus low Go and high NoGo versus low NoGo, resulting in a total of 28 regressors (4 trial types times 7) and an additional regressor for missed trials of all types. We included the 6 motion regressors described above, framewise displacement (FD) and RMS intensity difference from one volume to the next (DVARS)[55] as confound regressors. We also modeled out trials with FD and DVARS that exceeded a threshold of 0.5 by adding a single time point regressor for each “to-be-scrubbed” volume[56]. All regressors were entered at the first level of analysis and all (but the added confound regressors) were convolved with a canonical double-gamma hemodynamic response function. The temporal derivative of each regressor (but the added confound regressors) was included in the model. The model was estimated separately for each participant for each run.

To test which regions showed a greater modulation by the preference for an item we contrasted the parametric modulator of the chosen high Go items (regressor #2 above) with the same regressor for the high NoGo items (Fig. 3b). We masked this contrast by our pre-hypothesized vmPFC region. The mask encompass the medial PFC by combining Harvard-Oxford regions (frontal pole, frontal medial cortex, paracingulate gyrus, and subcallosal cortex) falling between x = 14, x = −14 and z < 0.

Four participants were excluded from the imaging analysis because their parametric modulator of choices was zeroed out. One chose all high Go items at exactly the same proportion and three chose all high NoGo items at the same proportion during probe. Thus, the parametric modulator was perfectly correlated with the intercept regressor (column of 1s) resulting in a rank deficient design matrix.

For all group analyses we averaged across individual subjects to perform a one-sample t-test to obtain the overall effects for the group. All reported statistical maps were corrected at the whole-brain level using a cluster-based Gaussian Random Field correction for multiple comparisons, with an uncorrected cluster-forming threshold of z = 2.3 and corrected extent threshold of P < 0.05, except for the comparison between modulation of high Go and high NoGo during probe, which was corrected only for the medial PFC mask.

Training

The GLM during the training phase included 4 regressors for each Go item broken down by subsequent 4 probe trial types (high Go versus high NoGo, low Go versus low NoGo, high Go versus. low Go and high NoGo versus low NoGo): 1) onsets of the Go trial, modeled with a fixed duration of 1 second; 2) same onset and duration but modulated by subsequent number of choices during probe; 3) same onset and duration but modulation by initial WTP; 4) same onset and duration but modulated by the Go Signal Delay for that trial. Thus there were 4 different Go trials and for each there were 4 regressors yielding a total of 16 regressors. Then for each of the different types of the NoGo trials there were 3 regressors similar to above except the modulation by the Go Signal Delay as there was no signal in the NoGo trials. Thus, there were 4 different NoGo trials and for each there were 3 regressors yielding a total of 12 regressors. To account for RT differences between all trials we added a regressor with the onsets of all Go trials and the modulator was the demeaned RT across all these trials. We further added a missed trial regressor for high Go and low Go and 2 regressors for an erroneous response for high and low NoGo. There were a total of 33 regressors. We added the same covariates as in the Probe design matrix: We included the 6 motion regressors described above, along with framewise displacement (FD) and RMS intensity difference from one volume to the next (DVARS)[55] as confound regressors.