Neural correlates of appetitive extinction in humans.

Appetitive extinction receives attention as an important model for the treatment of psychiatric disorders. However, in humans, its underlying neural correlates remain unknown. To close this gap, we investigated appetitive acquisition and extinction with fMRI in a 2-day monetary incentive delay paradigm. During appetitive conditioning, one stimulus (CS+) was paired with monetary reward, while another stimulus (CS-) was never rewarded. Twenty-four hours later, subjects underwent extinction, in which neither CS was reinforced. Appetitive conditioning elicited stronger skin conductance responses to the CS+ as compared with the CS-. Regarding subjective ratings, the CS+ was rated more pleasant and arousing than the CS- after conditioning. Furthermore, fMRI-results (CS+ - CS-) showed activation of the reward circuitry including amygdala, midbrain and striatal areas. During extinction, conditioned responses were successfully extinguished. In the early phase of extinction, we found a significant activation of the caudate, the hippocampus, the dorsal and ventral anterior cingulate cortex (dACC and vACC). In the late phase, we found significant activation of the nucleus accumbens (NAcc) and the amygdala. Correlational analyses with subjective ratings linked extinction success to the vACC and the NAcc, while associating the dACC with reduced extinction. The results reveal neural correlates of appetitive extinction in humans and extend assumptions from models for human extinction learning.

Introduction

Appetitive conditioning and extinction receive increasing attention as a model for psychiatric disorders (Martin-Soelch ). Especially, the extinction of reward associated cues plays an important role in the ability to flexibly interact with the environment and, in clinical cases, in the treatment of addiction-related disorders (Taylor ; Everitt, 2014). However, most research on the neural mechanisms of appetitive extinction is conducted in animals (Millan ). In humans, only the neural mechanisms of fear extinction have been extensively investigated (Quirk and Mueller, 2008; Sehlmeyer ).

Appetitive conditioning in humans can be studied using a classical (Pavlovian) conditioning paradigm or the monetary incentive delay paradigm (MID), which adds an operant conditioning component to the task. In both cases, a neutral stimulus (e.g. a colored rectangle; CS+) is paired with the chance to get a reward (UCS; e.g. money), whereas another neutral stimulus (CS−) is never paired with the chance to win. Additionally, in the MID paradigm, to gain the reward, subjects are required to press a button in reaction to an operant cue that appears in all trials with a varying delay after the CS. The repeated pairing of the CS+ with the UCS (reward) in appetitive conditioning paradigms results in conditioned responses (CRs). CRs comprise positive affective ratings of the CS+ and elevated skin conductance reactions (SCRs) as compared with the CS− (Kirsch ; Klucken , 2015; Andreatta and Pauli, 2015). The MID, however, has rarely been used as a learning task as subjects have in most cases been instructed about the CS-UCS contingencies beforehand.

Brain areas involved in appetitive conditioning mainly include the nucleus accumbens (NAcc) in the ventral striatum, the amygdala, the orbitofrontal cortex (OFC), as well as the caudate nucleus in the dorsal striatum and the anterior cingulate cortex (ACC) (Martin-Soelch ; Chase ). The amygdala mediates the association between CS and UCS during acquisition of conditioning (Balleine et al., 2003). The NAcc as part of the mesolimbic dopamine system is thought to encode the salience of the reward associated stimulus and plays an important part in mediating reward, while the dorsal striatum is involved in goal-directed behavior (dorsomedial striatum) and habit learning (dorsolateral striatum) (O’Doherty ; Burton ). The ventral ACC plays a role in early discriminative learning, while the dorsal ACC encodes the outcome of a CS+ (Gabriel ; Alexander and Brown, 2011). The roles of other areas like the OFC are still debated since different studies ascribed different roles to the OFC including the encoding of expected UCS-value, recall of reward and appetitive extinction (Cox , Moorman and Aston-Jones, 2014; Stalnaker ).

In appetitive extinction, the CS+ is no longer paired with a reward. This leads to the formation of a new extinction memory able to inhibit the previously learned CS-UCS association but not deleting it (Bouton, 2002). In effect, extinction learning reduces the CRs (Andreatta and Pauli, 2015). With respect to its neural correlates, animal studies highlight the role of the basolateral amygdala (Tye ; Portero-Tresserra ) and the ventral striatum (Janak ; Millan ) in appetitive extinction learning. In these brain areas, different neural populations encode the previously learned reward association as well as the extinction memory. The only fMRI-study using an appetitive extinction paradigm was conducted by Tobler . However, the authors focused on the association of reward learning-related activations in the striatum and the midbrain and individual finances and found slower learning in richer as compared with poorer participants. Regarding fear extinction in humans, the reported neural correlates including the amygdala, the NAcc, hippocampus and ventromedial prefrontal cortex are comparable to those reported for appetitive extinction in animals (LaBar ; Schiller ; Sehlmeyer ). A more detailed insight in the neural correlates of human appetitive extinction can help to translate animal findings and give insight into similarities and differences to fear extinction.

The aim of this study was the identification of the neural correlates of human appetitive extinction using the MID-task. After an appetitive conditioning paradigm on the first day, an extinction training was conducted 24 h later without any reinforcement of the CS+ and the CS−. We were interested in examining, whether among others the amygdala or the NAcc are involved in appetitive extinction learning in humans as they are in animals. We further explored correlations between neural activations and changes in subjective ratings from post-acquisition to post-extinction.

Materials and methods

Subjects

Twenty-one male subjects (M = 23.33 years; s.d. = 2.50; range = 18–28 years) were recruited to take part in the study. All subjects were right-handed, native German speakers with European background and had normal or corrected-to-normal vision. Subjects reporting any current or past mental problems or a consumption of psychotropic drugs were excluded as well as subjects with chronic illnesses or treatments preventing them from entering the MRI scanner. Because the experiment was part of an ongoing project exploring endocrinological effects on appetitive conditioning, only male subjects were recruited. All subjects gave written informed consent and received the money they won during the acquisition phase in addition to monetary compensation or course credit for their time. The study was conducted in accordance with the declaration of Helsinki and approved by the local ethics committee. Subjects also filled out the Barratt Impulsiveness Scale (BIS-15, German version, Meule ). The mean score was 32.24 (s.d. = 5.22), similar to the evaluation sample (M = 30.04; s.d. = 6.13).

Procedure

Acquisition and extinction of the appetitive conditioning paradigm took place on two consecutive days roughly 24 h apart and always between 1 and 6 p.m. in the afternoon. As part of a larger study, all subjects took part in a placebo version of the trier social stress test (TSST) on the day of the acquisition (Het ). This entails thinking of and subsequently talking about a pleasant topic (e.g. a favorite movie) and counting upwards in steps of five in an empty room for 15 min in sum. The placebo TSST was performed after the first subjective rating and a practice run of the experiment outside the scanner and directly before entering the scanner. Cortisol analyses confirmed that the placebo TSST did indeed not lead to a stress reaction. Moreover, PANAS (Positive and Negative Affect Schedule; Krohne ) ratings collected before and after the placebo TSST did not indicate any positive [t(20) = 1.67; P = 0.11] or negative [t(20) = 0.99; P = 0.33] mood change (for correlations of positive mood and acquisition of conditioning see the supplementary information).

Acquisition

A modified version of the MID paradigm (Knutson, 2000) was used as conditioning procedure and was conducted in the MRI (Figure 1). It consisted of 21 CS+ with partial reinforcement (∼62% of trials followed by a reward) and 21 CS− trials, with a yellow or a blue rectangle serving as CS+ or CS−. No more than two CS trials of a kind were presented in succession and CS trials were distributed evenly in two blocks of 20 trials (10 CS+ and 10 CS−), with one CS+ and one CS− trial presented at the beginning. All trials were presented in succession; the blocks were not discernible to the subjects. The first trials of the CS+ and the CS− were later excluded from the analyses because learning could not have taken place yet (Phelps ; Klucken ).

Fig. 1.

MID-task. Subjects first saw a CS− or a CS+ (colored rectangle). After a variable delay, a target appeared for a short time. Subjects were instructed to push a button as soon as the target appeared. During the acquisition phase in trials that began with a CS+, fast reactions led to a win. After the target had vanished, feedback about the win and the current total was displayed. During extinction, the same procedure was used but without wins after the CS+.

Every trial started with a variable intertrial interval ranging from 6 to 12 s followed by the presentation of a CS+ or a CS− for 6 s. Following the CS+ or CS−, after a variable (1–3 s) interstimulus interval displaying a fixation cross, the target (a white square) was presented for at least 16 ms up to a maximum of 750 ms (Figure 1). Subjects were instructed to press the button every time as soon as the target was presented regardless of the CS presented before, which they did (no significant differences in response frequency between CS or phases). Pressing the reaction button while the target was visible always resulted in a win of 0.50€, when a CS+ had been presented beforehand. To ensure a similar reinforcement for all subjects (aim: 6.50€ for wins in 62% of CS+ trials) distributed across the whole acquisition phase, the presentation time of the target was adjusted to individual reaction times (RT) in a multistep procedure (see later). Pressing the button during a target presentation that was preceded by the CS− never resulted in a win. Immediately after the target vanished, feedback on the win of money and the current balance was presented for 2.5 s.

Presentation time of the target was based on a training session with different stimuli and was longer in trials with a planned reward (win: MRT + 2 × s.d.RT loss: MRT − 2 × s.d.RT; Dillon ; Balodis ). If subjects did not win as planned during the acquisition phase, the presentation time of the target was adapted online based on the subjects’ performance (−20 ms if subjects won unplanned, +20 ms if subjects did not win as planned; Hahn ; Hermans ). CS+ trials that did not result in wins as planned or vice versa were adaptively repeated in scheduled CS+ trials with the according duration of target presentation. The adaptive online corrections of cue presentation times were necessary for only 4.1% of CS+ trials across all subjects or an average of <1 CS+ trial per subject. The target presentation times in CS− trials were taken from the CS+ trials and adapted accordingly.

Extinction

The extinction phase was conducted in the scanner 24 h later. It consisted of 40 trials with 20 CS+ and 20 CS− trials. Both early and late phase consisted of 10 CS+ and 10 CS− trials in pseudorandomized order with no more than two trials of the same condition presented directly after one another. As before, subjects were instructed to always press the button when they saw the target. In contrast to the acquisition phase, subjects could not win any money regardless how fast they reacted to the target.

Subjective ratings

Subjects were asked to rate the stimuli used as CS+ and CS− on the scales arousal, valence and UCS-expectancy prior to the acquisition phase, after the acquisition phase and after the extinction phase. To avoid reactivation of the memory trace before extinction (Schiller , 2013; Agren ; Agren, 2014), ratings were not collected again right before extinction. For the affective ratings, 9-point self-assessment manikin scales were used (Bradley and Lang, 1994), while UCS-expectancy was rated in 10% steps from 0 to 100%. Subjective ratings were analyzed in 2 (CS) × 3 (Time) analyses of variance (ANOVA) using SPSS 22 (SPSS 22.0 for Windows, SPSS Inc., Chicago, IL, USA). Significant interactions were followed up with paired t-tests and were corrected for multiple comparisons.

Skin conductance measuring

Skin conductance was measured during acquisition and extinction with reusable Ag/AgCl electrodes filled with isotonic (0.05M NaCl) electrolyte medium placed on the non-dominant left hand. Data were collected with a sampling rate of 1 kHz. Ledalab 3.4.4 was used for preprocessing and data analysis (Benedek and Kaernbach, 2010). For preprocessing, the data were downsampled to 100 Hz and smoothed with a 32 sample FWHM Gaussian kernel. All data were visually screened. Technical artifacts (e.g. short negative spikes likely caused by the fMRI) were interpolated with spline interpolation. Due to technical difficulties, three subjects had to be excluded from SCR analysis, leaving a sample of 18 subjects for SCR analysis. A time window of 1–6 s following the onset of CS+ and CS− was defined as analysis window. The extracted response was defined as the highest difference between a maximum and the minimum that directly preceded it. The preceding minimum had to be within the analysis window for the response to be counted. Responses smaller than 0.01 µS were considered zero responses. To screen for non-responders, we checked whether all participants showed at least two responses > 0.05 µS to the UCS, which was the case for all participants. All maximum responses were log (µS + 1) transformed to correct for violation of normal distribution of the data. Mean SCRs for CS+ and CS− were calculated for the acquisition phase as well as for the early and late phase of the extinction phase. Skin conductance data were then analyzed separately for the acquisition phase in a paired t-test (CS+ vs CS−) and for the extinction phase in a 2 (CS+ vs CS−) × 2 (time: early phase vs late phase) using SPSS 22 (SPSS 22.0 for Windows, SPSS Inc., Chicago, IL, USA).

fMRI

All MRI images were acquired using a 3 Tesla whole-body tomograph (Siemens Prisma) with a 64-channel head coil. The structural images consisted of 176 T1-weighted sagittal slices (slice thickness 0.9 mm; FoV = 240 mm; TR = 1.58 s; TE = 2.3 s). For the functional images, a total of 432 images was acquired on the first day (acquisition phase), while 420 images were acquired on the second day (extinction phase). This difference in the number of images is caused by two additional trials (1 CS+/1 CS−) during the acquisition phase compared with the extinction phase. Images were acquired with a T2*-weighted gradient echo-planar imaging (EPI) with 36 slices covering the whole brain (voxel size = 3 × 3 × 3.5 mm; gap = 0.5 mm; descending slice acquisition; TR = 2 s; TE = 30 ms; flip angle = 75; FoV = 192 × 192 mm; matrix size = 64 × 64; GRAPPA = 2). The field of view was positioned automatically relative to the AC-PC line with an orientation of −40°. Preprocessing, first and second level analysis was done using Statistical Parametrical Mapping (SPM8, Wellcome Department of Cognitive Neurology, London, UK; 2008) implemented in Matlab 7.14 (Mathworks Inc., Sherbourn, MA). For preprocessing, all EPI images were coregistered to an EPI template, realigned and unwarped, slice time corrected, normalized to MNI standard space and smoothed with a Gaussian Kernel at 6 mm FWHM. Functional data were analyzed for outlying volumes using a distribution free approach for skewed data (Schweckendiek ). Each resulting outlying volume was later modeled within the general linear model as a regressor of no interest.

The experimental conditions entered during the acquisition phase were CS+, CS−, UCS+, NoUCS+ (no win feedback following a CS+) and UCS− (no win feedback following a CS−). Because the target was presented for a few milliseconds before every feedback and inseparable from the feedback itself, it was not separately modeled. The experimental conditions during the extinction phase were similar to the acquisition but without the UCS+ condition because the CS+ was no longer reinforced. During acquisition, the first CS+ and CS− trial was modeled separately as learning could not have taken place yet. CS regressors were split into an early phase (CS+early/CS−early) and a late phase (CS+late/CS−late), to more clearly discern between early and late effects (LaBar ; Phelps ; Milad ). CS and UCS events were modeled using the duration the respective stimulus was presented (CS = 6 s; UCS = 2 s). The maximum collinearity between a CS+ and a UCS regressor was <0.20. All regressors were convolved with the canonical hemodynamic response function. The six movement parameters were entered as covariates alongside regressors of no interest for the identified outlying volumes. The time series was then filtered with a high pass filter (time constant = 128 s). For acquisition, a CS+ − CS− contrast was calculated for every subject, while for extinction CS+early − CS−early and CS+late − CS−late were computed.

On the group level, one-sample t-tests were performed for the computed first level contrasts to examine neural differences in appetitive conditioning and extinction. Whole brain analyses were conducted with P < 0.05 family-wise-error (FWE) corrected and k > 10 voxels. Region of interest (ROI) analyses were conducted using the small volume correction in SPM8 with P < 0.05 (FWE) and k > 5 voxels. The ROI masks for NAcc, amygdala and caudate were taken from the ‘Harvard-Oxford cortical and subcortical structural atlases’ provided by the Harvard Center for Morphometric Analysis. The masks for OFC, ventral ACC and dorsal ACC were created in MARINA (Walter ). To explore associations of neural activations and subjective measures of extinction, difference scores of CS+ ratings from post-acquisition to post-extinction were calculated. These were correlated with the extracted β estimates for significant peak voxels from the early and late extinction phase.

Results

ANOVA of the subjective rating scales (Table 1) revealed main effects of CS and Time as well as significant CS × Time interactions in valence [CS: F(1, 20) = 7.33; P = 0.014; Time: F(2, 40) = 7.87; P = 0.001; CS × Time: F(2, 40) = 5.15; P = 0.01], arousal [CS: F(1, 20) = 12.67; P = 0.002; Time: F(2, 40) = 5.49; P = 0.008; CS × Time: F(2, 40) = 8.40; P = 0.001] and UCS-expectancy ratings [CS: F(1, 20) = 78.10; P < 0.001; Time: F(2, 40) = 76.75; P < 0.001; CS × Time: F(2, 40) = 165.31; P < 0.001]. CS+ and CS− differed significantly on all three rating scales [Valence: t(20) = 3.63; P = 0.001; Arousal: t(20) = 4.13; P < 0.001; UCS-expectancy: t(20) = 19.80; P < 0.001] after conditioning while they did not differ before the acquisition phase [Valence: t(20) = 0; P = 1; Arousal: t(20) = −0.13; P = 0.900; UCS-expectancy: t(20) = 0.19; P < 0.851].

Table 1.

Mean (s.d.) subjective ratings of CS+ and CS − 

		Pre- acquisition	Post- acquisition	Post- extinction
Arousal	CS +	3.19 (1.86)	6.00 (1.92)*,†	3.95 (2.25)*
	CS −	3.24 (1.92)	3.52 (1.54)	3.52 (1.99)
Valence	CS +	5.43 (1.94)	6.76 (1.67)*,†	4.71 (1.98)*
	CS −	5.43 (1.60)	4.67 (1.62)	4.24 (2.05)
UCS-expectancy	CS +	5.62 (1.75)	8.19 (1.50)*,†	1.52 (2.02)*
	CS −	5.52 (1.57)	0.76 (0.83)*	0.90 (1.18)

*indicates a significant difference to the mean rating of the same CS at the previous time point (P < 0.05).

†indicates a significant difference to the mean rating of the CS- at the same time point (P < 0.05).

After the extinction phase, ratings of the CS+ dropped [valence: t(20) = 3.75; P < 0.001; arousal: t(20) = 4.02; P < 0.001; UCS-expectancy: t(20) = 16.49; P < 0.001] while ratings of the CS− remained unchanged [valence: t(20) = 1.34; P = 0.20; arousal: t(20) = 0; P = 1; UCS-expectancy: t(20) = −0.45; P = 0.658]. In addition, CS+ and CS− did no longer differ significantly on ratings of UCS-expectancy, arousal and valence after the extinction phase [valence: t(20) = 1.11; P = 0.280; arousal: t(20) = 1.12; P = 0.275; UCS-expectancy: t(20) = 2.03; P = 0.056].

Skin conductance responses

During the acquisition phase, SCRs showed a significant effect of conditioning [t(17) = 3.01; P = 0.004] (Figure 2). During the extinction phase, analysis of SCRs in a 2 × 2 ANOVA did neither show a main effect of CS [F(1, 17) < 1] nor a main effect of Time [F(1, 17) = 1.90; P = 0.186]. There was a significant CS × Time interaction [F(1, 17) = 4.58; P = 0.047]. The interaction was mainly driven by a significant reduction in mean SCRs toward the CS+ from the early to the late phase [t(17) = 2.41; P = 0.015] pointing to a successful extinction of conditioning.

Fig. 2.

SCRs (µS log-transformed) for the early and late acquisition phase and the early and late extinction phase. During acquisition, there is a main effect of CS with higher reactions to the CS+. During extinction, there is a CS × Time interaction with higher reactions to the CS+ during the early phase than the late phase. All error bars indicate SEM. *< 0.05; **< 0.01.

Hemodynamic responses

The acquisition phase on the first day reliably activated the reward network implicated in the acquisition of appetitive conditioning, including NAcc, amygdala and OFC (Table 2). In addition, we computed a model separating CS + presentations, which were later reinforced, from those that were not. A comparison of reinforced and unreinforced CS+ trials showed no trends or significant differences (all P > 0.31). A separate analysis of the early and late phase of the acquisition can be found in the supplementary information.

In the early extinction phase, activations were found in the left dorsal and ventral ACC as well as the right caudate nucleus and hippocampus (Table 3, Figure 3). In the late phase, bilateral activation of NAcc and amygdala reached significance (Figure 4). There were no significant differences in the contrast CS− − CS+ in the early or the late phase of extinction.

Fig. 3

Significant ROI activations during the early phase of extinction (CS + early − CS − early) on voxel level P < 0.05 (FWE-corrected). (a) caudate nucleus, (b) hippocampus, (c) dACC (d) and vACC. Displayed t-values are thresholded at t < 1.5. (Lower row) Mean β-weights of the respective CS+ and CS− activations. All error bars indicate SEM.

Fig. 4

Significant ROI activations during the late phase of extinction (CS + late − CS − late) on voxel level P < 0.05 (FWE-corrected). (a) Bilateral activation of the amygdala and (b) bilateral activation of the NAcc. Lines on the sagittal slices on the right side indicate the coronal slices depicted on the left. Displayed t-values are thresholded at t < 1.5. (Right) Mean β-weights of the respective CS+ and CS− activations. All error bars indicate SEM.

Significant correlations of extracted β-weights in significant ROIs during early and late extinction (CS+ − CS−) with differences in subjective ratings to the CS+ (post-extinction − post-acquisition). Higher differences indicate extinction success. (a) Negative correlation of neural activations in the dACC with extinction of UCS-expectancy to the CS+. (b) Positive correlation of neural activations in the vACC with extinction of CS+ arousal. (c) Positive correlation of neural activations in the NAcc with extinction of CS+ valence.

Correlations of BOLD responses during extinction with subjective ratings

Extinction indices were calculated separately for UCS-expectancy, valence and arousal by subtracting the post-acquisition ratings for the CS+ from the respective post-extinction ratings. These differences were then correlated with the extracted β estimates at the peak voxel coordinates of the significant neural activations from the early and late extinction phases.

We found activation of the dorsal ACC to be correlated with UCS-expectancy to the CS+ (r = −0.64; P = 0.002): increased activations of the dACC during early extinction were linked to less reduction of UCS-expectancy from post-acquisition to post-extinction (Figure 5). In contrast, higher activations of the ventral ACC during early extinction were correlated with greater reductions of CS+ arousal ratings (r = 0.48; P = 0.029). Moreover, activation of the left NAcc was positively correlated with a reduction in CS+ valence ratings (r = 0.44; P = 0.048).

Following a worthwhile suggestion from an anonymous reviewer, we also investigated the association of impulsivity with the extinction-related BOLD responses. It was hypothesized that higher impulsivity scores might impair extinction-related neural activation. Correlational analyses of BIS-15 scores with the BOLD responses in the late extinction phase did indeed reveal reduced neural activations in the left amygdala (r = −0.37; P = 0.049), the right amygdala (r = −0.52; P = 0.008), the NAcc (r = −0.40; P = 0.038) and a trend in the right NAcc (r = −0.31; P = 0.083) (see supplementary for further analyses).

Discussion

This study aimed to investigate the neural correlates of appetitive extinction in humans. Using the MID paradigm, neural, subjective and psychophysiological responses were measured on the first (acquisition phase) and second day (extinction phase). On the first day, subjects successfully acquired CRs to the CS+, which were reflected in increased SCRs to the CS+, increased subjective ratings and in an increased involvement of the neural circuit for appetitive conditioning as compared with the CS−. On the second day, CRs were extinguished as subjects did no longer receive a reward following the CS+. This led to a decline in SCRs and in subjective ratings of the CS+. fMRI-results showed activations of the dACC and vACC, as well as the caudate nucleus and the hippocampus during early extinction, while the NAcc and the amygdala were involved during late extinction. Moreover, activation of the dACC was associated with less extinction of subjective ratings, while activations of the vACC and the NAcc were associated with greater extinction of subjective ratings. Because there are still only few fMRI studies on the acquisition of appetitive conditioning, we will briefly discuss these results before discussing appetitive extinction in detail.

Acquisition phase

The results for the acquisition of appetitive conditioning are in line with previous studies on this topic, which also found conditioned SCRs and subjective ratings (Delgado ; Andreatta and Pauli, 2015). Neural activations mainly comprised the dorsal and ventral striatum, the amygdala, the midbrain, as well as the dorsal ACC and the OFC, which together form the reward circuitry and are considered to play key roles in appetitive conditioning (Haber and Knutson, 2010; Chase ). The striatum is thought to encode incentive salience as well as changes in valence (O’Doherty ; Berridge and Kringelbach, 2015). As this study used secondary reinforcement, i.e. money, this might have had an influence on the pattern of neural activations as well. It has previously been shown that different reinforcement types lead to the engagement of different but overlapping subregions in the reward network, especially the ventral striatum (Valentin ). In case of the amygdala, its involvement in conditioning with primary reinforcement has consistently been shown, while its involvement in appetitive conditioning with secondary reinforcers is still under debate (Martin-Soelch ). Some studies that used money as reinforcer did not find activation of the amygdala (Kirsch , Cox ), while others did (Jiang et al., 2014; Kumar ). The findings of the current study support the involvement of the amygdala in classical conditioning with secondary reinforcers. In general, the amygdala is thought to encode the association of CS and unconditioned reaction facilitating the conditioned reaction (Martin-Soelch ).

The present paradigm could be regarded as a more ‘active conditioning design’ because subjects had to press a button in each trial. In contrast, previous studies also used completely ‘passive conditioning designs’, in which subjects received rewards (e.g. money, food and pleasant pictures) without any personal effort (Delgado ; Andreatta and Pauli, 2015; Schweckendiek ). This active learning design could also have impacted the CRs. It could be assumed that an active learning paradigm may lead to a stronger CS+/CS− differentiation. Nevertheless, it would be interesting to directly compare active and passive conditioning designs and compare CRs and other relevant features like contingency awareness.

Appetitive extinction

The main research aim was to investigate the extinction of appetitive conditioning and to explore possible mechanisms through its neural correlates. Subjective ratings of UCS-expectancy, valence and arousal showed successful extinction of appetitive conditioning. This is in line with a recent study by Andreatta and Pauli (2015), who were also able to successfully extinguish appetitive conditioning in subjective ratings of valence and arousal. However, other studies on appetitive extinction found resistance to appetitive extinction especially in subjective ratings of valence (Baeyens ; Dwyer ). In contrast to this study, these two studies used a classical conditioning paradigm, without requiring the subjects to perform a task to obtain the reward. This may have increased the importance in addition to the effects on subjective ratings, the observed appetitive extinction also extended to psychophysiological recordings of skin conductance. This supports the study by Andreatta and Pauli (2015), who also report successful appetitive extinction in SCRs and startle responses.

Early phase of appetitive extinction

During the early phase of extinction, the contrast CS+ − CS− revealed significant activations in the caudate, the hippocampus, as well as the dorsal and the ventral ACC. At this point, SCRs to the CS+ were still higher than in the late phase and the reported neural correlates might also be part of a recall of the consolidated appetitive conditioning (Milad ). Although a few fear conditioning studies report increased activation to the CS− as compared with the CS+ (Hermann ; Merz ), no increased activation to the CS− was found in this study. As one of the main areas implicated in goal-directed behavior (O’Doherty ; Burton ), activation of the caudate/dorsomedial striatum during the early phase of extinction can be viewed as part of the recall of appetitive conditioning 24 h after acquisition. Previous experiments in rodents found that the inactivation of the dorsomedial striatum after appetitive conditioning mimicked the effect of appetitive extinction (Yin ). Activation of the hippocampus has also been associated with the recall of (reward) associative learning (Wolosin ; Hattori ). Studies on fear extinction, however, discuss a role for the hippocampus in encoding contextual information in extinction learning (Abraham ). Moreover, we found activations of the dorsal and ventral ACC. In a computational model based on a range of studies, the dorsal ACC has been proposed to encode predictions of appetitive and aversive outcomes based on previous experiences (Alexander and Brown, 2011). Increased activation of dorsal ACC in substance dependent subjects has been shown to reflect reduced sensitivity to omission of reward (Alexander ). Activation of the ventral ACC on the other hand is assumed to play a role in early discriminative learning or the updating of previously learned associations and might therefore be a part of the extinction learning process (Gabriel ; Schiller ). This functional difference between the dorsal and the ventral ACC has already been discussed for fear extinction (Etkin ). The authors review converging evidence showing correlations between dACC and expression of fear conditioning and vACC and inhibition of fear conditioning. This pattern is reflected in the presented correlations of activity in dACC and vACC with changes in subjective ratings and therefore seems to extend to appetitive extinction. Although higher activation of the dorsal ACC was associated with lasting UCS-expectancy following the CS+, higher activation of the ventral ACC was associated with reductions of arousal ratings of the CS+. Taken together with previous studies on fear conditioning, the present findings suggest that for both appetitive and aversive conditioning dACC encodes previously acquired conditioning while the vACC is part of early extinction processes.

Late phase of appetitive extinction

In the late phase, SCRs showed a decline in reactions to the CS+ indicating successful appetitive extinction. Neural correlates in the late phase can therefore be interpreted as part of the processing of extinction (Milad ). On a neural level, the amygdala and the NAcc were significantly activated during the late phase of extinction (CS+ − CS−). Based on animal studies, Quirk and Mueller (2008) already ascribed the (basolateral) amygdala an important role in human appetitive extinction. Studies with monkeys and rats showed that after functional inactivation of the (basolateral) amygdala, animals continued to show CRs throughout extinction. It has been hypothesized that the amygdala encodes the changing CS-UCS association during extinction (Martin-Soelch ). Tye reported neuron populations in the basolateral amygdala that became active only once extinction learning had begun. Notably, the neurons encoding extinction did not react to omission of reward during a partial reinforcement schedule or to the absence of reward at the beginning of extinction. Similarly, this study also employed a partial reinforcement schedule and found an activation of the amygdala only in the late phase of extinction. Therefore, the activation of the amygdala does not seem to reflect an early reaction to each omission of a possible reward but rather the formation of (new) extinction memory. Moreover, one study on fear extinction that investigated the late phase of extinction separately also found activation of the amygdala in the late phase (Milad ) while other studies report mixed results (for a systematic review see: Sehlmeyer ).

In addition to the amygdala, we found increased activation of the NAcc, which is one of the main research foci in animal studies on appetitive extinction (Janak ; Millan ; for a review see: Millan ). These studies show that the NAcc core mainly plays a role in acquisition of conditioning while the NAcc shell is involved in extinction of appetitive conditioning. Moreover, in human studies the NAcc has also been associated with the acquisition of new or changing contingencies (Klucken ; Sehlmeyer ). Interestingly, activations of the NAcc were only found during the late phase of extinction but not in the early phase. Continued involvement of the NAcc in the MID paradigm, however, has not yet been explored after a 24-h period of consolidation. Animal studies already showed inactivation of the NAcc after the acquisition phase only impaired the reversal of contingencies and extinction of conditioning but not the expression of the consolidated CS-UCS association (Lorens, 1971). Thus, extinction of appetitive conditioning was no longer possible without the NAcc (Reading and Dunnett, 1991). The explored associations of neural activations and changes in subjective ratings also support the involvement of the NAcc in mediating extinction and valence: activation of NAcc was associated with reductions of valence ratings of the CS+ from post-acquisition to post-extinction. This supports the involvement of the NAcc in appetitive extinction and is further in line with reports on the NAcc mediating stimulus valence (Cooper and Knutson, 2008; Berridge and Kringelbach, 2015). In sum, the present results support a role of the NAcc in encoding the update of conditioned valence during acquisition and extinction.

Several authors assume that conditioning is an important model for the development and maintenance of addiction, while extinction processes mirror the treatment of addiction. A central element in the treatment of addiction is therefore the extinction of drug-related cues (Millan ). We found increased activation of the NAcc during extinction. The NAcc has repeatedly been associated with changes in the relationship between CS and UCS (Schiller ; Klucken ), which is an important element in exposure-based therapy. Another central element of exposure-based therapy is the inhibition of conditioned reactions, which is thought to be mediated by the amygdala (Quirk and Mueller, 2008).

Moreover, in line with current models of addiction, activation of the NAcc and the amygdala during the late phase of extinction was negatively correlated with the participants’ impulsivity. This supports assumptions that impulsivity is a risk factor for addiction (Potenza and Taylor, 2009; Gullo and Potenza, 2014; Potvin ). Although previous studies showed associations of impulsivity with reward anticipation (Beck ; Balodis ), this study indicates that impulsivity is also associated with lower activations related to the extinction of previously learned reward associations. Because appetitive extinction serves as a model for the treatment of addiction, these findings indicate that low impulsivity might be a predictor for the success of exposure-based interventions or that these interventions might benefit from impulse control trainings.

A few limitations need to be addressed. First, the sample of this study consisted only of male subjects. Studies of the neural correlates of human fear conditioning showed effects of sex hormones on fear extinction (Merz ). Therefore, further studies need to include female subjects to allow for a generalization of the present findings. Second, in the early extinction phase, effects of recall of appetitive conditioning and aspects of early extinction learning probably overlap. This study cannot entirely disentangle these two processes during the early extinction phase. Based on animal findings on appetitive extinction, regions involved in the expression of conditioning and extinction were expected to overlap. Although animal studies are able to disentangle these neuron populations in multicell recordings, the spatial resolution of fMRI does not allow this. Therefore, differences between acquisition, early extinction phase, and late extinction phase on a neural level could not be studied by direct comparison but its neural correlates had to be analyzed separately.

In conclusion, this study aimed to investigate extinction of appetitive conditioning and its neural correlates. Processes during the early phase of extinction can reflect the recall of the acquired appetitive conditioning and early processes of extinction learning. These differing processes were most pronounced in the ACC. Although the dACC was associated with stable expectance of rewards, the vACC was associated with extinction of subjective ratings. This extends previous aversive conditioning results to appetitive conditioning leading to the assumption that the ACC is involved in emotional learning regardless of the specific valence. The activation of amygdala and NAcc during the late phase of extinction supports their role in encoding the new extinction memory, which is in line with animal studies on appetitive extinction. Thus, our study provides a first insight into the neural correlates of appetitive extinction in humans.

Funding

This work was supported in part by a scholarship from the Justus Liebig University Giessen to Onno Kruse.

Supplementary data

Supplementary data are available at SCAN online.

Conflict of interest. None declared.

Click here for additional data file.

Table 2.

ROI activations during the acquisition phase (CS+ − CS−)

structure	Side	k	x	y	z	zmax	r	Pcorr
Amygdala	L	121	−24	−2	−12	3.24	0.65	0.038
	R	106	20	−2	−12	3.37	0.66	0.029
Caudate	L	493	−8	16	−2	4.86	0.84	<0.001
	R	517	10	10	4	5.05	0.85	<0.001
dACC	L	1299	−6	10	42	5.19	0.86	<0.001
	R	1426	4	18	38	5.38	0.88	<0.001
Midbrain		621	4	−30	−2	4.80	0.83	<0.001
NAcc	L	104	−8	16	−2	4.53	0.81	<0.001
	R	85	8	8	−4	4.67	0.78	<0.001
OFC	L	120	14	16	−12	3.56	0.69	0.050

Localization, cluster size (k), effect size (r) and statistics (FWE-corrected) of the peak voxel in the respective ROI.

Table 3.

ROI activations during the early phase and late phase of extinction (CS+ − CS−)

contrast	Structure	side	k	x	y	z	zmax	r	Pcorr
CS + (early) − CS − (early)	dACC	L	616	−10	2	34	4.18	0.77	0.007
	vACC	L	330	16	38	20	3.58	0.69	0.034
	Caudate	R	189	8	16	8	3.52	0.69	0.033
	Hippocampus	L	194	−24	38	−4	3.33	0.66	0.073
		R	72	22	−38	6	3.53	0.69	0.041
CS + (late) − CS − (late)	Amygdala	L	85	−12	−6	−18	3.43	0.67	0.024
		R	130	16	−4	−18	3.36	0.66	0.032
	Nacc	L	71	−12	8	−8	3.24	0.64	0.020
		R	30	12	14	−6	2.99	0.61	0.033

Localization, cluster size (k), effect size (r) and statistics (FWE-corrected) of the peak voxel in the respective ROI.