The anticipation and outcome phases of reward and loss processing: A neuroimaging meta-analysis of the monetary incentive delay task.

The processing of rewards and losses are crucial to everyday functioning. Considerable interest has been attached to investigating the anticipation and outcome phases of reward and loss processing, but results to date have been inconsistent. It is unclear if anticipation and outcome of a reward or loss recruit similar or distinct brain regions. In particular, while the striatum has widely been found to be active when anticipating a reward, whether it activates in response to the anticipation of losses as well remains ambiguous. Furthermore, concerning the orbitofrontal/ventromedial prefrontal regions, activation is often observed during reward receipt. However, it is unclear if this area is active during reward anticipation as well. We ran an Activation Likelihood Estimation meta-analysis of 50 fMRI studies, which used the Monetary Incentive Delay Task (MIDT), to identify which brain regions are implicated in the anticipation of rewards, anticipation of losses, and the receipt of reward. Anticipating rewards and losses recruits overlapping areas including the striatum, insula, amygdala and thalamus, suggesting that a generalised neural system initiates motivational processes independent of valence. The orbitofrontal/ventromedial prefrontal regions were recruited only during the reward outcome, likely representing the value of the reward received. Our findings help to clarify the neural substrates of the different phases of reward and loss processing, and advance neurobiological models of these processes.

INTRODUCTION

Individuals must constantly make effective decisions that minimise harm and enhance well‐being (Fellows, 2004). Reward processing is crucial for directing actions towards positively valanced stimuli like rewards, while loss processing (also termed negative reward processing), facilitates the avoidance of negative outcomes, such as punishments (Lutz & Widmer, 2014). Reward and loss processing can be characterised by two distinct temporal phases—an anticipation phase, where the prospect of a reward/loss is initially encountered, and an outcome phase (also called the receipt phase), where the reward/loss is received or omitted (Haber & Knutson, 2010; Knutson, Westdorp, Kaiser, & Hommer, 2000; Knutson, Fong, Adams, Varner, & Hommer, 2001; Lutz & Widmer, 2014).

Many functional Magnetic Resonance Imaging (fMRI) studies have examined the neural correlates of reward and loss processing (Wang, Smith, & Delgado, 2016). Several meta‐analyses have synthesised the results of hundreds of fMRI studies examining reward and loss processing in healthy human adults (Bartra, McGuire, & Kable, 2013; Clithero & Rangel, 2014; Diekhof, Kaps, Falkai, & Gruber, 2012; Knutson & Greer, 2008; Liu, Hairston, Schrier, & Fan, 2011) and adolescents (Silverman, Jedd, & Luciana, 2015), with two key findings emerging so far. First, anticipating and receiving a reward recruits the ventral striatum, likely representing an initial prediction signal, and a positive prediction error signal (i.e., ‘a better than expected outcome’) respectively (Diekhof et al., 2012; Galtress, Marshall, & Kirkpatrick, 2012; Haber & Knutson, 2010; O'Doherty et al., 2004). Receiving a reward recruits the orbitofrontal (OFC) and ventro‐medial prefrontal (vmPFC) cortical regions (due to the uncertain anatomical and functional distinctions between the OFC and vmPFC, we present these as synonymous unless otherwise noted), which represent the subjective reward value (Galtress et al., 2012; Levy & Glimcher, 2012; Peters & Büchel, 2010). Yet, outstanding issues prevent the understanding of the neural substrates of reward and loss processing. It remains unclear whether reward and loss anticipation are dissociable or overlapping at a neural level, and whether the OFC/vmPFC are involved in either the anticipation of reward, its receipt, or both.

The evidence to date does not address whether the neural substrates of anticipation/receipt of loss and reward either dissociate or overlap as the findings are mixed. Some studies ascribe reward and loss processing (across both anticipation and receipt) to dissociable neural systems including the ventral striatum, and anterior insula and amygdala, respectively (Hardin, Pine, & Ernst, 2009; Knutson, Fong, Bennett, Adams, & Hommer, 2003; Liu et al., 2007; Yacubian et al., 2006). Other studies suggest striatal activity is engaged during loss and reward processing, especially their anticipation (Breiter, Aharon, Kahneman, Dale, & Shizgal, 2001; Carter, Macinnes, Huettel, & Adcock, 2009; Cho et al., 2013; Kohls et al., 2013; Pfabigan et al., 2014), where midbrain dopamine systems engage during stimulus approach or avoidance (Brooks & Berns, 2013; Pfabigan et al., 2014; Salamone & Correa, 2012). Behavioural and psychological data have also not shown clear distinctions between these processes (Baron & Galizio, 2005), thus similar neural mechanisms may underlie both. Previous meta‐analytic work on reward and loss anticipation has been limited in their ability to establish how similar or distinct reward and loss processing are, due to limited examination of loss processing (Bartra et al., 2013; Knutson & Greer, 2008), or not examining temporal and valence interactions (Liu et al., 2011).

While the evidence to date shows that the OFC/vmPFC plays a key role in reward processing (Bartra et al., 2013; Diekhof et al., 2012; Knutson & Greer, 2008; Liu et al., 2011), it remains unclear if the OFC/vmPFC is implicated during the anticipation or the receipt of rewards (or both). Some studies show responses specific to receiving rewards (Bjork, Smith, Chen, & Hommer, 2010; Kirk, Brown, & Downar, 2015; Knutson et al., 2001, 2003), yet others report it is active during both phases of reward processing (Breiter et al., 2001; Haber & Knutson, 2010; Kim, Shimojo, & O'Doherty, 2011; Liu et al., 2007; Peters & Büchel, 2010; Rushworth, Noonan, Boorman, Walton, & Behrens, 2011). More work is needed to elucidate whether the OFC/vmPFC are linked to specific temporal phases of reward processing.

Methodological limitations in the literature to date may prevent accurate mapping of the neural substrates of reward processing (i.e., striatal recruitment during reward and loss anticipation; OFC/vmPFC involvement in reward anticipation versus receipt). First, meta‐analyses of reward/loss processing to date have used outdated Activation Likelihood Estimation (ALE) techniques affected by within‐experiment bias and implementation errors, increasing the risk of false positives (Eickhoff et al., 2016; Eickhoff, Laird, Fox, Lancaster, & Fox, 2017; Turkeltaub et al., 2012), though recently available ALE techniques address this issue.

Second, heterogeneous reward processing tasks and task stimuli may systematically confound the findings to date. Tasks used to measure reward/loss could vary substantially in their complexity. For instance, some tasks required only a single, rapid response to obtain a reward, whereas in others a correct response needed to be selected from multiple alternatives and the optimal alternative would need to be learned overtime. Different reward/loss tasks types require different cognitive demands and abilities, which in turn recruit different neural patterns of activity within and outside the reward and loss systems (Balodis & Potenza, 2015; DePasque Swanson & Tricomi, 2014; Lutz & Widmer, 2014; Richards, Plate, & Ernst, 2013). This includes OFC/vmPFC activity, which can vary with probability contingencies, effort and the temporal delay between response and outcome (Haber & Knutson, 2010). Additionally, heterogeneous task stimuli (e.g. monetary, verbal feedback or food), may engage distinct cognitive processes and reward related networks (Bartra et al., 2013; Clithero & Rangel, 2014; Liu et al., 2011). For example, monetary rewards, compared to food and erotica, engage less activity in the amygdala/insula and uniquely engage the anterior OFC (Sescousse, Caldú, Segura, & Dreher, 2013).

This meta‐analysis builds upon previous work to map the neurobiology of reward and loss processing in healthy adults by addressing limitations of previous meta‐analyses to date. We aimed to address the confounding effect of task and stimuli heterogeneity by focusing on one specific task; the Monetary Incentive Delay Task (MIDT, explained in Figure 1).

Figure 1

The structure of the MIDT. (a) Examples of cues signifying trial type (e.g. a square with two lines indicates a punishment trial where $5 can potentially be lost, a circle with a single line indicates a reward trial where $1 can potentially be obtained). (b) Time course of a trial on the MIDT. In the cue stage (first screen), corresponding to the anticipation phase of reward/loss processing, a symbol appears indicating the trial type (reward, loss or neutral). After a delay (second screen) a target appears (third screen), and participants are instructed to press a button as quickly as possible when the target appears. If participants press the button quickly enough they gain money (reward trials) or avoid losing money (loss trials) and are informed of this during the feedback stage (fourth screen), corresponding to the outcome phase. The time window within which the participant has to make a response to obtain a successful outcome is constantly adjusted such that the participant succeeds on an expected 60‐66% of trials (Knutson et al., 2000). Note that the timings depicted are an example only. These timings vary from study to study

The MIDT is the most consistently used task to probe the neural substrates of reward and punishment processing in humans (about 200 MRI studies so far) (Lutz & Widmer, 2014) and has been developed based on pre‐clinical findings that anticipating a reward engages dopaminergic neurons in the ventral tegmental area (VTA) (Knutson et al., 2000). Advantages of focusing on the MIDT include: (i) it allows modelling the interaction between valence and temporal phase (i.e. reward and loss anticipation, reward and loss outcome); (ii) it requires a simple decision, minimising cognitive confounds (e.g. complex decision making) (Balodis & Potenza, 2015; Knutson & Greer, 2008; Lutz & Widmer, 2014); and (iii) it robustly engages the striatum, which is critical in reward processing (Haber & Knutson, 2010) and can be difficult to image when using fMRI (Knutson & Cooper, 2005; O'Doherty, 2009; Walter, Stadler, Tempelmann, Speck, & Northoff, 2008).

Based on the issues summarised, we aim to identify patterns of neural activity of reward and loss anticipation, and for reward anticipation and receipt, by conducting a meta‐analysis of 50 studies that used the MIDT in healthy adults, with an updated version of the ALE technique that minimise positive biases (Eickhoff et al., 2017). Based on the evidence to date, we examine if reward and loss anticipation would recruit a common neural substrate—the striatum—or if these processes are clearly distinguishable. Secondly, we explore whether the OFC/vmPFC would be engaged only during reward receipt or reward anticipation as well.

METHOD

Sample selection

We conducted searches in PubMed and Scopus using the terms “Monetary Incentive Delay Task AND fMRI” on 22 July 2016 and identified 300 studies (173 for PubMed and 127 for Scopus), which was reduced to 182 unique studies when duplicates were removed. In addition, if it was evident that studies used overlapping or the same sample (e.g. Boecker‐Schlier et al., 2016; Boecker et al., 2014; Hommer et al., 2003; Knutson et al., 2003) then only one of those was used (either the first published or one for which activation coordinates could be obtained).

FMRI studies of healthy human adults were selected if they included the following contrasts: reward anticipation versus neutral anticipation/no‐incentive anticipation (reward anticipation phase), punishment/loss anticipation versus neutral anticipation/no‐incentive anticipation (loss anticipation phase), and successful reward/gain outcome versus unsuccessful reward outcome/neutral trial outcome (reward outcome phase). We did not examine the contrast loss outcome versus avoided loss outcome/neutral outcome (loss outcome phase), as this was reported by less than 20 studies, which is insufficient for robust ALE meta‐analytic estimates (Eickhoff et al., 2016).

The ALE meta‐analysis technique assumes that the activation foci are obtained through a whole brain analysis. We thus excluded results from region‐of‐interest analyses. As shown in Table 1, we included studies using modified versions of the original MIDT (Knutson et al., 2000) (and classified each study according to the original MIDT version), if these (i) retained the same basic task structure and key stages—specifically, a cue stage where the cue is visualised and no choice or action has to be made; a target stage, where a response is made as fast as possible to an appearing target; and a feedback stage presenting the trial outcome; (ii) assessed a contrast of interest in a relevant stage; (iii) participants played for their own gain. If a study used the original MIDT version in a hybrid format with additional elements, the study was classified as having used the original version.

Table 1

Studies included in the present meta‐analysis

Study	N	Reward Anticipation	Loss Anticipation	Reward Outcome	Loss Outcome	MIDT type
Adcock et al. (2006)	12	✓	–	–	–	A
Balodis et al. (2012)	14	✓	✓	–	✓	B
Beck et al. (2009)	19	✓	✓	✓a	–	A
Behan et al. (2015)	20	✓	–	–	–	E
Bjork et al. (2004)	12	✓	✓	✓	–	A
Bjork et al. (2008)	23	✓	✓	✓	–	F
Bjork et al. (2010)	24	✓	✓	✓	✓	H
Bjork et al. (2012)	23	✓	–	✓	–	G
Boecker et al. (2014)	162	✓	–	✓	–	A
Bustamante et al. (2014)	18	✓	✓	✓	–	D
Carl et al. (2016)	20	✓	–	✓	–	A
Carter et al. (2009)	17	✓	✓	–	–	A
Choi et al. (2012)	15	✓	✓	–	–	A
Costumero et al. (2013)	44	✓	–	–	–	D
Damiano et al. (2014)	31	✓	–	✓	–	A
Dillon et al. (2010)	32	✓	–	✓	–	B
Enzi et al. (2012)	15	✓	✓	–	–	A
Figee et al. (2011)	19	✓	–	✓	–	A
Filbey et al. (2013)	27	–	–	✓	✓	I
Funayama et al. (2014)	20	✓	✓	–	–	B
Hägele et al. (2015)	54	✓	–	–	–	B
Hanssen et al. (2015)	57	✓	–	✓	–	E
Juckel et al. (2006b)	10	✓	✓	–	–	A
Juckel et al. (2012)	13	✓	✓	–	–	A
Jung et al. (2011)	20	✓	✓	✓	–	A
Kappel et al. (2013)	20	✓	–	–	–	A
Kaufmann et al. (2013)	19	✓	✓	–	–	B
Kirk et al. (2015)	44	✓	✓	✓	–	B
Knutson et al. (2001a)	8	✓	✓	–	–	A
Knutson et al. (2001b)	9	✓	–	–	–	A
Knutson et al. (2003)	12	✓	✓	✓	–	C
Knutson et al. (2008)	12	✓	✓	✓	–	B
Maresh et al. (2014)	84	✓	✓	–	–	A
Mori et al. (2016)	15	✓	✓	–	–	C
Mucci et al. (2015)	22	✓	✓	✓	–	A
Pfabigan et al. (2014)	25	✓	✓	–	–	B
Rademacher et al. (2010)	28	✓	–	✓	–	A
Romanczuk–Seiferth et al. (2015)	17	✓	✓	✓	–	B
Saji et al. (2013)	18	✓	✓	–	–	A
Samanez–Larkin et al. (2007)	12	✓	✓	✓a	–	C
Schlagenhauf et al. (2008)	10	✓	–	–	–	A
Stoy et al. (2011)	12	✓	✓	✓a	–	A
Stoy et al. (2012)	15	✓	✓	✓a	–	B
Ströhle et al. (2008)	10	✓	–	✓a	–	B
Treadway et al. (2013)	38	✓	✓	–	–	A
Weiland et al. (2014)	12	✓	–	–	–	A
Wrase et al. (2007)	14	✓	✓	✓	✓a	A
Wu et al. (2014)	52	✓	✓	✓	–	C
Yan et al. (2016)	22	✓	✓	✓	✓	A
Yau et al. (2012)	20	✓	✓	–	–	A

nonsignificant whole brain finding. A = standard version, B = standard version with a delay occurring after responding to the target, C = +$0, ‐$0 trials used instead of neutral, D = non‐incentive trials instead of neutral, E = no delay period after cue presentation, F = potentially had to repeat making a response to the target to achieve an outcome, G = on reward trials participants could potentially be notified they successfully hit the target but receive no monetary reward, H = after the target response and prior to feedback a screen would appear displaying the words “Did you hit?“, I = similar to the standard version however anticipation was only measured after responding to the presentation of the target but before receiving feedback.

Most MIDT studies that we included used the same contrast for examining reward/loss anticipation to neutral anticipation, where neutral trials led to no win/loss of money regardless of performance. We also included two studies (Bustamante et al., 2014; Costumero et al., 2013) that compared reward/loss anticipation versus no‐incentive anticipation—whereby a cue informs that the trial is a ‘no‐incentive type’ and the task does not have to be performed, as no target will follow the cue. Therefore, a no‐incentive trial type may lack any activity related to action planning, which has a key role in reward/loss processing (Cooper & Knutson, 2008). To ensure that the use of no‐incentive trials was not biasing the result, we ran the analysis both with and without the studies that used such a contrast.

We used two different types of contrasts to examine the construct of ‘reward receipt’ (i) “successful (i.e., gain of reward on a reward trial) versus unsuccessful (not gaining a reward during a reward trial)”, (ii) “successful versus neutral (neutral outcome received on a neutral trial)”. Using either contrast alone would have resulted in an underpowered analysis. However, we ran an exploratory ALE conjunction and subtraction analysis to compare these two contrast types to see if they produced different patterns of activation.

We also included one study (Bjork, Smith, Chen, & Hommer, 2012) that used a contrast of successful versus unsuccessful reward outcome for reward trials—but masked out the neural activity from the outcome of “Hit” trials (i.e., notifying participants, after target response, they would not receive a monetary reward regardless of performance). This construct was conceptually similar to the contrasts of interest, as the “Hit” trials resembled a neutral trial outcome.

Variability across studies in terms of the length of reward anticipation trials may confound results, since activation in some regions, such as the OFC/vmPFC, may only occur during lengthy anticipation phases (Diekhof et al., 2012). We performed additional analyses to ascertain whether this activation was driven by either the length of interval between reward‐anticipatory cue and feedback, or by a jittered time interval in trials between cue and feedback. Specifically, we grouped studies by the length of reward anticipation—defined as the time from initial exposure to the reward cue until presentation of the target within a trial. Anticipation trials were less than 3000ms in duration for 18 studies (short duration), and between 3000ms and 6000ms in duration for 22 studies (long duration). The cut‐off of 3000ms was chosen as it represented a clear threshold discriminating anticipation durations of different studies, and choosing a lower/higher cut off point would have led to an underpowered analysis (as at least one of the groups would be substantially below the recommended number of studies). We then ran a single ALE analysis separately for the short duration and long duration groups of studies, followed by a conjunction/subtraction ALE analysis to determine the similarities/difference in activation when different anticipatory durations are used.

If a study used different magnitudes (e.g. $1 and $5), then only the result where both magnitudes were combined into a single contrast was used. If this combined contrast was unavailable, then only the result for the highest magnitude was used. An additional analysis was conducted to assess if similar patterns of activation likelihood during reward anticipation are observed when only a low magnitude (less than $2 USD or equivalent) is used, in order to determine whether the results of our meta‐analysis were driven by using studies using different reward magnitudes. There was an insufficient number of studies that reported loss anticipation with low magnitudes to conduct a meaningful analysis for this contrast.

Finally, we converted all foci that were in Talairach space to MNI space using the Lancaster transformation function (Lancaster et al., 2007), so the results were in the same coordinate space.

Activation likelihood estimation (ALE)

Single analysis

We performed the meta‐analysis using the ALE technique implemented in the GingerALE software package (version 2.3.6, http://brainmap.org/ale) (Eickhoff et al., 2009; Eickhoff, Bzdok, Laird, Kurth, & Fox, 2012; Eickhoff et al., 2017). ALE treats activation foci from each given brain coordinate as a three‐dimensional spatial Gaussian probability density with a full‐width half maximum (FWHM) derived from the number of subjects in each study, so that using more subjects results in a smaller FWHM, with increased certainty that an activation occurred at that voxel. This method accounts for the spatial uncertainty associated with neuroimaging results (Eickhoff et al., 2009; Eickhoff et al., 2012).

We used the analysis pipeline outlined below (termed a single analysis) to examine neural activity relative the following contrasts: reward anticipation versus neutral anticipation/no‐incentive anticipation (i.e. reward anticipation), punishment anticipation versus neutral anticipation/no‐incentive anticipation (i.e. loss anticipation), and successful reward outcome versus unsuccessful reward outcome/neutral trial outcome (i.e. reward outcome).

The non‐additive ALE method was used to create a Modelled Activation (MA) map for each experiment (Turkeltaub et al., 2012). This method takes the maximum value of the Gaussian distributions encompassing a given voxel (as multiple Gaussians may overlap a single voxel), and thereby reduces the chance that the final ALE result is overly biased by any one single experiment. Then, the MA maps were combined to produce a statistical whole‐brain map where each voxel has an ALE value indicating its probability of activation. Next, non‐linear histogram integration was used to find the null distribution of ALE values under spatial independence, to ensure the results were not due to random convergence (Eickhoff et al., 2012). For this step, each MA map was converted to a histogram, thus removing spatial information. This histogram contained all the possible MA‐values sorted into bins. These histograms were then divided by the total number of voxels in the MA map and produced a probability of obtaining a given MA value. A table of p‐values for each ALE value—indicating the probability of obtaining a particular value—was then computed by combining these probabilities for each individual experiment.

To identify significant clusters of activation, cluster‐level inference was used (Eickhoff et al., 2012). First, a cluster‐forming threshold of 0.001 uncorrected was applied to identify areas of significant activation. A false‐discovery rate (FDR) threshold, which has often been used in previous meta‐analyses, was not used as this method is highly susceptible to false‐positives (Chumbley & Friston, 2009; Eickhoff et al., 2012, 2016). Next, these clusters were compared to an empirically derived null distribution of cluster sizes. To do this, a random data set was created with the same number of experiments, foci per experiment and subjects per experiment as the real data, but with the foci coordinates randomly distributed amongst these groups. An ALE analysis was then run on the randomised data (using the cluster‐forming threshold to identify clusters of activation). This process was repeated 5,000 times to create a null distribution of cluster sizes. The clusters identified in the real data were then assigned a p‐value based on the number of clusters in the null distribution which exceeded it in size. We used a cluster‐level threshold of 0.05 (i.e. a cluster was significant if less than 5% of all randomly formed clusters were greater in size) to identity significant clusters of activation (Eickhoff et al., 2012). Notably, this method, known as ‘cluster‐level family‐wise error (FWE) correction’, recently had an implementation error corrected (which could lead to non‐significant small clusters being incorrectly labelled as significant) in the 2.3.6 release of GingerALE (Eickhoff et al., 2017). Simulations using empirical data have found cluster‐level FWE corrections to be most appropriate for statistical inference (Eickhoff et al., 2016).

Conjunction and subtraction analysis

Conjunction analyses were run to identify areas of shared activation between reward anticipation and loss anticipation, and between reward anticipation and reward outcome, by taking the minimum value at each voxel of the two ALE maps, as calculated in the single analysis step. For these contrasts, a subtraction analysis was run to identify distinct areas of activation (Eickhoff et al., 2011), by repeating the following process 5,000 times: (i) all the results that contributed to either of the two contrasts being examined, were pooled and randomly split into two groups that were the same size as in the original data, (ii) for each of these randomly derived groups, an ALE score was calculated at each voxel, (iii) the difference between these ALE scores was recorded. Once finished, the ALE values were collated across all permutations to yield an empirical null distribution that was used for statistical inference. Next, a p‐value was assigned to each voxel based on how many times the difference in the null distribution exceed the actual difference between the two groups. Finally, a threshold of p < 0.001 uncorrected with a minimum cluster size of 100mm3 was applied to identify significant differences between the two contrasts and a Z‐score was also obtained at each voxel to indicate the size of any such differences.

RESULTS

MIDT and sample characteristics

We meta‐analysed 50 studies that comprised a total of 1,271 participants (718 males). Table 1 indicates the characteristics of each study including: the number of participants, the MIDT type used the contrasts used from each study. Of the included studies, 43 studies were obtained via a literature search, and seven via cross‐referencing (Bjork et al., 2004, 2012; Filbey, Dunlop, & Myers, 2013; Knutson, Adams, Fong, & Hommer, 2001; Knutson, Fong, et al., 2001; Samanez‐Larkin et al., 2007; Wrase et al., 2007). Individual studies had samples composed of between 8 and 162 healthy participants. Handedness was reported in 31 studies, of which 29 studies included only right‐handed participants, and two included also a minority of left‐handed and ambidextrous participants. Most studies reported a spatial resolution of between 3–4mm3, ranging from 3.75mm × 3.75mm × 7mm to 1.5mm × 1.5mm × 3mm.

All studies offered actual monetary reward, which was dependent on successful performance. Also, all studies manipulated (in a different fashion) the success/hit rates of the MIDT by controlling the duration of the target/the time required for a successful response. Most studies (38 of 50) used tasks that had a set success rate of 65–67%, while a minority used a set success rate of 50% (Dillon et al., 2010; Figee et al., 2011; Pfabigan et al., 2014) or 60% (Yau et al., 2012). Other studies used data from practice trials to inform reaction times, and used a success rates of 40–80% (Bustamante et al., 2014; Choi et al., 2012; Costumero et al., 2013; Jung et al., 2011; Kirk et al., 2015; Weiland et al., 2014). The remaining studies did not report the task success rate, but reported participants’ were successful on approximately 55‐80% of trials (Behan, Stone, & Garavan, 2015; Boecker et al., 2014).

Six studies reported non‐significant results, including five for reward outcome and one for loss outcome (see Table 1). As the ALE method cannot incorporate null findings, these were excluded from the analysis. We thereby examined 49 studies for reward anticipation (609 foci, 1,082 participants), 32 for loss anticipation (333 foci, 681 participants) and 22 for reward outcome (264 foci, 691 participants).

ALE analysis

We ran nine different analyses to identify areas of increased activation likelihood during reward anticipation, loss anticipation, reward outcome, and common and distinct regions between reward and loss anticipation, as well as between reward anticipation and outcome (Table 2). Each contrast used in the analysis has been assigned a label (e.g. RA, LA‐RA) to assist in keeping track of which contrast is being referred to in the results.

Table 2

Types of contrasts used for each ALE analysis

Analysis type	Legend	Contrast used for ALE analysis	N
Single analysis
	RA	Reward anticipation [reward anticipation vs neutral anticipation/no‐incentive anticipation]	49
	LA	Loss anticipation [loss anticipation vs neutral anticipation/no‐incentive anticipation]	32
	RO	Reward outcome [successful reward outcome vs unsuccessful reward outcome or successful reward outcome vs neutral outcome]	22
Conjunction analysis
	RA+LA	Reward anticipation (RA) and loss anticipation (LA)	81
	RA+RO	Reward anticipation (RA) and reward outcome (RO)	71
Subtraction analysis
	RA‐LA	Reward anticipation (RA) relative to loss anticipation (LA)	81
	LA‐RA	Loss anticipation (LA) relative to reward anticipation (RA)	81
	RA‐RO	Reward anticipation (RA) relative to reward outcome (RO)	71
	RO‐RA	Reward outcome (RO) relative to reward anticipation (RA)	71

No loss outcome contrasts were analysed as too few studies reported the results of contrasts investigating this construct. Only contrasts that produced significant findings were able to be included.

First, we performed an ALE analysis on each single contrast type (contrast RA, LA and RO). The result of reward anticipation (contrast RA) revealed a large cluster of brain regions around the striatal‐thalamic regions of the brain, encompassing the ventral, including the nucleus accumbens (NAcc), and the dorsal striatum, extending to the bilateral amygdala (Figure 2a). The insula, supplementary motor area, premotor cortex, occipital cortex, and cuneus were other regions of note where greater likelihood of activation was seen during reward anticipation (Table 3). When this analysis was repeated using the eleven studies that only used a low magnitude reward (Behan et al., 2015; Carl et al., 2016; Choi et al., 2012; Damiano et al., 2014; Dillon et al., 2010; Figee et al., 2011; Jung et al., 2011; Kappel et al., 2013; Knutson, Bhanji, Cooney, Atlas, & Gotlib, 2008; Pfabigan et al., 2014; Romanczuk‐Seiferth, Koehler, Dreesen, Wüstenberg, & Heinz, 2015), we also found increased activation likelihood in the bilateral ventral and dorsal striatum, left supplementary motor and premotor areas and right insula (Table S1, Supporting Information).

Figure 2

ALE single analysis results. (a) Single analysis of reward anticipation (contrast LA). (b) Single analysis of loss anticipation (contrast LA). (c) Single analysis of reward outcome (contrast RO). Single analyses were conducted with a cluster‐forming threshold of p < .001 uncorrected and a cluster‐level threshold of p < .05 [Color figure can be viewed at http://wileyonlinelibrary.com]

Table 3

ALE results for activations during reward anticipation (contrast RA)

Region	Left/Right	x	y	z	ALE (10−3)	Volume (mm3)
Ventral striatum	Right	12	10	−4	104.29	28,368
Ventral striatum	Left	−10	10	−6	91.52
Thalamus	Right	4	−18	8	35.50
Thalamus	Left	−8	−16	8	31.98
Thalamus	Left	−4	−8	8	30.19
Amygdala	Right	16	−4	−14	28.59
Midbrain	Left	−4	−24	−10	25.60
Supplementary motor area	Right	2	6	52	38.60	4,168
Supplementary motor area	Right	2	10	48	37.03
Anterior insula	Right	34	24	−2	51.60	3,256
Anterior insula	Left	−40	14	−8	25.19	2,416
Anterior insula	Left	−34	22	4	24.16
Anterior insula	Left	−32	22	−10	17.83
Occipital cortex	Left	−30	−92	8	27.68	1,680
Occipital cortex	Left	−18	−96	−6	20.24
Premotor cortex	Left	−38	−12	52	25.59	1,520
Occipital cortex	Right	32	−92	6	27.17	1,288
Occipital cortex	Right	26	−94	2	25.22
Premotor area	Right	44	0	48	21.88	1,024
Cuneus	Right	52	4	48	19.61

All results are significant at a cluster‐forming threshold of p < .001 uncorrected and a cluster‐level threshold of p < .05.

Second, loss anticipation (contrast LA) also showed a cluster of increased likelihood of activation in striatal‐thalamic‐amygdala regions (Figure 2b). Greater probability of activation also was found in the insula, supplementary motor area, premotor cortex and occipital cortex (Table 4).

Table 4

ALE results for activations during loss anticipation (contrast LA)

Region	Left/Right	x	y	z	ALE (10−3)	Volume (mm3)
Ventral striatum	Left	−12	8	−4	44.14	11,768
Thalamus	Left	−12	−2	12	34.94
Thalamus	Left	−10	−2	4	33.96
Thalamus	Left	−8	−18	0	26.41
Midbrain	Right	6	−18	−4	20.48
Thalamus	Right	4	−14	10	18.64
Ventral striatum	Right	12	6	0	66.33	7,728
Caudate	Right	14	0	14	25.69
Amygdala	Right	20	0	−14	15.12
Anterior insula	Left	−32	24	0	20.10	2,016
Supplementary motor area	Right	4	2	52	25.22	1,896
Supplementary motor area		0	2	60	21.19
Anterior insula	Right	34	24	2	25.83	1,824
Anterior insula	Right	38	18	−6	19.08
Cerebellum	Right	6	−62	−14	18.16	856

All results are significant at a cluster‐forming threshold of p < .001 uncorrected and a cluster‐level threshold of p < .05.

Finally, the contrast of reward outcome (contrast RO) showed a large cluster in the ventral striatum which extended into the subcallosal, followed by another large cluster in the OFC/vmPFC, and a smaller clusters in the dorsal PCC and amygdala cortex (Figure 2c and Table 5).

Table 5

ALE results for activations during reward outcome (contrast RO)

Region	Left/Right	x	y	z	ALE (10−3)	Volume (mm3)
Ventral striatum	Right	12	10	−10	47.21	4,336
Amygdala	Right	22	−2	−14	21.29
OFC/vmPFC	Left	−2	42	−6	47.21	4,336
OFC/vmPFC	Right	2	44	−10	21.29
OFC/vmPFC	Left	−6	52	−14	24.77	3,312
Ventral striatum	Left	−14	8	−8	24.00
Amygdala	Left	−18	0	−16	23.69
PCC	Right	2	−36	36	31.66	2,368

OFC = orbitofrontal cortex, vmPFC = ventro‐medial prefrontal cortex, PCC = posterior cingulate cortex. All results are significant at a cluster‐forming threshold of p < .001 uncorrected and a cluster‐level threshold of p < .05.

Reward and loss anticipation

An ALE conjunction analysis was run to identify regions engaged in both reward and loss anticipation contrasts. Reward and loss anticipation (contrast RA + LA) showed overlap in two large clusters of greater activation probability in the striatal‐thalamic region, encompassing the dorsal and ventral striatum and the amygdala, followed by smaller clusters in the anterior insula and supplementary motor area (Figure 3c, Table 6). Additionally, the region of the ventral striatum where overlapping activation between reward and loss anticipation resided is considered to be part of the NAcc core (Xia et al., 2017).

Figure 3

ALE results for the conjunction and subtraction analysis between of reward anticipation and loss anticipation. (a) Subtraction analysis of reward anticipation relative to loss anticipation (contrast RA‐LA). (b) Subtraction analysis of loss anticipation relative to reward anticipation (contrast LA‐RA). (c) Conjunction analysis of reward anticipation and loss anticipation (contrast RA + LA). Subtraction analyses were conducted with a significance level of p < .005 [Color figure can be viewed at http://wileyonlinelibrary.com]

Table 6

ALE results for conjunction analyses between reward and loss anticipation (contrast RA + LA) and reward anticipation and outcome (contrast RA + RO)

Region	Left/Right	x	y	z	ALE (10−3)	Volume (mm3)
Conjunction of reward and loss anticipation (contrast RA+LA)
Ventral striatum	Left	−12	8	−4	44.14	8,176
Thalamus	Left	−8	−6	10	24.04
Thalamus	Left	−6	−18	4	20.44
Thalamus	Right	4	−14	10	18.64
Thalamus	Right	6	−18	−2	18.60
Midbrain	Left	−4	−24	−4	16.01
Ventral striatum	Right	12	6	0	66.33	5,816
Amygdala	Right	20	0	−14	15.12
Anterior insula	Right	34	24	2	25.83	1,440
Anterior insula	Right	38	18	−6	19.08
Anterior insula	Left	−32	22	0	18.81	1,168
Anterior insula	Left	−30	22	−10	15.08
Supplementary motor area	Right	4	2	52	25.10	952
Thalamus	Right	6	−6	12	12.63	8
Conjunction of reward anticipation and reward outcome (contrast RA+RO)
Ventral striatum	Right	12	10	−10	47.21	3,176
Amygdala	Right	20	−2	−14	20.95
Ventral striatum	Left	−14	8	−8	31.66	2,064
Amygdala	Left	−18	0	−16	16.92

Reward anticipation and loss anticipation showed no difference in their neural substrates (contrasts RA‐LA and LA‐RA). At a less conservative threshold of p < .005, loss anticipation in contrast to reward anticipation activated the caudate body/medial‐dorsal thalamus (Figure 3a, Table 7), while reward anticipation contrasted with loss anticipation engaged the left supplementary motor area, left occipital lobe and a part of the right ventral NAcc which corresponds to the NAcc shell (Figure 3b, Table 7) (Xia et al., 2017).

Table 7

ALE results for subtraction analyses between reward anticipation and loss anticipation (contrast RA‐LA and contrast LA‐RA)

Region	Left/Right	x	y	z	Z‐score	Volume (mm3)
Reward anticipation relative to loss anticipation (contrast RA‐LA)
Occipital cortex	Left	−32	−90	10	2.99	264
Occipital cortex	Left	−32	−88	6	2.95
Supplementary motor area	Left	−6.7	7.3	45.3	3.24	256
Ventral striatum	Right	8	12	−9.3	2.75	176
Ventral striatum	Right	12	10	−14	2.69
Loss anticipation relative to reward anticipation (contrast LA‐RA)
Caudate	Right	10	2	14	3.35	320
Caudate	Right	14	0	12	3.16

All results are significant at p < .005.

Reward anticipation and outcome

The conjunction analysis examining common regions to reward anticipation and outcome (contrast RA + RO) showed engagement the ventral striatum and amygdala (Figure 4c, Table 6). Subtraction analyses showed, for (i) reward anticipation versus reward receipt (contrast RA‐RO), activity of the caudate/dorsal striatum, supplementary motor area, thalamus and right anterior insula (Figure 4a, Table 8); while (ii) reward receipt versus reward anticipation (contrast RO‐RA) engaged the OFC/vmPFC and PCC (Figure 4b, Table 8).

Figure 4

ALE results for the conjunction and subtraction analysis between reward anticipation and reward outcome. (a) Subtraction analysis of reward anticipation relative to reward outcome (contrast RA‐RO). (b) Subtraction analysis of reward outcome relative to reward anticipation (contrast RO‐RA). (c) Conjunction analysis of reward anticipation and reward outcome (contrast RA + RO). Subtraction analyses were conducted with a significance level of p < .001 [Color figure can be viewed at http://wileyonlinelibrary.com]

Table 8

ALE results for the subtraction analyses between reward anticipation and outcome (contrast RA‐RO and contrast RO‐RA)

Region	Left/Right	x	y	z	Z‐score	Volume (mm3)
Reward anticipation relative to reward outcome (contrast RA‐RO)
Supplementary motor area	Right	2.6	4.5	55.3	3.72	2,640
Supplementary motor area	Left	−1.0	0.8	51.9	3.54
Supplementary motor area	Left	−8.2	−0.6	52.4	3.35
Ventral striatum	Right	10.2	1.9	1.3	3.72	1,880
Dorsal striatum	Left	−8.0	9.5	4.0	3.35
Anterior insula	Right	27.7	23.0	−2.3	3.72	1,344
Anterior insula	Right	31.3	24.6	−1.0	3.54
Thalamus	Left	−12.2	−11.2	5.6	3.72	616
Thalamus	Left	−13.3	−18.0	6.7	3.54
Reward outcome relative to reward anticipation (contrast RA‐RO)
OFC/vmPFC	Left	−1.9	47.8	−10.5	3.72	2,936
PCC	Right	2.9	−36.5	36.5	3.72	600

OFC = orbitofrontal cortex, vmPFC = ventro‐medial prefrontal cortex, PCC = posterior cingulate cortex. All results are significant at p < .001.

Additional ALE analyses

We ran post‐hoc analyses to disentangle the potential impact of several possible confounders that may have biased the results (these are presented in Tables S2‐S4, Supporting Information). The inclusion of reward/loss versus no‐incentive contrasts might be biased by action planning, which has a central role in appetitive motivation. We therefore reran all single and conjunction/subtraction analyses presented thus far (i.e. all those presented in Table 2) after excluding the two studies using non‐incentive trials instead of neutral trials. The results remained unaltered, with activation of striatal‐thalamic, insula, amygdala and supplementary motor regions during reward and loss anticipation (Table S2, Supporting Information).

Second, we aimed to disentangle whether the specificity of OFC/vmPFC activation for feedback depended on either the length of interval between reward‐anticipatory cue and feedback or the jittered time interval in trials between cue and feedback. We therefore contrasted studies that had an anticipation duration (time from initial exposure to the cue to the presentation of the target) of less than 3000ms (short duration) with those which had a duration of between 3000ms to 6000ms (long duration) for reward anticipation contrasts. Eighteen studies used an anticipation phase of a short duration (Bjork et al., 2004; Bustamante et al., 2014; Costumero et al., 2013; Enzi et al., 2012; Juckel, Schlagenhauf, Koslowski, Wüstenberg, et al., 2006; Kappel et al., 2013; Knutson, Adams, et al., 2001; Knutson et al., 2008, 2003; Knutson, Fong, et al., 2001; Maresh, Allen, & Coan, 2014; Mori et al., 2016; Mucci et al., 2015; Rademacher et al., 2010; Saji et al., 2013; Wrase et al., 2007; Yan et al., 2016) whereas 22 used a long duration (Carter et al., 2009; Choi et al., 2012; Damiano et al., 2014; Funayama et al., 2014; Hägele et al., 2015; Juckel et al., 2012; Jung et al., 2011; Kaufmann et al., 2013; Kirk et al., 2015; Pfabigan et al., 2014; Samanez‐Larkin et al., 2007; Schlagenhauf et al., 2008; Stoy et al., 2011, 2012; Treadway, Buckholtz, & Zald, 2013; Weiland et al., 2014; Wu, Samanez‐Larkin, Katovich, & Knutson, 2014; Yau et al., 2012). Within each condition, studies with both a long and short duration showed increased activation likelihood in striatal, amygdala and insula regions. Studies that used a short duration also showed increased likelihood in supplementary motor, premotor and dorsolateral prefrontal areas while those that used longer durations had occipital and posterior thalamic activation. The conjunction analysis of long and short reward anticipation durations found increased activation likelihood in striatal, amygdala and insula areas (see Table S3, Supporting Information). The subtraction analysis found no significant differences.

Of the 22 studies that examined reward receipt in some form, 13 used a contrast of successful reward outcome versus unsuccessful reward outcome (Bjork et al., 2004, 2010, 2012; Bjork, Smith, & Hommer, 2008; Boecker et al., 2014; Jung et al., 2011; Kirk et al., 2015; Knutson et al., 2008, 2003; Romanczuk‐Seiferth et al., 2015; Wu et al., 2014; Yan et al., 2016) and nine used successful reward outcome versus neutral trial outcome (Bustamante et al., 2014; Carl et al., 2016; Damiano et al., 2014; Dillon et al., 2010; Figee et al., 2011; Filbey et al., 2013; Hanssen et al., 2015; Mucci et al., 2015). To check if there was a difference between these two contrast types an additional ALE analysis was done. First, condition‐specific analyses were performed on each contrast type. Successful reward outcome versus unsuccessful reward outcome showed significant activations in the ventral striatum, OFC/vmPFC, PCC, the right amygdala and in the medial prefrontal cortex area (Table S4, Supporting Information), which correspond to similar regions as the combined contrast (successful reward outcome versus unsuccessful reward outcome and successful reward outcome versus neutral trial outcome). However, the single ALE analysis for successful reward outcome versus neutral trial outcome revealed no significant activation, precluding a subsequent conjunction/subtraction analysis.

DISCUSSION

We performed a neuroimaging ALE meta‐analysis of results from studies using the MIDT, to examine brain regions that are implicated in reward anticipation, loss anticipation, as well as reward outcome. First, we found considerable overlaps in the networks activated during reward and loss anticipation, including the dorsal and ventral striatum, amygdala, insula and supplementary motor cortex. Second, we identified neural networks engaged by reward receipt—including the ventral striatum, the OFC/vmPFC, amygdala, and PCC.

Common neural substrates of reward and loss anticipation

We found that anticipation of reward and loss have a common neural substrate (i.e., striatum, thalamus, insula and amygdala). Previous meta‐analyses reported partly comparable results as the striatum and thalamus were found to be common to both processes, but not the amygdala and insula (Bartra et al., 2013; Knutson & Greer, 2008; Liu et al., 2011). Previous studies also implicate the striatum in the anticipation of positive or negative outcomes, regardless of stimulus type—monetary (Carter et al., 2009; Delgado, Jou, Ledoux, & Phelps, 2009; Guitart‐Masip et al., 2011; Pfabigan et al., 2014; Stark et al., 2011; Tom, Fox, Trepel, & Poldrack, 2007), social (Kohls et al., 2013), or physical (Delgado, Jou, & Phelps, 2011). Overall, these and our results suggest that a generalised system—centred on the striatum—is implicated in approach and avoidance behaviour and has considerable overlap with what is typically considered to be reward circuitry (Brooks & Berns, 2013; Jensen et al., 2003; Kohls et al., 2013; Lammel, Lim, & Malenka, 2014; Pfabigan et al., 2014; Salamone & Correa, 2012).

The ventral striatum has been conventionally ascribed to reward processing primarily, but mounting evidence—including the results from this study—points to broader functions (Brooks & Berns, 2013; Lammel et al., 2014; Salamone & Correa, 2012). The ventral striatum (and mesolimbic dopamine system more generally) has been linked to motivational processes independent of stimulus valence, such as behavioural activation, exertion of effort and sustained task engagement (Boureau & Dayan, 2011; Salamone & Correa, 2012). Many of these motivational processes are required during the anticipation trials of the MIDT to perform the task successfully (Pfabigan et al., 2014). As these processes occur in the MIDT, ventral striatal activity is robustly observed when such processes are occurring, and the ventral striatum is associated with these motivational processes, our results are aligned with evidence that the ventral striatum is involved in the motivational aspect of valence independent outcome anticipation (Kohls et al., 2013; Mogenson, Jones, & Yim, 1980; Pfabigan et al., 2014; Salamone & Correa, 2012). Also supporting this, ventral striatal activity has also previously been linked to the amount of effort required to obtain an outcome (Bjork & Hommer, 2007; Guitart‐Masip et al., 2011; Kurniawan, Guitart‐Masip, Dayan, & Dolan, 2013). Overall, these findings suggest that a general motivation system—with the striatum playing a critical role—is engaged when anticipating either rewards or losses.

Ventral striatal activity during both reward and loss anticipation was localised to the dorsolateral NAcc, which is recruited when anticipating salient stimuli, irrespective of their valence (Zink, Pagnoni, Chappelow, Martin‐Skurski, & Berns, 2006; Zink, Pagnoni, Martin‐Skurski, Chappelow, & Berns, 2004; Zink, Pagnoni, Martin, Dhamala, & Berns, 2003) and has a key role in motivational processes (Salamone & Correa, 2012). Recent work has identified the dorsolateral proportion of the NAcc to correspond to the NAcc core (Xia et al., 2017), a region thought to encode the motivational salience of stimuli, and which mediates cognitive functions such as orientation towards the stimuli, cognitive control, action selection, and effort (Bromberg‐Martin, Matsumoto, & Hikosaka, 2010). Thus, the NAcc, and specifically its core, may be a key hub for motivational processing in response to potentially rewarding or aversive stimuli.

Though previous MIDT work has largely focused on the ventral striatum (Balodis & Potenza, 2015), our findings also implicate the dorsal striatum in reward and loss processing, aligning with previous work (Balleine, Delgado, & Hikosaka, 2007; Palminteri et al., 2012). While anticipating an outcome, the ventral striatum encodes the expected value of the outcome (Delgado, Li, Schiller, & Phelps, 2008; Jensen et al., 2003, 2007; O'Doherty et al., 2004; Schönberg, et al., 2007). These expected value signals subsequently modulate motivational processes, potentially via the dorsal striatum (den Ouden, Kok, & de Lange, 2012; Jensen et al., 2007; Kim, 2013; Kurniawan et al., 2013; Miller, Shankar, Knutson, & McClure, 2014; Niv, Daw, Joel, & Dayan, 2007; Schmidt, Lebreton, Cléry‐Melin, Daunizeau, & Pessiglione, 2012; Vassena et al., 2014), which is involved in motivational processes such as selecting/initiating motor responses aimed to achieve optimal outcomes (Atallah, Lopez‐Paniagua, Rudy, & O'Reilly, 2007; Balleine et al., 2007; Kurniawan et al., 2013; O'Doherty et al., 2004; Palminteri et al., 2012).

The anticipation of rewards and losses engaged other brain areas including the anterior insula and the thalamus (and amygdala as discussed further below), in line with previous work (Bartra et al., 2013; Diekhof et al., 2012; Knutson & Greer, 2008; Liu et al., 2011). The anterior insula is implicated in responding to outcome uncertainty (Bossaerts, 2010; Rolls, McCabe, & Redoute, 2008), a key element in the MIDT as the success rate is pre‐set so that a set so participants can never be certain they will succeed on any given trial. Similarly, the thalamus plays a role in anticipating rewards and losses. It has been suggested that the thalamus represents an “alerting” signal to respond to valanced stimuli, which converges with introspective information from the insula to the striatum, where an appropriate action response is selected (Cho et al., 2013).

This is the first time for a meta‐analysis to detect amygdala activation during both reward and loss anticipation, as this region is typically associated with loss processing. Our finding is consistent with evidence that suggests the amygdala responds to stimulus arousal, rather than stimulus valence (Anderson et al., 2003; Haber & Knutson, 2010; Hardin et al., 2009; Small et al., 2003). It is possible that anticipation of an uncertain outcome engages arousal to enhance spatial attention towards valence‐related stimuli (Ousdal et al., 2014; Peck & Salzman, 2014) and maximise task performance (i.e., gaining rewards and avoiding losses).

Overall, our results fit with findings that show signals within the anterior insula, thalamus, and amygdala project to and converge within the ventral striatum (Haber & Knutson, 2010) to update predictions of the best outcomes and allow appropriate action selection, while the dorsal striatum selects the optimal choice (Atallah et al., 2007; Cho et al., 2013; Delgado et al., 2008; Mogenson et al., 1980; O'Doherty et al., 2004). All of these elements putatively form a general motivation system in the brain that devises a response to a stimulus regardless of its valence.

The anticipation of an outcome did not recruit the OFC/vmPFC area in our study, which contrasts previous results (Bartra et al., 2013; Liu et al., 2011). This may be due to the fact that the OFC/vmPFC is recruited in reward tasks where one must select from multiple options/choices (Doll, Simon, & Daw, 2012; Hampton, Bossaerts, & O'Doherty, 2006; Peters & Büchel, 2010), and in meta‐analyses including reward processing tasks with decision making/comparative elements OFC/vmPFC activity was found during anticipation (Bartra et al., 2013; Clithero & Rangel, 2014; Liu et al., 2011). Our results on the lack of OFC/vmPFC recruitment are consistent with those from studies using tasks that require no selection between distinct options (Plassmann, O'Doherty, & Rangel, 2007; Schoenbaum, Takahashi, Liu, & Mcdannald, 2011) such as the MIDT, and in meta‐analyses of tasks/contrasts that do not include decision making elements (Diekhof et al., 2012; Knutson & Greer, 2008). Thus, the OFC/vmPFC may be implicated only in reward processing tasks requiring selection between multiple choices and decision making.

Additionally, the uncertainty in achieving an outcome on the MIDT may also explain why the OFC/vmPFC were not involved during the anticipation stage. If it is highly likely that a reward will be obtained following the presentation of a cue, the cue will take on the property of the received reward, as per classical conditioning principles. Thus, the cue will activate regions involved during reward outcome, e.g., the OFC/vmPFC (Diekhof et al., 2012). In support of this, one study which used a similar paradigm to the MIDT, but in which the outcome was highly certain, reported OFC/vmPFC activity during reward anticipation (Kim et al., 2011). Given enough time, individuals may also begin to mentally imagine obtaining the reward and this mental stimulation may also activate the OFC/vmPFC (Bray, Shimojo, & O'Doherty, 2010). In a task like the MIDT, the anticipation stage is relatively short, preventing participants from engaging in such mental activities (Diekhof et al., 2012).

We found no differences between reward and loss anticipation at our nominal significance threshold, but some differences emerged with a more liberal threshold (p < .005 as opposed to p < .001) during anticipation of reward but not loss. Specifically, reward anticipation engaging an area of the ventral striatum possibly corresponding to the NAcc shell (Xia et al., 2017). NAcc shell activity has been ascribed to reward processing specifically and may reflect learning of the reward value (Bromberg‐Martin et al., 2010) and hedonic response to the stimuli (Castro & Berridge, 2014; Pecina & Berridge, 2005).

When investigating loss anticipation (versus reward anticipation) using the reduced statistical threshold, we found increased activation likelihood in the caudate body/dorsal‐medial thalamus. This may represent the prediction of an aversive outcome, as existing work shows that this area codes punishment‐based prediction errors (Mattfeld, Gluck, & Stark, 2011). However further characterisation of loss processing is needed to determine what the activity in this area may indicate.

Additional analyses were performed to examine the impact of various potential confounds. We found no evidence for a bias associated with inclusion of studies that compared reward/loss anticipation to non‐incentive MIDT trials. We also found that the exclusion of studies that used a non‐incentive trial did not alter the findings. The use of short vs long durations in the anticipation phase were also not associated with differences in activation, but we could only run this analysis on a relatively small dataset (∼20 studies in each condition), so further confirmation is required. Identifying the impact of the anticipatory phase duration on subsequent brain activity facilitate optimization of task design (e.g., in determining optimal trial length) and shed light on how anticipatory activity unfolds over time.

Brain activity during reward outcome

Reward outcome was linked to activity in a range of brain regions including the ventral striatum, OFC/vmPFC, PCC, subcallosal cortex, and thalamus, consistent with previous evidence (Bartra et al., 2013; Clithero & Rangel, 2014; Diekhof et al., 2012; Knutson & Greer, 2008; Liu et al., 2011). Of these regions, increased likelihood of OFC/vmPFC and PCC activity was observed only during reward outcome and not during reward anticipation.

Ventral striatal activity during rewarding outcomes has been observed consistently (Bartra et al., 2013; Clithero & Rangel, 2014; Diekhof et al., 2012; Knutson & Greer, 2008; Liu et al., 2011). This ventral striatal activity likely represents a prediction error signal that tracks the difference between the expected and received reward (Haber & Knutson, 2010; Knutson, Fong, et al., 2001) when contingencies are uncertain (McClure, York, & Montague, 2004; Yacubian et al., 2006). Such a signal would be critical for reward processing and learning, increasing the likelihood a behaviour that leads to a better than expected outcome will be repeated (McClure et al., 2004; Schultz &Dickinson, 2007; Yacubian et al., 2006), like in the MIDT.

We found that the anterior portion of the prefrontal cortex—including the OFC/vmPFC and subcallosal cortex—were implicated in reward outcome (but not reward anticipation). This is in line with previous findings that implicate these areas in the receipt of abstract rewards, like money (Kringelbach & Rolls, 2004; Sescousse et al., 2013). The OFC/vmPFC may represent the subjective value of a received reward (Bartra et al., 2013; Diekhof et al., 2012; Haber & Knutson, 2010; Levy & Glimcher, 2012; Peters & Büchel, 2010) and the subcallosal cortex may represent the subjective experience of pleasure (Anderson et al., 2003), especially as its dysfunction is linked to anhedonia (Hamani et al., 2011; Young et al., 2016). While the OFC and vmPFC are often presented as synonymous in function (Levy & Glimcher, 2012; Rushworth et al., 2011), they may have dissociable roles. The OFC is potentially engaged in processing the value of a gain, and the vmPFC in encoding/strengthening the relationship between a stimulus and outcome (Walton, Chau, & Kennerley, 2015). However, research into the distinct roles of these regions is limited, thus this will need to be addressed in order to properly understand reward/loss processing.

Receiving a reward also implicated the PCC, consistent with previous work and further suggests this region is crucial for reward processing (Bartra et al., 2013; Clithero & Rangel, 2014; Diekhof et al., 2012; Liu et al., 2011; Pearson, Heilbronner, Barack, Hayden, & Platt, 2011). The PCC is implicated in monitoring the environment and remembering past outcomes, thus may track the outcomes achieved and signal required behavioural changes when optimal outcomes are no longer being achieved (Nakao, Ohira, & Northoff, 2012; Schacter, Addis, & Buckner, 2007). This information would be projected to prefrontal regions, which devises new responses to maximise outcomes (Pearson, Hayden, Raghavachari, & Platt, 2009; Pearson et al., 2011). Notably, the PCC is linked to self‐referential activation and is part of the default mode network (Brewer, Garrison, & Whitfield‐Gabrieli, 2013; Northoff et al., 2006), thus PCC activity may represent positive self‐appraisal upon receipt of a reward.

Limitations and future use of the MIDT

While the MIDT avoids several potential cognitive confounds, there are still some limitations in using the task to understand reward and loss processing (Balodis & Potenza, 2015; Limbrick‐Oldfield, Van Holst, & Clark, 2013). First, using the neutral trial as a baseline to contrast reward and loss trials might not be optimal as a successful neutral trial may be experienced as rewarding. For example, a participant might find performing the task intrinsically rewarding, regardless if a monetary outcome is at stake (Lutz & Widmer, 2014). This may elicit a similar neural response to both neutral and reward/loss cues. The structure of the MIDT could also affect brain activity during neutral trials. For example, neutral outcomes could have a positive valence (i.e., absence of loss) or a negative valence (i.e., absence of reward) if loss trials are or are not present in the task, respectively (Hardin et al., 2009; Limbrick‐Oldfield et al., 2013; Nieuwenhuis et al., 2005).

Individual differences like learning rate could also produce different neural activity. Individuals who are still learning how the MIDT works would be expected to show less anticipatory but greater outcome activity, as they have not developed the proper neural responses needed to successfully predict the outcome, compared to those who have learnt the task (Balodis & Potenza, 2015). Computational modelling may be able to be used to assess how learning changes over the course of the task (O'Doherty, et al., 2007).

Finally, while the MIDT is cognitively simple task there may be some cognitive factors which could be further addressed. For instance, during the anticipation stage of the standard MIDT (Knutson et al., 2000) there is activity related to both motivational processes and the prediction/expectation of an outcome (Limbrick‐Oldfield et al., 2013). It is possible to develop the task to have a secondary anticipatory stage occurring after the participant responds to the target, but prior to feedback delivery (Andrews et al., 2011; Balodis et al., 2012; Bjork et al., 2012). Characterising the differences between these two types of anticipation will help more clearly disentangle motivational processes, which should not need recruiting after a response is made, from prediction/expectation signals. Another possible confound could be working memory demands due to participants having to remember what the different symbols in the MIDT mean, but this could be addressed by using words instead (Andrews et al., 2011; Balodis et al., 2014).

Finally, many of the meta‐analysed studies used gradient‐echo sequences, which are vulnerable to MR susceptibility artefacts (Ojemann et al., 1997) and could have caused signal loss in areas targeted by the MIDT that are close to sinuses e.g., ventral striatum, amygdala, OFC/vmPFC (O'Doherty, 2009). Only a few studies (Adcock, Thangavel, Whitfield‐Gabrieli, Knutson, & Gabrieli, 2006; Knutson et al., 2008; Weiland et al., 2014; Wu et al., 2014; Yau et al., 2012) minimised these artefacts using a spiral in/out sequence (Glover & Law, 2001), and we recommend future MIDT studies to use such susceptibility artefact‐reducing methods (e.g., spin‐echo, z‐shimming, optimisation of the echo time and spatial resolution) to further refine the spatial specificity, though other factors should be considered if choosing to employ such artefact‐reduction methods (e.g., for instance spin‐echo can result in a loss of sensitivity at field strengths of 3T or below)(Glover, 1999; Norris, 2012; Weiskopf, Hutton, Josephs, Turner, & Deichmann, 2007).

Despite these limitations, the MIDT is a valuable tool to investigate reward and loss processing in both healthy and psychiatric populations (Balodis & Potenza, 2015; Knutson & Heinz, 2015; Lutz & Widmer, 2014), especially as many of the aforementioned issues could be addressed by studies through careful design or improved analysis techniques. The MIDT has been used to investigate the neural correlates of reward processing in a number of psychiatric disorders (Knutson & Heinz, 2015) including: substance abuse (Balodis et al., 2016; Balodis & Potenza, 2015; Bjork et al., 2008; Jia et al., 2011; Patel et al., 2013), pathological gambling (Balodis et al., 2012; Romanczuk‐Seiferth et al., 2015), depression (Knutson et al., 2008), schizophrenia (Juckel, Schlagenhauf, Koslowski, Wüstenberg, et al., 2006), binge eating (Balodis et al., 2014, 2013), and ADHD (Stoy et al., 2011). In addition to investigating psychiatric disorders, the MIDT has also been used to investigate both typical development of reward and loss processing neurobiology (Bjork et al., 2004, 2010; Cho et al., 2013; Lutz & Widmer, 2014; Rademacher, Salama, Gründer, & Spreckelmeyer, 2014; Richards et al., 2013; Samanez‐Larkin et al., 2007), as well as atypical development (Guyer et al., 2006; Peters et al., 2011; Schneider et al., 2012). The results of the present study should assist researchers in establishing what brain activations are expected when the MIDT is used and inform new hypothesis/directions of research.

Our results further stress the importance of examining the temporal phases (i.e. anticipation and receipt) of reward/loss processing separately, as we showed dissociable neural substrates. This is particularly relevant for psychiatric conditions where distinct temporal phases or reward and loss are affected. For instance, relative to controls, individuals with substance addiction show decreased and increased striatal activation during reward anticipation and receipt, respectively (Luijten, Schellekens, Kühn, Machielse, & Sescousse, 2017). Additionally, different valences should also be used. A dysfunctional ventral striatal response to both reward and loss anticipation has been observed in patients with substance abuse (Balodis & Potenza, 2015), problem gambling (Balodis et al., 2012; Romanczuk‐Seiferth et al., 2015), and schizophrenia (Juckel, Schlagenhauf, Koslowski, Filonov, et al., 2006). As our results show reward and loss anticipation are neurobiological similar, this altered activity may be caused by abnormalities in the general motivation system rather than deficits specific to one type of valence (Salamone & Correa, 2012). To properly evaluate this, both reward and loss trials should be used and analysed when employing the MIDT, especially when psychiatric populations are being investigated, to further characterise similarities and differences between reward and loss processing neurobiology. Thus, investigating the interaction between valence and temporal phase is likely to lead to much more insightful results regarding psychiatric populations.

Limitations

The ALE method is a coordinate‐based meta‐analysis (CBMA) technique which only incorporates information regarding sample size and points of activation. An optimal approach would use an image‐based meta‐analysis, which uses the full statistical parametric map when conducting the analysis, avoiding the information loss that is inherent to CBMA techniques, allowing for a more powerful and detailed analysis (Salimi‐Khorshidi, Smith, Keltner, Wager, & Nichols, 2009). However, conducting such a meta‐analysis is currently problematic, as statistical parametric maps from other studies are often difficult to acquire. Recent advances in neuroimaging databases (Poldrack & Gorgolewski, 2014), will hopefully make obtaining such data significantly easier. This is an opportunity that research teams investigating the neurobiology of reward and loss processing should seek to implement in future. Also, as the ALE approach does not take account of null findings, we did not include the results from a small number of studies that reported no activations during reward outcome. This may have caused a slight bias towards positive findings.

We could not investigate MIDT studies that had examined loss received against loss avoided/neutral outcome as too few had conducted or reported this contrast. Future studies should aim to explore this contrast to improve our understanding of reward and loss processing.

Another limitation is that we did not extensively examine the impact of reward magnitude on the results, which may increase activity in some regions (ventral striatum) but not other regions (e.g., insula, amygdala) (Diekhof et al., 2012; Knutson, Adams, et al., 2001). Unfortunately, few studies reported results using a single magnitude, and most studies using multiple magnitudes collapsed these trials together without segregating individual magnitudes. Thus, we cannot rule out that our results in some regions may be partly driven studies that used larger magnitudes in their design. Nonetheless, we observed striatal (both dorsal and ventral), amygdala, anterior insula and supplementary motor area activation during reward anticipation using a low magnitude (for the 11 studies which used a low magnitude), corroborating the validity of the reported effects on the striatum.

Finally, we included studies with heterogeneous success rates. Most studies (n = 38) used a 66–67% hit rate, but the other studies used hit rates of anywhere between 40–80%, preventing a direct comparison of the varying hit rate level (as too few studies used a consistent hit rate other than 66–67%). Theses varying rates may affect the probability and uncertainty of outcomes, and ultimately neuronal activity (Fiorillo, Tobler, & Schultz, 2003; O'Doherty et al., 2004). For example, more uncertain outcomes could produce increased ventral striatal activity during anticipation and more certain outcomes could produce reduced ventral striatal activity during receipt and possibly produced OFC/vmPFC activity during anticipation. A systematic investigation of the effect of hit‐rate will help to clarify these effects.

Summary

The present meta‐analysis used the MIDT to examine the neural substrates of reward and loss processing, in particular to assess if there was a specific or generalised system to these processes. We found the striatum, thalamus, amygdala and insula were activated during both reward and loss anticipation, indicating a generalised system is activated during this processing phase. This system likely plays a key role in generating motivation responses that allow an optimal outcome to be achieved. Unlike some previous studies, our results did not implicate the OFC/vmPFC during anticipation (but did during reward outcome), suggesting the OFC/vmPFC is engaged in the anticipation of rewards if there are multiple choices to be made or if outcomes are highly certain. Future research is warranted to further investigate the overlaps and differences between reward and loss processing to better understand learning, approach, and avoidance behaviours.

CONFLICT OF INTEREST

All authors report no conflict of interest.

Additional Supporting Information may be found online in the supporting information tab for this article.

Supplementary_Tables

Click here for additional data file.