Shifts in reinforcement signalling while playing slot-machines as a function of prior experience and impulsivity.

Electronic gaming machines (EGMs) offer significant revenue streams for mercantile gambling. However, limited clinical and experimental evidence suggests that EGMs are associated with heightened risks of clinically problematic patterns of play. Little is known about the neural structures that might mediate the transition from exploratory EGM play to the 'addictive' play seen in problem gamblers; neither is it known how personality traits associated with gambling activity (and gambling problems) influence reinforcement processing while playing EGMs. Using functional magnetic resonance imaging in healthy participants, we show that a single episode of slot-machine play is subsequently associated with reduced amplitudes of blood-oxygenation-level-dependent signals within reinforcement-related structures, such as the ventral striatum and caudate nucleus, following winning game outcomes; but increased amplitudes of anticipatory signals within the ventral striatum and amygdala while watching the game reels spin. Trait impulsivity enhanced positive signals within the ventral striatum and amygdala following the delivery of winning outcomes but diminished positive signals following the experience of almost-winning ('near-misses'). These results indicate that a single episode of slot-machine play engages the well-characterised reinforcement-learning mechanisms mediated by ascending dopamine mesolimbic and mesostriatal pathways, to shift reward value of EGMs away from game outcomes towards anticipatory states. Impulsivity, itself linked to problem gambling and heightened vulnerability to other addictive disorders, is associated with divergent coding of winning outcomes and almost-winning experiences within the ventral striatum and amygdala, potentially enhancing the reward value of successful slot-machine game outcomes but, at the same time, modulating the aversive motivational consequences of near-miss outcomes.

Introduction

Electronic gaming machines (EGMs) such as slot-machines and devices that deliver poker, lotteries, roulette and other casino games represent one of the most popular forms of gambling activity and constitute one of the most profitable revenue streams for commercial gambling outlets.[1, 2] Income per unit can be considerable: for example, 23 000 slot-machines in the state of Ontario generated ∼$3 135 660 000 during 2004, amounting to over $130 000 per machine.[3] As such, EGMs are known to generate a very significant proportion of revenues within casinos (∼80%) and other betting outlets.[4]

Notwithstanding their commercial value, EGMs may also represent a form of gambling activity with heightened ‘addictive-potential'.[5, 6] Increased availability of EGMs has been linked to the severity of gambling problems in the Australasian context,[7, 8] and have aroused significant public concern in other jurisdictions.[9] Tentative evidence suggests elevated rates of gambling problems, alongside other psychiatric difficulties, amongst individuals who play EGMs regularly,[10] as well as shorter intervals to the onset of problem gambling in individuals who choose EGMs over other forms of gambling.[11]

At the current time, little is known about how experience with EGMs engages the neural systems that support reinforcement learning in order to modulate the value of these games. Investigating this engagement can help us understand the transitions from exploratory play to the problematic play, sometimes observed in the clinic. Contemporary biological models of reinforcement learning provide a sound framework for investigating the impact of prior experience on slot-machine play.[12, 13] According to these models, as learning proceeds, activity of dopamine neurones within the ventral tegmental area (VTA) elicited by rewards tends to diminish, whereas activity evoked by cues that signal the imminent delivery of rewards tends to increase,[13] implementing a form of temporal-difference learning.[14] Consistent with this, experiments using functional magnetic resonance imaging (fMRI) in humans show that learning shifts reward-evoked activity within the striatum away from the delivery of reinforcers towards preceding conditioned stimuli or instrumental responses.[14, 15]

Therefore, in this experiment, we used fMRI to test the hypothesis that prior experience with a simplified slot-machine game is associated with shifts in reward-based neural activity within mesolimbic and striatal reinforcement sites away from the presentation of winning game outcomes towards the reel spins that precede them, reflecting the changing evaluation of anticipatory and consummatory aspects of the game. Understanding such shifts in reinforcement signalling can help us understand how experience of a single episode of slot-machine play might increase the likelihood of individuals returning to play these games subsequently.

Other research suggests an association between heightened impulsivity and addictive disorders, including problem gambling.[16] Both behavioural and clinical expressions of impulsivity enhance neural responses to delivered rewards within the ventral striatum,[17, 18] perhaps reflecting altered D2 receptor expression.[19] Little is known about how impulsivity influences the signalling of rewarding outcome events in dopaminergic sites while playing EGMs, or how impulsivity influences neural responses to game features such as ‘near-misses' that have been claimed to prolong EGM play and increase the likelihood of gambling-related harm.[20, 21, 22, 23, 24] Therefore, we also tested whether variation in impulsivity in our participants was associated with altered signalling within reinforcement sites—in particular, the ventral striatum—in response to slot-machine outcomes.

Materials and methods

All participants provided written, informed consent. Full details of participant screening, the slot-machine and control games, behavioural and fMRI data acquisition, timecourse modelling and additional analyses are provided in the Supplementary Information.

Design and participants

We used a classic between-groups experimental design to test the effects of learning. In these designs, participants (drawn from the same population) are randomly allocated to one group that undergoes some form of training or another group that does not. Subsequently, both groups are compared at test, and differences in the relevant outcome measures are interpreted in terms of learning that occurred in the former but not the latter participant sample. This kind of design is common in learning and memory research,[25] but is rarely attempted with fMRI.

Forty three healthy young adults completed 60 plays of a computerised, simulated slot-machine as part of a standard fMRI protocol. However, 21 of these individuals were randomised to the ‘practiced' group of participants who completed 120 plays of the slot-machine game in a single episode between 1 and 3 days before fMRI scanning (mean=2.10±0.17 days; mode=2 days), whereas 22 individuals were randomised to the ‘unpracticed' group who were given no prior experience with the game. These two groups of participants were closely matched for gender, age, cognitive ability and impulsivity as measured using the I-7 Impulsiveness Scale[26] (Supplementary Table T1), all Fs(1,39)<1.29. Participants were free of gambling problems, with no score above 3 on the South Oaks Gambling Scale.[27]

Detailed analyses of participants' past gambling histories revealed that seven of the 21 practiced participants reported having played slot-machines, poker machines or other EGMs at a frequency of less than once a week, and one practiced participant reported having played EGMs at a frequency of once a week or more. Out of the 22 unpracticed participants, 13 reported having played EGMs less than once a week and none had played EGMs once a week or more. An independent-samples Mann−Whitney U-test revealed that the frequency of EGM play was not significantly different between the practiced and the unpracticed groups (P>0.1).

Computer-simulated slot-machine game

The slot-machine game is shown in Figure 1. It mimics the simplest form of commercial slot-machine game with two sequential events: reel spins followed by game outcomes. The durations of the reel spins and intervals between plays were jittered in order to isolate their evoked blood-oxygenation-level-dependent (BOLD) signals. Participants in our experiment were given £5 credit at the start of the game and each play cost 25p. Participants made a single button-press response with the index finger of their right-hand to start each play. The three reels ‘spun' for between 4 s and 10 s (mean=7 s) before stopping simultaneously to show the game outcomes. Three identical fruits won monetary prizes of values ranging between 50p and £3. The slot-machine game was constructed so that 1/6 of plays terminated with winning outcomes (variable-ratio=6) and 1/5 terminated with near-misses (variable-ratio=5). All other plays terminated with losing outcomes.

Figure 1

Structure of simulated slot-machine. On being shown the cue ‘Click to play', participants made a single button-press to start the slot-machine. Immediately, each of the three reels displayed a random sequence of six different fruits, with a frequency of 5 Hz. All three reels stopped following a Poisson-distributed latency of 4–10 s and showed the game outcomes for a fixed 4 s. The game reels were bordered in green to indicate winning outcomes, and in red to indicate near-miss and losing outcomes. The functional magnetic resonance imaging (fMRI) model included three 1s regressors for reel spins following winning, near-miss and losing outcomes, one extended regressor for the jittered duration of the reel spins, and three 1s impulse regressors for winning outcomes, near-miss outcomes and losing outcomes. The display was blanked before the next play started, with a Poisson-distributed inter-trial interval (ITI) of 3.5–9.5 s. When this game was played outside the scanner, this latter ITI was shortened to a fixed 2.5 s. See Materials and methods above for more details.

Our slot-machine game implemented a return-to-player of 80%, which is representative of at least some commercial EGMs widely available in the United Kingdom (http://www.gamblingcommission.gov.uk/shared_content_areas/gaming_machines_technical_stan.aspx). This reinforcement schedule was designed to mirror the typical experience that players might recognise from their own play in a pub or bar, in which game credits are steadily eroded during a slot-machine episode despite the occasional delivery of winning prizes.

Procedure

On study visit 1, participants completed the screening as described above (see also Supplementary Information). Participants allocated to the practiced group played an extra session with the slot-machine game. They were told that winnings from the game would be paid out in actual money and added to their experimental payment. The reinforcement schedule of the slot-machine game was constructed so that every participant started with £10 and finished with £4. On study visit 2, participants played the game inside a 3-Tesla Siemens MAGNETUM Trio scanner (Siemens Medical Solutions, Erlangen, Germany). In this shorter version of slot-machine game, every participant started with £5 and finished with £2.

FMRI model and timecourse analyses

BOLD amplitudes evoked by winning outcomes and their values, near-miss outcomes and their values, and losing outcomes were all modelled as 1 s impulses of neural activity. BOLD amplitudes evoked by reel spins were modelled both as 1 s impulses (with separate regressors for spins following winning, near-miss and losing outcomes on the immediately previous play), and as the jittered intervals between participants' click responses to start each play and the delivery of game outcomes (collapsed across all plays). To explore the effects of trait impulsivity, I-7 scores were entered as a covariate variable. Regions of interest (ROIs) were drawn on the whole-brain activation map using appropriate Z-statistic thresholds (see Results below). The timecourse of signals across the reel spins and game outcomes are shown for illustrative purposes[28] within ROIs identified using a cluster-corrected threshold of P<0.05. One-sample t-tests and univariate ANOVA analyses (see text) were used to determine the statistical significance of signals using a conventional α-level of P<0.05. Note that the β-values provided in the main text and in the Supplementary Information were obtained from the fitting of haemodynamic response gamma functions to the time series data (see Supplementary Information for full details) and refer to unstandardised regression coefficients.

Results

Prior experience and the neural coding of slot-machine events

Our analysis was conducted in three stages. Our principal focus involved structures innervated by mesolimbic dopamine pathways, especially the ventral striatum and amygdala, that have been shown to support reinforcement learning.[12, 13, 14, 15] However, we also wished to test the effects of practice in other frontostriatal sites that might be involved in the representation of slot-machine play and outcomes. In the first stage of our analysis, we identified limbic, striatal and cortical ROIs that showed significant BOLD signals in the comparison between winning and losing outcomes. These ROIs were identified using a Z-score of 3.09 and a cluster-corrected threshold of P<0.05 (Supplementary Figure S1 and Supplementary Table T2), and were orthogonal to our planned comparisons involving practiced and unpracticed participants; thus, avoiding the problem of circularity when selecting statistical tests in fMRI.[28]

In the second stage, we modelled the BOLD amplitudes elicited by the slot-machine reel spins, and winning, near-miss and losing outcomes within these ROIs. One-sample t-tests over the resultant β-values were used to test the significance of BOLD responses against baseline. Testing our hypotheses involved demonstrating opposing effects of prior experience upon the BOLD amplitudes evoked by the reel spins (signal increases) and winning outcomes (signal decreases). Therefore, the third part of our analysis used an omnibus repeated-measures analyses of variance (ANOVA) over the β-values, with the between-subject factors of practice, gender, impulsivity (high impulsive (HI) vs low impulsive (LI)) and the within-subject factors of game event (reel spins vs winning outcomes) and ROI (see below). HI (10 practiced, 9 unpracticed) and LI (11 practiced, 13 unpracticed) participants were identified using the median split of the I-7 scores, with the LI group scoring equal or lower than 7 and the HI group scoring higher than 7.[29] The effects of practice and impulsivity upon the BOLD amplitudes within individual ROIs of a priori interest—the ventral striatum and amygdala—were tested with univariate ANOVAs (see Supplementary Information for full details).

Collapsing across the practiced participants and unpracticed participants, winning outcomes were associated with significant increases in BOLD amplitudes compared with losing outcomes within the midbrain (VTA/substantia nigra), ventral striatum, caudate nucleus, amygdala, anterior cingulate cortex, dorsomedial prefrontal areas, and the anterior insula cortex. As expected, winning outcomes evoked significant positive BOLD signals against the time series baseline within each of these ROIs, t(41)s >5.32, P<0.0001.

Prior experience with the slot-machine game produced opposite changes in the BOLD responses to reel spins and winning outcomes, as indicated by a significant two-way interaction between practice and slot-machine event, F(1,35)=6.88, P<0.05. This effect was not modulated by gender (F(1,35)=0.369, P>0.1). Testing the simple effects, practice diminished the BOLD amplitudes evoked by winning outcomes, F(1,35)=6.02, P<0.05, but increased the amplitudes evoked by the reel spins (see Figure 2), F(1,35)=4.03, P<0.05, effects which were again not influenced by gender (Fs (1, 35)<1, Ps>0.1). As predicted, these results were most clearly expressed within limbic and striatal sites. Compared with unpracticed participants, practiced participants showed smaller BOLD responses to winning outcomes within the caudate nucleus, F(1,35)=5.399, P<0.05, and ventral striatum, F(1,35)=9.093, P<0.05, with a similar trend in the midbrain, F(1,35)=3.134, P=0.085. By contrast, we observed significantly larger BOLD responses within the ventral striatum and the amygdala to the reel spins in the practiced compared with the unpracticed participants (Figure 2), F(1,35)=4.339, P<0.05 and F(1,35)=6.193, P<0.05, respectively. Comparable practice effects were also evident in the ROIs drawn around the dorsomedial prefrontal areas and anterior insula, though not the anterior cingulate cortex (see Supplementary Figure S2).

Figure 2

Time series plots of blood-oxygenation-level-dependent (BOLD) signals within four regions of interests (ROIs) constructed using the comparison between winning and losing outcomes (thresholded at Z=3.09, whole-brain cluster-corrected at P<0.05). Coronal and axial slices are shown for each of the ROIs. MNI (Montreal Neurological institute) y coordinates are provided below the coronal slices and z coordinates below the axial slices. Upper plots: % BOLD signal changes while watching the game reels spin (displayed for a mean of 7 s following ‘Play') and the winning outcomes of the game (displayed for 7 s following ‘Reel stop'). For the reel spins, plays completed by the practiced participants are indicated by the red lines and plays completed by the unpracticed participants are indicated by blue lines. Means % signal values (relative to baseline) are shown together with standard errors. Lower plots: hemodynamic response function (HRF) gamma model used to fit the BOLD % signals. An ‘impulse' or phasic HRF and mean response latency of 6 s was adapted for the model. The midbrain (including the ventral tegmental area/substantia nigra) is marked in cyan (a); the ventral striatum is marked in red (b); the caudate nucleus is marked green (c); the amygdala in light green (d).

The practice-related changes in reinforcement signalling while playing our slot-machine game showed some degree of anatomical specificity, being most clearly expressed within dopaminergic reinforcement sites compared with posterior cortical regions. To explore this issue, we examined BOLD amplitudes within two posterior ROIs centred round the intra-parietal sulcus and the primary visual area (incorporating the calcarine fissure). These ROIs were identified using the original contrast between winning and losing outcomes (thresholded at Z=3.09 and whole-brain cluster-corrected at P<0.05). Prior experience with the slot-machine game did not influence the BOLD responses evoked by winning outcomes differently from the BOLD responses evoked by the reel spins within these posterior regions (see Supplementary Figure S3), F(1,35)=0.18. Similarly, comparison of the effects of practice on the BOLD signals within the ventral striatum and intra-parietal sulcus area showed shifts in reward signalling away from the outcomes towards reel spin in the former structure, but not in the latter cortical region (Supplementary Figure S3).

Finally, the patterns of BOLD amplitudes evoked by the events of our slot-machine game in the practiced and unpracticed participants are not attributable to their gross visual and motor features. Participants also completed a ‘control' game, which involved similar visual displays and identical motor demands, except that all fruit and credit symbols were replaced by coloured hashes (‘#') (Supplementary Figure S4). The event-structure and timings of the control game were identical to those of the slot-machine game. In general, the BOLD amplitudes elicited by the reel spins and, especially, the outcomes of the control game were weak and statistically unreliable within those mesolimbic and striatal ROIs that were responsive to the comparable events of our slot-machine game (Supplementary Figure S5).

Impulsivity and the neural coding of slot-machine events

We also tested how heightened impulsivity is associated with altered signalling of slot-machine outcomes within the striatum among the practiced and unpracticed participants. We did this in two stages. First, we tested the impact of impulsivity, as a covariate, on the speed at which participants initiated new game plays following winning, losing and near-miss outcomes using repeated-measures ANOVA. Participants with high I-7 scores were significantly faster to start new plays following winning compared with near-miss and losing outcomes (see Supplementary Table T3) (F(2,78)=7.15, P=0.001). In general, higher I-7 scores were associated with faster responses to start new plays following winning outcomes (r(43)=−0.314, P<0.05) but not following near-misses (r(43)=0.015) or losing outcomes (r(44)=−0.057), confirming that the reinforcing effects of successful plays were selectively enhanced in the HI participants. Past analyses of gambling behaviour have identified delays in initiating new EGM plays following winning compared with losing outcomes,[30] which might reflect delays during which positive affect is enjoyed,[30] or the need to initiate new plays promptly to dissipate aversive states generated by losing outcomes.[31] Our observations indicate that winning outcomes specifically facilitate the initiation of new gambling behaviours in individuals with heightened trait impulsivity.

Next, we entered participants' I-7 scores as a covariate of interest into the general linear model of the BOLD time series data (pooled across the practiced and the unpracticed participants) to identify ROIs that showed impulsivity-dependent signals, cluster-corrected at P<0.05. This isolated activity within the ventral striatum and caudate nucleus bilaterally and left amygdala in the comparison between signals evoked by winning outcomes and near-misses (Supplementary Figure S6 and Supplementary Table T4). This analysis revealed that high I-7 scores enhanced the positive signals evoked by winning outcomes within the ventral striatum and amygdala but diminished the signals evoked in the same structures by near-misses (Supplementary Figure S7).

Experiments suggest that the experience of almost-winning, while playing a slot-machine, can prolong gambling sessions,[20, 21, 24] suggesting that near-misses operate as ‘mini-wins' that function as actual winning outcomes[20, 22, 23] or conditioned stimuli that signal imminent success.[20, 23] Alternatively, near-misses can prolong gambling by inducing aversive states such as frustration or cognitive regret.[23, 32] In either case, the capacity of near-misses to prolong gambling behaviour may be most marked in games whose features tend to foster illusions of control.[32] Three recent studies report increased BOLD signals within the mesolimbic dopamine pathways evoked by near-misses compared with losing outcomes.[32, 33, 34] If this activity merely signals the representation of near-misses as 'mini-wins' or conditioned stimuli that predict future gambling success, we would expect impulsivity—a trait linked to gambling involvement and gambling problems[16]—to enhance the positive signals evoked by their presentation, just as it does for winning outcomes. However, our observations unequivocally falsify this prediction.

To further characterise the influence of impulsivity on the near-miss signals within the ventral striatum and amygdala, we examined the timecourse of BOLD responses elicited in HI and LI participants within the ROIs identified using our original (and orthogonal) comparison of winning and losing outcomes, thresholded at Z>3.09 and cluster-thresholded at P<0.05. We also extracted an additional ROI from the same comparison in a region of the putamen posterior and lateral to the nucleus accumbens (see Figure 3). Recently, Clark and colleagues[32] reported enhanced BOLD amplitudes within this region of the putamen following the delivery of near-miss compared with other losing outcomes. Therefore, we tested whether positive near-miss signals within the putamen are modulated by impulsivity.

Figure 3

Time series plots of blood-oxygenation-level-dependent (BOLD) signals evoked by near-miss and losing outcomes within the ventral striatum, putamen and amygdala among low-impulsive (LI) participants (11 practiced; 13 unpracticed) with I-7 scores[32, 33] equal or lower than 7 and in high-impulsive (HI) participants with scores higher than 7 (10 practiced; 9 unpracticed). The regions of interests (ROIs) were identified using the orthogonal comparison of winning outcomes versus losing outcomes (Z-score >3.09, cluster-thresholded at P<0 0.05). Coronal and axial slices are shown for both ROIs. MNI (Montreal Neurological institute) y coordinates are provided below the coronal slices and z coordinates below the axial slices. Upper plots: % BOLD signal changes evoked by near-miss outcomes (indicated in red for LI and blue for HI participants). Means % signal values (relative to baseline) are shown with standard errors. Lower plots: hemodynamic response function (HRF) gamma model used to fit the BOLD % signals. An 'impulse' or phasic HRF and mean response latency of 6 s was used. The ventral striatum is marked in red (a); the amygdala in light green (b) and putamen in blue (c).

Among LI participants, and consistent with previous reports,[32, 34] near-misses evoked significant or near-significant positive BOLD responses within the ventral striatum (t(23)=1.7438, P=0.0945), putamen (t(23)=2.0959, P=0.0473) and amygdala (t(23)=2.89, P=0.0083) (see Figure 3). However, these positive signals were significantly reduced among HI participants in all three sites, F(1,35)=6.528 P=0.015, F(1,35)=7.472 P=0.01 and F(1,35)=8.596, P=0.006, respectively. This confirms that impulsivity diminishes, rather than enhances, the neural signals previously linked to near-miss outcomes.[32, 34]

Finally, there was little evidence that the effects of impulsivity on the neural coding of winning outcomes and near-misses were significantly modulated by practice (Supplementary Figure S8). This implies that the divergent coding of winning and near-miss outcomes represents a relatively stable feature of the way that game outcomes are encoded within reinforcement sites in HI individuals, rather than something that is only apparent during the early acquisition of slot-machine games. Similarly, trait impulsivity did not modulate the positive BOLD signals elicited by the reels spins of the slot-machine (Supplementary Figure S9). Previous studies have reported enhanced BOLD signals within the striatum in HI, non-clinical participants while anticipating monetary rewards,[35] but decreased responses in impulsive abstinent alcoholics.[36] However, these investigations tested the effects of impulsivity when participants were required to adjust their instrumental responding on the basis of conditioned cues to secure identifiable large or small rewards. By contrast, our data suggest that impulsivity exerts its principal influence on the consummatory aspects of slot-machine play; that is, the processing of winning outcomes rather than the anticipatory processes associated with watching the reels spin that precede them.

Discussion

Our results show that prior experience with playing a simulated slot-machine game diminished the positive BOLD signals within the ventral striatum and the caudate nucleus elicited by the winning outcomes but potentiated the BOLD responses within the ventral striatum and the amygdala elicited by the preceding reel spins. In other words, a single episode of slot-machine play engages well-characterised reinforcement-learning mechanisms mediated by mesolimbic dopamine pathway,[13, 14] in order to shift reinforcement signalling from game outcomes to their preceding anticipatory states. Our data also show that trait impulsivity enhanced the reward signals elicited by winning outcomes within the ventral striatum and amygdala but diminished the signals evoked by near-misses. Thus, impulsivity is associated with divergent coding of winning and almost-winning experiences within dopaminergic sites. These latter findings indicate that near-misses do not operate simply as ‘mini-wins' that activate reinforcement circuits to signal imminent gambling success.

Before discussing how our findings help us understand EGM play, we first consider some methodological issues that bear on interpretation. We note that the design and structure of our slot-machine game differed from commercial EGMs in several respects, reflecting the constraints imposed by fMRI. First, our fMRI-compatible game delivered comparable rates of winning and near-miss outcomes while commercial slot-machines deliver near-miss outcomes at markedly greater frequencies than those of the winning outcomes.[37]

Second, near-miss outcomes in our game were constrained to be AAB patterns, whereas those of commercial machines take the forms ‘ABA' or ‘ABB'. Third, unlike our game, commercial slot-machines incorporate variable prize structures, including the delivery of small prizes following the presentation of single symbols or even configurations of different symbols. Furthermore, the outcomes of our slot-machine game were delivered according to variable ratios (that is, 6 and 5 for winning and near-miss outcomes), whereas commercial machines often operate with random-ratios, minimising fluctuations in expectancy across extended consecutive plays.[38, 39] Finally, the durations of our reel spins were both longer and more variable (‘jittered') than those of commercial slot-machines, in order to ensure that we could properly separate BOLD responses evoked by reel spins and game outcomes. These differences mean that our findings do not reflect the neural signalling evoked by some of the more salient structural characteristics of commercial EGMs. However, our model was optimally designed to test the effects of prior experience upon neural activity within the reinforcement-related circuits, and it also improves upon previous experiments with slot-machines[24, 32, 34] by implementing a commercially realistic percent return-to-player.

Playing experience with slot-machine engages dopaminergic reinforcement-learning mechanisms

Instantiating what is known about how the dopamine system supports reinforcement learning,[12, 13, 14] and the involvement of striatal sites in the consolidation of reinforcement learning over periods of days,[40] our results are consistent with the proposal that a single episode of slot-machine play diminishes the positive reward value of winning outcomes but enhances the value of the game events that precede them. Our findings link these shifts in reward value to the representation of positive prediction errors by dopamine pathways, and their efferent mesolimbic, striatal and amygdala targets,[13, 14] to clinical evidence of dysfunction within dopamine pathways in pathological gamblers,[41] and to preliminary evidence that altered D2 receptor activity mediates problematic EGM play.[42]

Further experiments are needed to establish the precise psychological correlates of the enhanced signals within the ventral striatum evoked by watching the reels spin following prior game experiences. However, we consider two possibilities. First, phasic activity of the VTA and its target sites in the ventral striatum signal both the magnitude and the probability of rewards against their variances in a way that represents expected value.[43] Corresponding signals reflecting linear and non-linear subjective probabilities of reward can also be detected in the human striatum using fMRI;[44, 45] representations that may mediate gambling behaviours in humans.[46] Therefore, increased activity within the ventral striatum following a single episode of slot-machine game play may reflect learning that links the game's reels spins to their likely expected value in terms of frequency of positive outcomes.

On the other hand, the enhanced BOLD signals within the ventral striatum elicited by slot-machine reel spins are unlikely to reflect reward uncertainty per se. Winning outcomes in our slot-machine game were delivered, on average, once every six plays. Uncertainty about the likelihood of winning outcomes would have been maximal in those participants completing the game for the first time, predicting that BOLD signals within the ventral striatum evoked by reels spin should have been increased in the unpracticed compared with the practiced participants. Here, we found the opposite pattern of signal changes following practice (see Figure 2).

Second, other evidence suggests that the activity of the VTA and its dopaminergic targets code the incentive salience,[47] as well as physical salience,[12] of environmental stimuli, with indications that salience signals can be detected in the human striatum using fMRI.[48] Clinical reports suggest that habitual slot-machine players find that the reel spins, and the configurations of prominent visual and auditory stimuli that accompany them, capture attention and encourage play.[49] Thus, the heightened activity within the ventral striatum may signal the enhanced salience of reel spins in practiced players. Relatedly, substantial evidence indicates that the amygdala plays a significant role in coding the reinforcement value of stimuli that predict rewards including those underlying addictive behaviours.[50] Amygdala activity allows these stimuli to sustain reward-seeking behaviours.[50, 51] Thus, our findings that practice enhanced the BOLD amplitudes evoked while watching the reels spin are also consistent with the proposal that the incentive and conditioned value of the reel spins, and other visual-auditory features of slot-machines, are passed from the amygdala to ventral striatum via their efferent projections,[51] perhaps promoting continued play.

Impulsivity modulates neural responses to the structural characteristics of slot-machine games

Heightened impulsivity is associated with cognitive problems, and compromises treatment outcomes, in pathological gamblers.[16] However, little is known about how impulsivity influences behavioural responses to slot-machine outcomes, or their neural coding within reinforcement circuits. Here, we found that high I-7 scores[26] speeded the initiation of new slot-machine plays following winning outcomes, suggesting that impulsivity counteracts ‘post-reinforcement pauses' in play,[30] and instead facilitates subsequent gambling participation by enhancing the reward value of gambling successes.

Impulsivity—in the form of greater discounting of temporally-delayed rewards—is linked to heightened BOLD responses to monetary rewards within the ventral striatum[17, 18] and to the representation of delayed rewards within the amygdala.[52] In humans, reduced midbrain D2/D3 receptor expression is associated with heightened psychometric scores of impulsivity and increased dopamine release following amphetamine challenge.[19] Our finding that impulsivity enhanced BOLD responses within the ventral striatum and amygdala, and speeded the initiation of new game plays, following winning outcomes suggests that impulsivity enhances the reward value of gambling successes within mesolimbic dopamine systems, promoting play in individuals vulnerable to gambling problems and comorbid addictions.[53]

Our findings that impulsivity modulated BOLD responses to near-misses within the ventral striatum and amygdala are more challenging, but still advance our understanding of the neural basis of EGMs in several ways. Our data show that, in some individuals, the experience of almost-winning can elicit positive BOLD signals (against baseline) within the ventral striatum and the amygdala.[32] They also demonstrate that the positive reinforcement signals elicited by near-misses can be observed even in simple slot-machine games with little player involvement that might foster ‘illusions of control'. However, near-misses cannot operate simply as ‘mini-wins' or signals that winning outcomes are imminent.[20, 23, 24] If they did, impulsivity would have enhanced the positive signals evoked by their presentation, just as it did for winning outcomes. Here, we found that heightened impulsivity was associated with reduced BOLD responses to near-misses within the ventral striatum and amygdala.

There are two possible explanations for these observations. First, clinical reports suggest that near-misses prolong gambling episodes by inducing negative affective states such as frustration or cognitive regret;[23, 32, 54] and these states can be markedly enhanced in impulsive individuals such as those vulnerable to gambling problems.[55] One possibility is that aversive states triggered by near-misses are signalled by decreased activity within the reinforcement systems in HI individuals. On the other hand, aversive states have been linked to heightened activity within the striatum and amygdala.[56, 57, 58] Aversive signals within the amygdala can also be diminished by cognitive re-appraisal.[58] Continued gambling frequently depends upon the ability of players to ‘rationalise away' gambling losses to legitimise further play.[59] Therefore, the attenuation of ventral striatum and amygdala signals following near-misses in high impulsive participants may reflect the cognitively-mediated attenuation of aversive states following near-miss outcomes.

Second, it is natural to suppose that players attempt to predict the occurrence of near-misses alongside winning outcomes, as part of the process of understanding the reinforcement contingencies underlying any given slot-machine game. Indeed, there is evidence that mistaken beliefs about such contingencies are more strongly held in problem slot-machine players.[60] Other data indicate that salient aversive events generate positive BOLD signals within the ventral striatum, perhaps encoding the prediction errors that mediate aversive learning,[61] while monetary losses and their associated prediction errors may be encoded within the amygdala.[62] This raises the possibility that impulsivity is linked to the attenuation of the positively-signed prediction errors evoked by near-misses, potentially facilitating continued play.

Finally, we note two ways in which the current results might be extended to help us understand problematic EGM play. First, other psychological factors may also modulate the neural signalling of slot-machine play. For example, in addition to impulsivity, depression and anxiety are prominent in the development and maintenance of gambling problems.[63, 64] Problematic EGM play can be motivated by coping with negative emotional states, possibly involving distraction and even dissociation states.[65, 66] Our data raise the possibility that such emotional disturbance may enhance, or modulate, the shift in reward signalling within mesolimbic pathways from consummatory processing of winning outcomes to the anticipatory processing driven by reel spins. Future experiments could test this possibility.

Second, the anticipation of rewards in many situations can reflect at least two processes: those appetitive processes elicited by the delivery of reward-related cues (including the slot-machine game displays used here) and separate processes that follow instrumental responses made to obtain rewards.[67] The BOLD amplitudes evoked by the reel spins of our game were modelled following participants' motor responses to start each new play; and, therefore, presumably reflect the anticipatory signals driven by these latter instrumental processes. Further work will be needed to isolate precisely the different kinds of anticipatory processes that drive the reward value of slot-machine games and other EGMs in both occasional and frequent players, or players with gambling problems.

In summary, our data show, for the first time, that prior experiences with slot-machine games are associated with shifts of positive reinforcement signalling away from the game outcomes towards the preceding reel spins. Impulsivity is associated with divergent coding of winning and near-miss outcomes, in particular attenuating the signals evoked by the experience of almost-winning in the ventral striatum and amygdala. Collectively, these data delineate how single episodes of slot-machine play trigger the transfer of value away from game outcomes towards the anticipatory states within dopaminergic reinforcement pathways. They also demonstrate how personality traits that confer vulnerability for gambling problems may interact with structural features of EGMs to enhance the likelihood of continuing play.