Occasion setters attain incentive motivational value: implications for contextual influences on reward-seeking.

The context in which reward-paired cues are encountered can resolve ambiguity and set the occasion for appropriate reward-seeking. The psychological processes by which contexts regulate reward-seeking remain unclear as contexts are diffuse and difficult to isolate from other stimuli. To overcome this, we modeled a context as a phasic and discrete event-an occasion setter (OS)-which allowed for control over its presentation and influence on cue-driven reward-seeking. This allowed us to directly assess how OSs regulate the predictive and motivational significance of Pavlovian cues. Male rats (n = 50) were trained in a Pavlovian procedure where the presentation of an ambiguous conditioned stimulus (CS) was reinforced only if preceded by an occasion setting cue. We assessed the motivational value of the OS and CS alone or in combination using tests of conditioned reinforcement. Rats enhanced conditioned approach to the reward port during the CS when it was preceded by the OS. When allowed the opportunity, rats responded more to obtain presentations of the CS in combination with the OS than the CS alone. Critically, rats also worked to obtain presentations of the OS alone more than the CS alone, and this was resistant to manipulations of the value of the OS. We conclude that occasion setting can act via incentive motivational mechanisms and that, apart from resolving predictive information about ambiguous reward-paired cues, OSs themselves generate states of appetitive motivation that can facilitate reward-seeking.

Cues paired repeatedly with reward not only acquire a predictive relationship with reward but also attain incentive motivational properties, or incentive salience, that render Pavlovian reward-paired cues attractive and desirable. Indeed, the incentive motivational properties of cues are thought to be a primary trigger for food-seeking as well as relapse in drug-abstinent addicts (Stewart et al. 1984; Robinson and Berridge 1993; Kelley 2004; Flagel et al. 2009; Milton and Everitt 2010; Robinson et al. 2014b). Cue-based exposure therapy is employed clinically, sometimes alongside cognitive behavioral therapies, in an effort to extinguish the incentive motivational properties of drug-paired cues but is generally ineffective in producing lasting reductions in drug-seeking behavior and in preventing relapse (Carter and Tiffany 1999a,b; Conklin and Tiffany 2002; Kavanagh et al. 2006; Martin et al. 2013; Mellentin et al. 2017). A possible explanation for these failures comes from viewing drug-seeking as being controlled directly by a simple cue-reward relationship, which fails to capture the multitude of situations in which the cue may or may not be motivationally relevant (Bouton 2002).

These issues are exemplified in the ability of a reward-associated context to renew responding to Pavlovian cues previously associated with reward despite their extinction in a separate, distinct setting. These findings indicate that the contexts in which reward-predictive cues are encountered can modulate the ability of those cues to trigger reward-seeking (Chaudhri et al. 2008; Crombag et al. 2008; Bouton et al. 2006). However, the exact underlying psychological process by which contexts act to produce renewal of reward-seeking has remained unclear (Bouton 1988; Holland and Bouton 1999). For instance, a context may enter into a direct relationship with reward such that when a weakly predictive cue is presented it triggers responding as a result of summing the strength of the context-reward and cue-reward relationships. Others have proposed that contexts act in a more complex way as occasion setters (OS) which would instill them with the ability to resolve the ambiguity of reward-predictive cues (Grahame et al. 1990; Holland and Bouton 1999; Valyear et al. 2017; Fraser and Holland 2019). To date, however, there has been little investigation into the precise psychological mechanisms underlying context-induced effects on reward-seeking behavior.

To address these issues, we adapted an animal behavioral model of occasion setting to directly probe the underlying psychological mechanisms by which OSs, like a context, may act to modulate reward-seeking. In this model, a brief cue informs that in the near future a typical conditioned stimulus (CS) will be followed by reward. If either the occasion setting cue or CS are presented in isolation, they are nonreinforced. In essence, this model reduces a context to a brief, phasic, and localizable event in the environment. We investigated if occasion setting may act by magnifying the underlying incentive motivational value of ambiguous reward-predictive cues in tests of conditioned reinforcement. Indeed, OSs modulated both the predictive and motivational properties of their conditioned stimuli, but to our surprise we found that OSs acquired incentive motivational value in their own right, and that the modulatory actions and motivational value of OSs were resistant to extinction. Together, these findings have important implications for our understanding of complex cue interactions in triggering reward-seeking and relapse.

Results

A novel procedure to observe occasion setting

Rats were trained in an occasion setting task in which a CS was reinforced only if its presentation had been preceded by the presentation of a separate OS cue. If either the OS or the CS were presented in isolation they were not reinforced (Fig. 1A). This resulted in a situation in which rats had to constantly update their expectations of reinforcement based on the events surrounding encounters with the ambiguously predictive CS. To understand how the OS might affect responding produced by the CS we examined food cup activity when the CS was present, or in the corresponding time interval when the CS was withheld in the case of OS alone trials (the “period of interest,” depicted in Fig. 1A). The primary form of the conditioned response was head jerk-related movements inside the food cup throughout the CS, in agreement with previous reports that in occasion setting the primary conditioned response resembles the form supported by the CS (Holland 1977, 1992). Rats in the paired condition responded maximally on OS + CS trials, that is, when the OS preceded the CS, and this pattern of responding was identical for those rats who would undergo (n = 20) or not undergo (n = 20) extinction of the OS (interaction of session × group × trial type F(40,106) = 1.589; P = 0.032; simple effect of trial type within each group P < 0.0001; all within group Bonferroni post-hoc comparisons between OS + CS versus CS alone and OS alone for sessions 11–18 P < 0.001 for paired groups; all Bonferroni post-hoc comparisons between paired groups for each session P > 0.9; Fig. 1B). In contrast, the unpaired group (n = 10) did not develop a noticeable degree of conditioned responding during the CS period (no simple effect of trial type within the unpaired group across sessions). Thus, this procedure, with a relatively large number of trials per session, and equal presentations of all trial types in a session, produces behavior akin to occasion setting with minimal training.

Figure 1.

A novel model for occasion setting. (A) Schematic of the final stage of the occasion setting task. In each session rats randomly receive one of three trials types, 10 of each per session. Only when the OS and CS are paired (with a 5 sec gap between) is the termination of the CS followed by reward delivery. (B) Normalized percent time in port during the CS period across training. Paired rats reinforced as in A acquire discriminatory responding in the task, and this is not true for Unpaired rats receiving truly random delivery of reward (n = 10; small gray circles). Paired rats who would (n = 20; open symbols) or would not (n = 20; solid symbols) undergo OS extinction following training did not differ from each other at any point during training. Data are presented as mean ± SEM. (OS) occasion setter, (CS) conditioned stimulus. Purple represents reinforced trials, blue represents CS alone trials, and red represents OS alone trials.

Occasion setters have incentive motivational value

We then assessed the motivational value of the OS, CS, and the combination of the OS + CS in a series of conditioned reinforcement tests for a subset of paired rats (n = 20) and rats who received unpaired training (n = 10). In this test, rats were asked to learn a novel response to earn presentation of one of the following stimuli in the absence of food reward: the OS alone, the CS alone, or a combination OS + CS stimulus. We found that paired rats earned the combination OS + CS stimulus and the OS alone more than unpaired rats (Main effect of cue F(1.856,51.03) = 3.958, P = 0.0289, main effect of group F(1,28) = 8.095, P = 0.0082, interaction F(2,55) = 3.28, P = 0.0451; post hocs P < 0.05; Fig. 2A). There was no effect of cue within unpaired rats suggesting that there was no contribution of differences in the degree of sensory reinforcement among tests to the pattern of responding we observed in paired rats (Meyer et al. 2014). Paired rats earned the combination of the OS + CS more than CS alone (P = 0.0006), but also earned more OS alone presentations than CS alone (P = 0.0403) while the number of cues earned did not differ between OS + CS and OS alone tests suggesting these cues had similar conditioned reinforcing value (P = 0.285). This pattern was similar for the difference between active and inactive responses (planned comparisons in paired group OS + CS vs. CS alone P = 0.0089; OS alone vs. CS alone P = 0.0443; OS + CS vs. OS alone P = 0.4085; Fig. 2B). In addition, only in tests where rats could earn the OS + CS or OS alone did paired rats discriminate and respond significantly more on the active over inactive lever than expected by chance (Wilcoxon tests P < 0.01). To better understand potential differences in the representations evoked by earning the OS + CS in combination and the OS alone we examined port entries made during the brief 2 sec cue for each cue type. Although port entries during the 2 sec cues were predictably low, paired rats made more port entries during each cue presentation when the cue they earned was the CS alone or the combination of OS + CS than for the OS alone (interaction of cue and group F(2,83) = 3.991, P = 0.0221; post-hoc comparisons P < 0.001; Fig. 2C). This suggests that earning the OS alone in these tests did not evoke immediate reward-seeking behavior, but the addition of the OS to the CS increased both motivation to earn the CS and spurred reward-seeking. In contrast, rats did not work to earn the CS alone suggesting that in this preparation a reward-adjacent and ambiguous CS does not support conditioned reinforcement. Collectively, this pattern of results indicates that an OS, typically thought to be a cue that modulates the predictive significance of a cue-reward relationship, can increase the motivational value of an otherwise undesirable CS, as well as develop incentive motivational properties in its own right.

Figure 2.

Occasion setters have incentive motivational value. (A) Number of cues earned during the conditioned reinforcement test for each cue. (B) Magnitude of responses on the active minus inactive operant. (C) Number of port entries made while the 2 sec cue was present following a response on the active operant. (D–F) same as A–C but the individual data. For all figures bars indicate mean + SEM. Purple reflects the OS + CS test, red reflects the OS alone test, and blue reflects the CS alone test. (OS) occasion setter, (CS) conditioned stimulus. (*) P < 0.05 for post-hoc comparisons, (^) P < 0.05 for Wilcoxon signed rank test against null value of 0.

Extinction of an occasion setter does not alter its ability to resolve predictive information of its conditioned stimulus

Behavior in the occasion setting task and the subsequent conditioned reinforcement tests could be explained by either of two hypotheses: (1) animals use the visual stimulus as an OS to modulate the significance of the auditory CS and/or (2) the OS and CS each are relatively weakly associated with reward and rats sum these two strengths to increase reward-seeking during combined OS + CS presentation, compared to OS or CS alone. The latter could explain why paired rats spent more time in the food cup following the OS alone than unpaired rats (Fig. 1B), and why paired rats work to earn just the OS (Fig. 2), as the OS itself could have become weakly associated with reward. To directly assess these possibilities, a separate group of rats (n = 20; future extinction group from Fig. 1B) were tested in the occasion setting procedure under extinction conditions where reward was withheld to examine the microstructure of their behavior across trial types. By quantifying the probability of being in the reward port on a second by second basis across the serial presentation of the OS and CS, it appeared that the OS alone evoked a small increase in the chance a rat would enter the reward port prior to CS onset (Fig. 3A). We then extinguished responding to the OS alone in a series of four sessions consisting only of OS presentations resulting in percent time in port on the final day of OS extinction being 3.023 ± 0.9% SEM. After this, we asked if this extinction of any direct links between the OS and reward would reduce its ability to serve as an OS in a subsequent extinction test with all trial types. The effect of this manipulation was apparent in the microstructure of behavior across all three trial types, with rats no longer exhibiting any increase in reward port approach to the OS alone following OS extinction, yet still using the OS to increase their reward-seeking on trials where both the OS and CS were presented (Fig. 3B). Analyzing time in port during the CS period also revealed that, while responding in the second extinction test, after OS extinction, was lower overall (main effect of test F(1,19) = 113.4, P < 0.0001), extinction of the OS did not produce a deficit in the ability of rats to use the OS as an OS, as responding was still highest on the OS + CS trials (main effect of trial type F(1,19) = 87.3, P < 0.0001; test × trial type interaction F(2,38) = 4.122, P = 0.0240; post-hoc comparisons between OS + CS and CS alone and OS alone post OS extinction, all P < 0.01; Fig. 3C).

Figure 3.

Extinction of an OS does not impair its ability to enhance the predictive value of its CS. (A) Probability of observing a rat in the reward port across each trial type during the initial extinction test (Interaction between seconds and trial type F(78,1482) = 42.69, P < 0.0001). Lines indicate periods when OS + CS trials are significantly greater (Bonferroni post hoc P < 0.05) than CS alone (purple) and CS alone is significantly greater than OS alone (blue), and OS alone is significantly greater than CS alone (red). (B) Probability of observing a rat in the reward port across each trial type in the second extinction test following OS extinction (Interaction between seconds and trial type F(78,1482) = 40.52, P < 0.0001). Lines indicate periods when OS + CS trials are significantly greater (Bonferroni post hoc P < 0.05) than CS alone (purple) and CS alone is significantly greater than OS alone (blue). (C) Normalized percent time in port during the CS period for each extinction test. Purple represents reinforced trials, blue represents CS alone trials, and red represents OS alone trials. (D) Discrimination scores for each extinction test. Empty bars represent data from the session prior to OS extinction and filled bars data from the session following OS extinction. For all figures symbols indicate mean ± SEM. (OS) occasion setter, (CS) conditioned stimulus. (***) P < 0.05 for main effects; (*) P < 0.05 for post-hoc comparisons.

The preservation of the occasion setting abilities of the OS after extinction was also readily apparent when looking at bias scores which are resistant to changes in the total amount of conditioned approach (see Materials and Methods). When analyzing their behavior during the CS period of interest, rats were better at discriminating between OS + CS versus OS alone trials than OS + CS versus CS alone trials (main effect of discrimination F(1,19) = 51.42, P < 0.001; Fig. 3D). An interaction between extinction and discrimination (F(1,19) = 8.608, P = 0.0085) revealed that after OS extinction rats responded even less during OS alone trials than before OS extinction (P = 0.0169) but discrimination between CS alone and reinforced trials was unaffected by OS extinction (P > 0.9999). These data also confirm that the behavior observed during conditioning meet an important criterion for occasion setting, and is not simple summation of responding between the partially reinforced OS and CS. Thus, in the absence of possible direct predictive associations with reward, the OS cue still acts to set the occasion for reward-seeking.

The incentive motivational properties of an occasion setting cue are extinction resistant

We next asked whether direct extinction of the OS would affect the conditioned reinforcing properties of the OS by using the same conditioned reinforcement tests as before, immediately following the second extinction test. Rats who underwent OS extinction did not work for the CS alone but did work to earn combined presentations of the OS + CS, replicating our original finding (Friedman statistic for cue effect = 15.1, P = 0.0005; post hoc P = 0.0008; Fig. 4A). Critically, OS extinction did not alter the willingness of rats to earn the OS alone and rats earned significantly more presentations of the OS alone compared to the CS alone test (P = 0.008; Fig. 4A). This pattern of responding was similarly reflected in the magnitude of active–inactive responses with rats working more on the active lever to earn the OS + CS and OS alone but not for the CS alone (Friedman statistic for cue effect = 6.3, P = 0.0429; Wilcoxon tests for OS + CS and OS alone P < 0.05; Fig. 4B). Interestingly, despite it being more than 2 wk since these rats had received reward, they still made more port entries when they earned the combination of the OS + CS than the OS alone and CS alone (Friedman statistic for cue effect = 13.59, P = 0.001; post hoc P < 0.05; Fig. 4C). Taken together, these findings indicate that the incentive motivational value of an OS, and the ability of that OS to enhance the incentive motivational value of its CS, are both extinction resistant.

Figure 4.

The incentive motivational properties of an OS are extinction resistant. (A) Number of cues earned during the conditioned reinforcement test for each cue. (B) Magnitude of responses on the active minus inactive operant. (C) Number of port entries made while the 2 sec cue was present following a response on the active operant. (D–F) same as A–C but the individual data. For all figures bars indicate mean + SEM. (OS) occasion setter, (CS) conditioned stimulus. (*) P < 0.05 for post-hoc comparisons, (^) P < 0.05 for Wilcoxon signed rank test against null value of 0.

Discussion

Cues repeatedly paired with reward become predictors of reward availability, but may also acquire incentive motivational properties that can render these cues desirable on their own and endow them with the ability to spur and energize action (Bindra 1974, 1978; Stewart et al. 1984). While much is known about the predictive and incentive properties of cues that have a deterministic, absolute relationship with reward availability (Robinson and Flagel 2009; Meyer et al. 2014; Ahrens et al. 2016), considerably less is known about ambiguous cues and the factors that regulate their predictive and motivational value. We assessed whether a special class of cues that regulate the strength of an ambiguous cue-reward relationship, called OSs (Holland 1992; Meyer and Bucci 2016; Shobe et al. 2017; Fraser and Holland 2019), could engender their own incentive motivational properties. We find that while cues trained as OSs do not obligatorily elicit reward seeking on their own, they acquire incentive salience and can act to enhance both the predictive and motivational value of a CS.

That OSs support conditioned reinforcement may suggest that this result is the consequence of second-order conditioning to the occasion setting cue (Rizley and Rescorla 1972; Gewirtz and Davis 2000), but a number of distinctions rule out that second-order conditioning could be responsible. First, second-order conditioning is most frequently observed when the introduction of the second-order cue follows the formation of a strong associative pairing of the first-order cue with reward, whereas training here proceeded from the outset with serial pairings of the cues. Second, the presence of reward generally discourages second-order conditioning, requiring the introduction of the second-order cue in extinction. In our preparation the serial presentation of cues was always followed by reward, but either cue in isolation was never reinforced, making it unlikely that the motivational and predictive value of the CS is resulting in new learning about the occasion setting cue. Moreover, what is learned about the CS should be equivalent for the OS if second-order conditioning is occurring, but we observed conditioned reinforcement for the OS and not the CS. Third, we rarely observed behavior in the food cup during the OS suggesting that the representations evoked by the OS were distinct from those evoked by the CS. Finally, extinction of the OS did not prevent conditioned reinforcement, whereas extinction of a first-order CS prevents the observation of conditioned reinforcement (Lindgren et al. 2003; Holland 2016). Given second-order stimuli directly inherit their predictive and motivational properties from first-order stimuli, their conditioned reinforcing properties, if any, should also be sensitive to extinction (Sharpe et al. 2017). While occasion setting and second-order conditioning may appear procedurally similar, there is little indication that second-order conditioning can explain the conditioned reinforcing properties of an occasion setting cue and the ability of this cue to enhance conditioned approach to an ambiguous CS. This suggests that the incentive and predictive properties of the occasion setting cue are not readily explained by existing associative models (Fraser and Holland 2019).

A common model to assess the modulation of reward-seeking is the use of physical contexts to denote situations where cues will or will not lead to various outcomes like food reward, aversive footshock, or drugs of abuse (Bouton and Bolles 1979; Holland and Bouton 1999; Bouton 2002; Chaudhri et al. 2008; Crombag et al. 2008). However, it has remained difficult to isolate and mechanistically understand the underlying psychological processes that allow contexts to facilitate reward-seeking as contexts are multidimensional and have long-lasting temporal effects. To overcome this, we empirically assessed one of the proposed mechanisms of physical contexts, occasion setting, by substituting a brief and phasic event for a context. This overcame obstacles associated with using physical contexts. In particular, this allowed us to extinguish behavior resulting from the OS, which has been attempted but as contexts alone fail to evoke obvious observable behavior it has been unclear if context extinction occurs (Remedios et al. 2014). Surprisingly, extinction of an OS did not impair its ability to resolve ambiguity about reward-paired cues, nor did extinction of the OS affect its motivational value. Because we found that an OS acquired incentive motivational value, as well as serving to disambiguate both the incentive motivational and predictive properties of conditioned stimuli, we suggest by extension that physical contexts may act in these ways. Taken together, occasion setting may be an essential and enduring process contributing to relapse as contexts, physiological states, and discrete cues can each function as OSs to generate states of motivation preceding encounters with cues directly associated with drug use that may ultimately overcome goal-directed attempts to maintain abstinence.

Our data suggest that OSs meet at least one of these criteria for an incentive stimulus, as the OS on its own was able to reinforce the acquisition of novel responses to earn its brief presentation. Incentive stimuli are learned cues that are able to evoke motivational and emotional states (Bolles 1972; Bindra 1978; Stewart et al. 1984; Toates 1994). These stimuli can elicit conditioned approach upon their presentation, reinforce behavior in the absence of reward, and spur action (Berridge 2001; Cardinal et al. 2002). In our procedure, we rarely observed conditioned approach to the OS, a localizable houselight, and it remains to be demonstrated if an OS can act to invigorate reward-seeking actions in tests of Pavlovian-to-instrumental transfer. It was also rare to observe any approach to the food cup during the OS's presentation, so in the absence of any overt behavioral response the OS still became imbued with incentive motivational properties and was later able to support conditioned reinforcement. However, it is evident that the OS regulated both the predictive and incentive motivational properties of reward-paired cues as evidenced by enhancing conditioned approach to the food cup during the CS and enhancing the CS's otherwise minimal or nonexistent conditioned reinforcing value. Despite both the OS and the CS having an equal overall probability of reward, we only observed conditioned reinforcement for the OS. This is contrary to observations that uncertainty amplifies the incentive motivational value of Pavlovian cues (Anselme et al. 2013; Robinson et al. 2014a; Zack et al. 2014), but perhaps the hierarchical nature of occasion setting focuses the motivational enhancing aspects of uncertainty to the resolving, occasion setting cue. Together, these data suggest a dissociation between predictive and incentive motivational properties of OSs, in that an OS can produce a state of incentive motivation upon its presentation, but on its own it does not act as a predictor of reward to trigger reward-seeking. Instead, we argue that this OS-evoked motivational state makes reward-associated cues desirable targets of motivation thereby facilitating reward-seeking for food, and potentially other rewards such as drugs of abuse.

Given that OSs support conditioned reinforcement, it is likely mesolimbic dopaminergic projections from the midbrain to the nucleus accumbens, which have been proposed to mediate both incentive motivation and reward prediction error, are involved in this behavioral process (Taylor and Robbins 1984; Berridge 2012; Saddoris et al. 2015; Saunders et al. 2018; Keiflin et al. 2019). Glutamatergic input to the nucleus accumbens from the basolateral amygdala may also be essential for the occasion setting studied here (Holland and Gallagher 1999; Everitt et al. 2003; Wassum and Izquierdo 2015). Lesions of the basolateral amygdala result in a profound deficit in updating the value attributed to a simple CS and adapting responding appropriately, suggesting that in the amygdala's absence the proper encoding, updating, and utilization of state value is lost (Hatfield et al. 1996; Morrison and Salzman 2010; Sharpe and Schoenbaum 2016). The occasion setting procedure utilized here could be especially helpful for facilitating investigations into neural circuitry underlying dynamic regulation of cue-triggered motivation in freely moving rodents.

That OSs have conditioned reinforcing value may suggest that their actions are the result of a model-free representational system (Parkinson et al. 2005; Dayan and Berridge 2014). Our evidence supports this, as manipulations of association between the OS and reward, via direct extinction, failed to alter motivation to work for the OS in isolation. A strong test of model-free versus model-based systems is whether a given cue engages a representation of the outcome it predicts (Dayan and Berridge 2014). In our conditioned reinforcement tests we left the food cup available despite the absence of reward availability in an effort to test this possibility. As rats earned the OS in isolation they did not immediately proceed to check the food cup, suggesting that the OS did not evoke a representation of the outcome that was sufficient to result in reward-seeking actions. However, when the OS was paired with its CS in this test rats frequently made entries into the food cup during this brief cue. These data suggest that the representations evoked solely by an OS in isolation may be model-free, that is, they are reflective of an averaged value that is independent of representation of reward. However, in conjunction with its CS the actions of the OS to enhance both conditioned reinforcement may reflect the actions of a model-based system, given that the combination of these cues trigger reward-seeking behaviors. These could suggest differing underlying neural substrates for the motivational states evoked by OSs versus their actions to guide appropriate reward-seeking triggered by ambiguous cues. In particular, conditioned reinforcement for Pavlovian cues can be sensitive to manipulations of the value of the outcome that they predict and that this sensitivity is dependent on the orbitofrontal cortex and basolateral amygdala (Burke et al. 2007, 2008). In contrast, conditioned reinforcement for well-trained, perhaps model-free, Pavlovian cues is dependent on dopamine and its actions within the nucleus accumbens core (Taylor and Robbins 1984; Saunders et al. 2018). This suggests that an observation of conditioned reinforcement for a cue may not represent the sole contribution of either a model-free or model-based system and that there may be different neural substrates for model-based versus model-free conditioned reinforcement. It will be important to determine in future studies if direct manipulations of outcome value differentially affect responding for an OS alone or in combination with its CS.

We have demonstrated that a unique class of cues that modulate the significance of a cue-reward relationship has the potential to generate states of motivation, even in the absence of direct associations with reward, which may energize and ultimately lead to pursuit of rewards like food and drugs. This property of OSs may explain the ability of physical contexts to renew food- and drug-seeking and, more broadly, the invigoration of reward-seeking brought about by the myriad cues encountered in the environment. Occasion setting need not solely be for regulating conditioned reward-seeking as other behavioral responses to a given dose of a drug of abuse, such as sensitization and tolerance, can also come under control of occasion setting mechanisms (Anagnostaras et al. 2002; Ramos et al. 2002). Further investigations into both the psychological and neurobiological processes underlying occasion setting may provide new avenues for future clinical interventions with lasting benefits for chronic relapsing disorders, like addiction and PTSD.

Materials and Methods

Subjects

Male Long–Evans rats (n = 50) weighing 250 g were purchased from ENVIGO and were single-housed in a temperature- and humidity-controlled colony (lights on at 07:00) with enrichment in their cages. Following 1 wk of acclimation to the colony room, rats were food-restricted (95% of free-feeding weight). To acclimate them to the reinforcer used during training, rats were given 24 h access to 15% sucrose (w/v in tap water) one day before behavioral procedures began. All behavioral training took place during the light cycle. Sample sizes were determined based on pilot studies. All procedures were approved by the Animal Care and Use Committee at Johns Hopkins University and are in accordance with the Guidelines for the Use and Care of Animals in Research, 8th Edition.

Apparatus

Behavioral training and testing took place in 10 MedAssociates chambers in individual sound- and light-attenuating cabinets and were controlled by a computer running MedPC IV software. Each chamber was equipped with a recessed port on the front wall of the chamber where liquids could be delivered via tubing attached to a 60 mL syringe placed in a motorized pump outside the cabinet. Port entries and exits were detected by infrared beams located within the recessed port. A white houselight (28 V) was located on the wall opposite the recessed port along with a white noise generator. Outside the behavioral chamber but within the cabinet was a red houselight (28 V) that provided background illumination during each behavioral session.

Occasion setting

Rats were initially trained to drink reward freely from the port in a single session where the reward pump was randomly activated 80 times for 2 sec (∼0.08 mL per delivery) with a 60 sec variable time schedule. Conditioning began the following day with 30 trials with a 200 sec average (100–300 sec range) inter-trial interval. We gradually introduced different trial types in this task as pilot studies indicated rats failed to learn if presented with all trial types from the onset of training. Initially, for each trial, the white houselight (occasion setter; OS) was illuminated for 5 sec, followed by a 5 sec gap with no stimuli, then the white noise generator (conditioned stimulus; CS) was active for 5 sec, and, finally, upon CS termination, the reward pump was active for 5 sec delivering ∼0.2 mL of 15% sucrose reward. There was one session a day with each session lasting ∼2 h. Following four sessions, rats began discrimination training where 12 trials were reinforced as before, but the remaining 18 trials were nonreinforced presentations of the CS alone. After six more sessions, rats proceeded to the full occasion setting task where 10 trials were reinforced, 10 were nonreinforced presentations of the CS alone, and 10 were nonreinforced presentations of the OS alone, with trial type determined pseudorandomly, such that no trial type occurred more than four times in succession. Rats were trained for eight sessions in the full task prior to either extinction or conditioned reinforcement tests.

The OS was a visual cue and the CS was an auditory cue as this arrangement has been previously found to promote the development of occasion setting and maximize differences in behavior (Holland 1992). In particular, the use of an auditory cue for the CS encourages food cup approach (Holland 1977), and given the cue is only presented for 5 sec, this encouraged an easily quantifiable and unbiased metric of conditioned reward-seeking.

Rats in the unpaired condition received an identical number of trial types in all phases with conserved timing of presentation of cues, but reward was delivered according to a separate ITI schedule that matched the rate of reward delivery in the paired condition.

For rats undergoing extinction of the OS, following the eighth day of training in the full occasion setting task, they were first tested in a session under extinction conditions where reward delivery was withheld. The OS was then extinguished across four sessions by presenting the OS alone for 30 unrewarded trials per session. The day after the last OS extinction session the rats were tested again in the final occasion setting task also without reward delivery. The following day rats proceeded to conditioned reinforcement testing without rewarded retraining in the occasion setting task.

Conditioned reinforcement

Each conditioned reinforcement test lasted 40 min during which levers on either side of the recessed port were extended or nose pokes were available for responding. Each rat received two tests for different cues on one operant (e.g., CS alone and OS + CS) and then two tests on the other operant (e.g., OS + CS and OS alone), with a test to earn the OS + CS conducted on each. There was only cue available to be earned during each test. The order of testing, the identity (nosepoke vs. lever), and the side (left vs. right) of the active and inactive operant were counterbalanced and reversed between tests on the same operant. This required rats to acquire a unique operant response during each test. There was at least 1 d without testing between each conditioned reinforcement test. In OS alone tests, responses on the active operant produced a 2 sec presentation of the houselight OS. In CS alone tests, responses on the active operant produced a 2 sec presentation of the white noise CS. In OS + CS tests, active responses produced simultaneous 2 sec presentation of the houselight OS and the white noise CS; we presented these cues simultaneously as brief cue presentations promote conditioned reinforcement (Taylor and Robbins 1984; Robinson and Flagel 2009; Fraser et al. 2016; Saunders et al. 2018) and because we surmised that a time gap in their presentation during free operant responding would make it difficult for the subject to link their responses to serial cue presentation. In each test, responses on the inactive operant were without consequence.

Statistical analysis

Linear mixed-models were used to assess behavior across training using SPSS 24 (IBM) with session and trial type as repeated measures and group was a between subjects factor. Time in port was normalized by subtracting average time in port during a 10 sec period prior to the onset of the first cue during a trial from time during the CS period. Repeated measures ANOVA were used to analyze the impact of extinction of the OS on behavior. For extinction tests, we examined the microstructure of reward port approach by calculating the probability of observing a given rat in the reward port across all trials in 1 sec bins, and then averaging these across all rats. Bias scores were calculated by subtracting responding during the CS period on either CS or OS alone trials from OS + CS trials and dividing by the sum of these values (e.g., ([OS + CS] − OS Alone)/([OS + CS] + OS Alone)) giving a score between 1 and −1, with a value of 1 representing perfect discrimination in responding exclusively on OS + CS trials relative to either CS alone or OS alone trials. One-way repeated measures ANOVA were used to analyze active to inactive responding, cues earned, and port entries in the conditioned reinforcement tests. There were 2 OS + CS tests for each rat, one on each operant, and we averaged responding across these tests. For conditioned reinforcement, active to inactive responding was analyzed two ways (1) using ANOVA to compare between tests and (2) using Wilcoxon signed-rank tests to assess if active ratios were significantly different than random responding (median value of 0). In cases where data were nonnormally distributed, nonparametric tests were used. One rat was excluded from the paired group for the CS alone test due to his responding on all measures being an outlier as detected by Grubbs’ test. Post-hoc comparisons were conducted when significant main effects and interactions were observed. For all analyses, α = 0.05.

Competing interest statement

The authors have no biomedical financial interests or potential conflicts of interests.