Stress Transiently Affects Pavlovian-to-Instrumental Transfer.

Stress has a strong impact in the brain, impairing decision-making processes as a result of changes in circuits involving the prefrontal and orbitofrontal cortices and the striatum. Given that these same circuits are key for action control and outcome encoding, we hypothesized that adaptive responses to which these are essential functions, could also be targeted by stress. To test this hypothesis we herein assessed the impact of chronic stress in a Pavlovian-to-instrumental transfer (PIT) paradigm, a model of an adaptive response in which a previously conditioned cue biases an instrumental goal-directed action. Data reveals that rats submitted to chronic unpredictable stress did not display deficits in pavlovian conditioning nor on the learning of the instrumental task, but were impaired in PIT; importantly, after a stress-free period the PIT deficits were no longer observed. These results are relevant to understand how stress biases multiple incentive processes that contribute to instrumental performance.

Introduction

Exposure to a stressful stimulus activates a physiological response intended to restore the organism’s homeostasis. However, when stressors are maintained for long periods of time, this response becomes maladaptive and results in several disruptions at the level of the regulatory systems, of which the brain is a key element. Thus, it is not surprising that chronic stress exposure is associated with significant behavioral impairments such as deficits in spatial reference and working memory (Mizoguchi et al., 2000; Cerqueira et al., 2007), behavioral flexibility (Cerqueira et al., 2007), anxiety (Pêgo et al., 2006), and mood (Bessa et al., 2009). Such functional deficits are paralleled by structural changes in several brain regions (Sousa and Almeida, 2002), that render chronic stress as an important risk-factor for the development of several neuropsychiatric disorders. Importantly, several studies have also shown that the behavioral and structural effects of stress are transient, and important plastic phenomena take place after the removal of the stressful stimuli (Sousa et al., 2000; Bloss et al., 2010).

Recent studies from our laboratory show that chronic stress bias decision-making processes, by favoring the shift from goal-directed actions to habit based behaviors (Dias-Ferreira et al., 2009). These alterations in instrumental behavior are correlated with changes in neuronal circuits involving different areas of the prefrontal cortex (PFC) [including the medial PFC (mPFC) and orbitofrontal cortex (OFC)] and dorsal striatum (Dias-Ferreira et al., 2009). Given that these regions are implicated in action control (Balleine and O’Doherty, 2010) and outcome encoding necessary for adaptive responses (Chudasama and Robbins, 2003; Hornak et al., 2004), we hypothesized that stress-induced changes in neuronal circuits involving the PFC and the striatum could lead to outcome encoding deficits and to changes in the multiple incentive processes that contribute to instrumental performance.

One of these processes is Pavlovian-to-instrumental transfer (PIT; Estes, 1948; Colwill and Rescorla, 1986). PIT encompasses three distinct components:(1) Pavlovian learning, in which stimuli are associated with rewards; (2) instrumental conditioning, in which associations between actions and out comes are learned; and (3) a test phase, in which the impact of previous cues on instrumental response is assessed. The associative value of cue and its motivational significance are determinants found crucial to proper transfer, a phenomenon which resembles cue-mediated increased drive seeking for drugs seen in drug abusers (Dickinson et al., 2000; Corbit and Janak, 2007). Because of that, PIT has been used as a useful model of maladaptive learning observed in several conditions, namely addictive disorders.

The neural basis of PIT is not completely established but several studies implicate regions associated with emotional processing [amygdala and nucleus accumbens (NAc)], executive commands (dorsal striatum), and their integration (mPFC and OFC); importantly, it is known that key regions involved in PIT operate in parallel. In fact, lesion studies in the amygdala and NAc have demonstrated that these brain regions are necessary for the behavioral expression of PIT (Corbit et al., 2001; Hall et al., 2001; de Borchgrave et al., 2002; Holland and Gallagher, 2004). Moreover, it is relevant to note that different regions of the amygdala and NAc display different roles in this process. While the amygdala basolateral nucleus (BLA) mediates the association between specific sensory and emotional features of stimulus and the responses that are elicited by each one (consummatory conditioning), the central nucleus (CN) mediates the association between cues and affective properties of stimuli (preparatory conditioning; Killcross et al., 1997; Balleine and Killcross, 2006). In what concerns the NAc, the core mediates the general excitatory effects of reward-related cues, whereas the shell mediates the effect of outcome-specific reward predictions on instrumental performance (Corbit and Balleine, 2011). In addition, these regions are known to regulate the activity of cortical sites integrating affective stimuli with executive commands, such as the mPFC and the OFC (Christakou et al., 2004; Kelley, 2004; Pasupathy and Miller, 2005; Saddoris et al., 2005; Stalnaker et al., 2007).

Surprisingly, given the fact that stress influences many of the above mentioned areas involved in PIT, the impact of stress in PIT is largely unknown. Indeed, while acute stress was shown to enhance Pavlovian learning (Shors et al., 2000) and chronic stress failed to influence instrumental learning (Dias-Ferreira et al., 2009), no study addressed the effect of either acute or chronic stress on the interaction between these two key processes and thus on the impact of conditioned clues in goal-directed behavior. Thus, in the present study, we tested the impact of chronic stress in the modulation of instrumental behavior according to cues previously associated with rewards by studying the behavior of control and stressed rats in a PIT paradigm using a two-lever operant chamber. Moreover, to assess whether the impairments are reversible after chronic exposure to stress, PIT was also assessed after a period free of exposure to stressful stimuli.

Materials and Methods

Animals

All experiments were conducted in accordance with local regulations (European Union Directive 86/609/EEC) and National Institutes of Health guidelines on animal care and experimentation and approved by Direção Geral Veterinária (DGV; the Portuguese National Institute of Veterinary).

Thirty-two adult male Wistar rats (Charles River Laboratories, Barcelona, Spain; 250–300 g at the start of the experiment), aged 3 months and weighing 400–500 g, were housed in groups of two under standard laboratory conditions with an artificial light-dark cycle of 12:12 h (lights on from 8:00 a.m. to 8.00 p.m.) in a temperature- and humidity-controlled room. Animals were given 2 weeks to acclimate to the housing conditions with ad libitum access to food and water. A food deprivation regimen was initiated 24 h before the initiation of training and testing to maintain the subjects at approximately 90% of their free-feeding body weight. Rats had free access to water while in the home cage.

Chronic unpredictable stress paradigm

Animals assigned to the chronic unpredictable stress (CUS) group were exposed during 60 min once a day to one of five different stressors: cold water (18°C), vibration, restraint, overcrowding, and exposure to a hot air stream. Stressors were randomly distributed throughout a 28-day period. This type of chronic stress paradigm, mixing different stressors (including physical and psychological components) presented in an unpredictable schedule, was shown previously to result in persistently elevated plasma levels of corticosterone (for details, see Sousa et al., 1998) and is thought to better mimic the variability of stressors encountered in daily life (Sousa et al., 1998). Controls were handled daily during the same period.

To assess the impact of chronic stress exposure in Pavlovian-instrumental transfer but also ascertain its reversibility, a first group of animals (eight stressed and eight controls) were behaviorally characterized immediately after stress while a similar group (eight stressed and eight controls) was left to recover for 6 weeks before being tested. This recovery period was set-up in light of previous studies showing that, at least, 4 weeks are necessary to complete reversion of behavioral and structural changes induced by CUS treatment (Sousa et al., 2000). Importantly, animals were randomly allocated to each of the four groups before the beginning of stress exposure.

Pavlovian-Instrumental Transfer

Behavior was assessed using the Pavlovian-instrumental transfer protocol as described by Ostlund and Balleine (2007). This task took place in operant chambers (30.5 cm L × 24.1 cm W × 21.0 cm H, MedAssociates, CA, USA) housed within sound attenuating cubicles. Each chamber was equipped with two retractable levers on either side of the food magazine and a house light (3W, 24V) mounted on the opposite side of the chamber. Reinforcers were delivered into the magazine through a pellet dispenser that delivered 45 mg regular “chow” pellets or a liquid dipper that delivered 0.1 ml of 20% sucrose solution. A computer equipped with MED-PC IV software controlled the equipment and recorded lever presses and head entries. As described previously, animals were placed in a food deprivation schedule, having access to food during 1 h per day after the training or testing session, allowing them to maintain a body weight above 90% of their baseline weight. Water was removed for 2 h before each daily session.

Training began with eight daily sessions of Pavlovian conditioning in which each of two auditory conditioned stimuli (tone and white noise) were paired with a different outcome (pellets and sucrose). Each CS was presented four times per session using a pseudo-randomized order and a variable ITI (mean 5 min). In the ninth day, animals were submitted to an outcome devaluation to ensure they were able to associate each outcome to the conditioned stimulus; this was assessed by comparing the number of head entries into the food dispenser during stimuli presentation and during ITI.

Animals were then trained to obtain two different outcomes (pellets and sucrose) by pressing left and right levers. Training was performed in two separate daily sessions and the order of training was alternated during days (average interval between the two daily sessions was 3 h). Each session finished after 15 outcome deliveries or 30 min. In the first 2 days, lever pressing was continuously reinforced (CRF) which means that each action resulted in one outcome delivered (p = 1.0). The probability of getting a reward decreased according the following sequence: days 3–4, p = 0.2; days 5–6, p = 0.1; and days 7–9, p = 0.05.

Two sessions of outcome devaluation (by free access to the reward until satiety) were then performed, 48 h apart. In order to do this, one of the two outcomes (pellets or sucrose) was given ad libitum during 1 h before each session. Then the rats were placed during 5 min into the testing operant chamber where both levers were inserted but no outcome was delivered.

Forty-eight hours later, subjects were placed in the operant chamber to test Pavlovian-instrumental transfer with both levers inserted. After an initial period of response extinction that lasts for 8 min, four blocks of each auditory conditioned stimulus were presented randomly over the next 40 min and lever presses were registered. During each stimulus presentation, lever presses where considered correct if encoded the same reward as the audible sound. When different, actions were considered incorrect.

Statistical analysis

Results are expressed as group means ± SE. Pavlovian, instrumental behavior and results of transfer test were compared between and within groups using two-way ANOVA. Differences were considered to be significant if p < 0.05.

Results

Pavlovian training resulted in conditioning of animals both in control and stress groups. Comparison of head entries on CS presentation and on ITI during Pavlovian training (Figure 1) shows that all animals associate the stimuli to the outcome[head entries: F(1,28) = 88.762, p < 0.001] without differences between experimental groups [stress exposure: F(1,28) = 0.163, p = 0.689], thus implying that CUS does not affect Pavlovian conditioning.

Figure 1

Pavlovian conditioning. There were no differences between groups, with all animals increasing the number of head entries during conditioned stimulus exposure. ITI – intertrial interval between presentations of conditioned stimuli CS – conditioned stimulus.*p < 0.05.

In what regards to instrumental training, the number of lever presses per minute increased during the task indicating that animals in both groups can learn it equally well (Figure 2). This is confirmed by the results of the outcome devaluation test performed at the end of instrumental conditioning, in which animals of both groups [stress exposure: F(1,28) = 1.019, p = 0.321]could correctly associate each reward to a specific lever[lever: F(1,28) = 25.787, p < 0.001; Figure 2. These results are in accordance with our previous data (Dias-Ferreira et al., 2009) showing that CUS does not impair outcome devaluation when performed early during the period of training.

Figure 2

Instrumental conditioning. Both experimental groups acquired the lever pressing task and were able to correctly distinguish devalued from valued levers.*p < 0.05.

Subsequently, we assessed the Pavlovian-instrumental transfer. Figure 3 displays the number of lever presses per minute when the conditioned sound predicted the same outcome as the response (same) and the number of lever presses per minute when the conditioned sound predicted a different outcome (diff). Our results (Figure 3A) show that stress significantly impairs the transfer [stress exposure: F(1,28) = 5.397, p = 0.028], preventing exposed animals, contrary to controls, [interaction: F(1,28) = 7.558, p = 0.010] from associating levers to the corresponding sound cues [lever: F(1,28) = 7.630, p = 0.010].

Figure 3

Stress-induced impairment of Pavlovian-to-instrumental transfer is reversible. Stress exposure resulted in an impairment of the association between Pavlovian and instrumental behavior (A), which was no longer observed after a 6-week period without exposure to stressful stimuli (B). Same – conditioned stimulus (sound) predicted the same outcome as the lever pressed; Diff – conditioned stimulus (sound) predicted a different outcome as the lever pressed. *p < 0.05.

Importantly, we also assessed whether these stress-induced effects were sustainable in time after the end of the exposure to stress and found that these effects of stress were reversible. In fact, a similar assessment of stressed-recovered animals and controls (Figure 3B) failed to show any difference between groups [stress-recovery exposure: F(1,28) = 0.976, p = 0.332], with all animals from both groups [interaction: F(1,28) = 0.178, p = 0.676] being able to associate conditioned sound and appropriate responses[matching vs. non-matching lever: F(1,28) = 18.217, p < 0.001].

Discussion

The present results show for the first time that chronic stress disrupts the modulation of instrumental responses by conditioned cues and that these stress-induced impairments are transient, being absent after a 6-weeks recovery period. This is of relevance for decision-making, as it is well established that environmental cues can have a strong modulatory effect upon instrumental responses (Estes, 1948), which are the basis of most decision-making processes.

Chronic stress has a strong modulatory influence (either negative or positive) on learning processes, including spatial memory (Sousa et al., 2000), working memory (Mizoguchi et al., 2000; Cerqueira et al., 2007), and behavioral flexibility (Cerqueira et al., 2007), but also in decision-making processes by biasing instrumental actions to habits (Dias-Ferreira et al., 2009). In the present study we show that chronic stress does not affect Pavlovian conditioning nor instrumental learning. Although the effects of chronic stress upon Pavlovian conditioning have never been reported, the latter finding is in accordance with a previous report showing that chronic stress promotes the transfer from goal-directed actions to habit based behaviors without affecting instrumental learning per se (Dias-Ferreira et al., 2009). Indeed, in that study, when tested on a devaluation paradigm after 8 days of training (similar to the present protocol), stressed animals were still able to effectively suppress the devaluated response, which was not the case when the test was performed later during training (Dias-Ferreira et al., 2009).

Since neither Pavlovian conditioning nor instrumental learning are affected by chronic stress, the most likely explanation for the herein observed stress-induced impairment of PIT seems to be a deficit in the transfer between the two networks. The precise neuronal networks implicated in PIT are still being described. As stated before, several regions that are known to be susceptible to chronic stress are crucial to PIT. In fact, several studies demonstrate that the mPFC and OFC encode distinct components of both Pavlovian and instrumental processes (Gallagher et al., 1999; Chudasama and Robbins, 2003; Ostlund and Balleine, 2007; Homayoun and Moghaddam, 2008) and a recent study reveals that the OFC and mPFC orchestrate the integration of Pavlovian and instrumental processes during PIT (Homayoun and Moghaddam, 2009). This integration involves distinct operations as mPFC and OFC display predominantly inhibitory and excitatory phasic responses to the same events, respectively. Taken into account our previous observations that stress triggers atrophy in the mPFC and hypertrophy in the OFC (Dias-Ferreira et al., 2009), we suggest the existence of an imbalance in these inhibitory/excitatory responses and, as a consequence, a failure in the reinforcement of goal-directed actions by conditioned stimuli.

While the excitatory response of lateral OFC neurons may signify a positive motivational signal associated with the expected reward (Tremblay and Schultz, 1999; Schoenbaum et al., 2003), the inhibitory response of mPFC neurons evokes their pattern of activity in goal-directed actions (Homayoun and Moghaddam, 2006; Moghaddam and Homayoun, 2008). The fact that there is a re-emergence of inhibitory pattern in prelimbic mPFC neurons re-activates its representation of instrumental action under the influence of the Pavlovian incentives (Homayoun and Moghaddam, 2009). This integration of Pavlovian and instrumental processes, where cue-evoked incentives recruit instrumental representations, may provide a mechanism for the prelimbic mPFC, to execute motivational control over goal-directed behavior; importantly, this re-activation is likely to be compromised after chronic stress as the present results demonstrate. Of course, other regions targeted by stress, such as the striatum, could also be implicated in the stress-induced PIT impairment; in fact, there are studies demonstrating that the dorsolateral striatum is critical for the formation of specific stimulus-outcome associations, whereas the dorsomedial striatum is involved in the formation of specific response-outcome associations. Disruption of either form of learning impairs PIT (Corbit and Janak, 2010) and stress is known to influence the structure and function of both divisions of the dorsal striatum (Dias-Ferreira et al., 2009). In the same vein, ventral striatal areas could also play an important role in the observed impairments as integrity of NAc shell was found to be critical to the transfer effect (Corbit et al., 2001). In fact, previous studies of our lab showed that stress reduces the total volume of this region, impairing its function (Leão et al., 2007). These reported alterations could underlie impairments we have found.

Additionally, current evidence suggests that BLA is involved in the formation of stimulus-reward associations by assigning an affective value to associated rewards (Everitt et al., 1991) and in the production and direction of instrumental actions (Everitt and Robbins, 1992). Although BLA lesions completely abolish both outcome-selective PIT and outcome devaluation, this area integrates different circuits that connect differently with other relevant brain structures. Anterior BLA connects with OFC and shell NAc and posterior BLA connects with prelimbic cortex, medial accumbens core, and key components of instrumental conditioning circuitry (Balleine, 2005). Structural stress-induced alterations described by Vyas et al. (2002) could configure an interesting possibility to explain our results.

Stress has a strong impact in hippocampal structure and function, impairing the learning and storage of newly acquired information (Sousa et al., 2000). In this regard, the herein observed PIT deficits could be due to a disruption of these hippocampal functions, interfering with the consolidation of stimulus-outcome associations. Alternatively, the stress-induced hippocampal dysfunction could also interfere with the hippocampal role in appetitive Pavlovian conditioning (Ito et al., 2005). However, neither of these hypotheses is supported by the fact that chronic stress did not impair Pavlovian conditioning.

Importantly, the stress-induced impairment of PIT was no longer evident after a stress-free period. This reversibility of stress effects is in accordance with previous studies showing the recovery of other stress-induced deficits, including spatial memory (Sousa et al., 2000), and behavioral flexibility (Bloss et al., 2010), after similar stress-free periods. Of note, recovery of these functions is paralleled by synaptic regrowth and reorganization on the hippocampus and the mPFC, which are also involved in PIT. Altogether, these results highlight the extreme plastic capabilities of areas involved in PIT and explain why most stress-induced deficits, including those described in the present paper, are, at least in part, reversible. A better knowledge of the mechanisms underlying these events, to be pursued in future studies, is crucial to optimize therapeutic interventions in altered cue-controlled behaviors, particularly in those situations in which spontaneous recovery is not likely.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.