Normal Aging does Not Impair Orbitofrontal-Dependent Reinforcer Devaluation Effects.

Normal aging is associated with deficits in cognitive flexibility thought to depend on prefrontal regions such as the orbitofrontal cortex (OFC). Here, we used Pavlovian reinforcer devaluation to test whether normal aging might also affect the ability to use outcome expectancies to guide appropriate behavioral responding, which is also known to depend on the OFC. Both young and aged rats were trained to associate a 10-s conditioned stimulus (CS+) with delivery of a sucrose pellet. After training, half of the rats in each age group received the sucrose pellets paired with illness induced by LiCl injections; the remaining rats received sucrose and illness explicitly unpaired. Subsequently, responding to the CS+ was assessed in an extinction probe test. Although aged rats displayed lower responding levels overall, both young and aged rats conditioned to the CS+ and developed a conditioned taste aversion following reinforcer devaluation. Furthermore, during the extinction probe test, both young and aged rats spontaneously attenuated conditioned responding to the cue as a result of reinforcer devaluation. These data show that normal aging does not affect the ability to use expected outcome value to appropriately guide Pavlovian responding. This result indicates that deficits in cognitive flexibility are dissociable from other known functions of prefrontal - and particularly orbitofrontal - cortex.

Introduction

Adaptive behavior requires the ability to rapidly update patterns of responding following shifts in contingencies. Normal aging is associated with a decline in this type of cognitive flexibility (Rapp and Heindel, 1994; Lamar and Resnick, 2004; Marschner et al., 2005; Weiler et al., 2008), and numerous studies in animals have demonstrated age-related deficits in flexible decision making processes thought to depend on prefrontal function (Zyzak et al., 1995; Schoenbaum et al., 2002a; Nicolle and Baxter, 2003; Smith et al., 2004; Joly et al., 2006; Brushfield et al., 2008; LaSarge et al., 2009; Mizoguchi et al., 2009, 2010; Simon et al., 2010).

Prominent among these reports are deficits in reversal learning. Notably, reversal learning deficits are prominently associated with orbitofrontal cortex (OFC) dysfunction in humans, non-human primates, and rats (Rolls et al., 1994; Meunier et al., 1997; Schoenbaum et al., 2002b; Chudasama and Robbins, 2003; Fellows and Farah, 2003; Izquierdo et al., 2004b; Marschner et al., 2005; Mell et al., 2005; Brushfield et al., 2008; Walton et al., 2010). Although the precise basis of this learning deficit is unclear, it is likely to reflect the OFC's role in signaling associations between predictive cues and expected outcomes (Rolls et al., 1996; O'Doherty et al., 2000, 2002; Gottfried et al., 2003; Schoenbaum et al., 2003; Lara et al., 2009). Accordingly, aged rats displaying reversal impairments exhibit fewer cue-selective neurons in OFC during a cued-odor discrimination task, and neurons that were cue-selective are markedly less flexible upon reversal (Schoenbaum et al., 2006b).

Of course, cue-outcome associations encoded in OFC are important not only for learning but also for guiding behavior. This is evident in the critical role OFC plays in experimental settings in which normal behavior requires access to specific information about the expected outcome, even in the absence of any specific requirement for new learning (Mobini et al., 2002; Pickens et al., 2003a, 2005; Izquierdo et al., 2004b; Winstanley et al., 2004; Ostlund and Balleine, 2007; Burke et al., 2008). Particularly relevant in this regard is the role of OFC in mediating the normal spontaneous decline in Pavlovian conditioned responding caused by reinforcer devaluation. Lesions of OFC prevent this normal decline, even when it is conducted only at the time of testing, after both the cue-outcome association and the taste aversion have been successfully acquired. This suggests that OFC plays a critical role in the final process whereby outcome expectancies are integrated with the new outcome value to guide performance (Schoenbaum and Esber, 2010).

In the present study, we used Pavlovian reinforcer devaluation to test whether normal aging also affects the use of cue-outcome associations to guide behavior. Young and aged rats underwent Pavlovian conditioning followed by devaluation of the food reinforcer by pairing with illness. Subsequently, responding to the cue was assessed in an extinction probe test. While there were minor differences in the overall responding (conditioned and baseline) in the two groups, both young and aged rats showed comparable declines in conditioned responding after devaluation of the predicted outcome. These data suggest that within normal aging, the ability to use information about the expected outcome to appropriately guide behavior is preserved whereas the use of this information to facilitate learning can be impaired. This dissociation has significant implications for the underlying circuit abnormalities.

Materials and Methods

Subjects

Male Long-Evans rats in the aged group (n = 17) were acquired at approximately 10 months of age (Charles River Laboratories, Wilmington, MA, USA) and housed for ∼1 year in preparation for the experiment. During this time they were handled weekly. Testing began when they were 22–24 months of age. Young controls (n = 24) were acquired at approximately 3 months of age approximately 2 weeks prior to testing (Charles River Laboratories, Wilmington, MA, USA). During this time they were handled daily. During testing, all rats were individually housed and were given ad libitum access to food and water, except during testing. Beginning 5 days before testing and continuing until testing ended, rats were food deprived to 85% of their baseline body weight. For maintenance at 85% during training, a constant food quantity was used for each group: roughly 3 pellets (15 g) of chow for young; 4.5–5 pellets (22–25 g) for aged. Most importantly, the food restriction between aged-matched control groups were not different, and neither were their testing body weights. The animals were placed on a 12-h light/dark cycle and tested only during the light cycle. Rats were tested at the University of Maryland, School of Medicine in accordance with University of Maryland and NIH guidelines.

Apparatus

Sixteen standard behavior boxes (12′′ × 10′′ × 12′′) in sound attenuating cubicles were used for testing (Coulbourn Instruments, Allentown, PA, USA). A recessed food cup was placed in the center of the right wall approximately 2 cm above the floor. A feeder, mounted outside of the behavioral box, contained 45 mg grape sucrose pellets (Bio-Serv, Frenchtown, NJ, USA) and was connected to the food cup. The house light was mounted above the food cup.

Pavlovian conditioning

All behavioral training procedures are outlined in Table 1. After rats reached 85% of their baseline body weight, they were trained to eat from the food cup over 2 days during daily 64 min shaping sessions that each included 16 deliveries of three 45 mg pellets (Bio-Serv, Frenchtown, NJ, USA). Following food cup shaping, rats underwent 10 days of Pavlovian conditioning. Rats received 16 10 s presentations of the house light (CS+) followed by delivery of one sucrose pellet. Inter-trial intervals varied from 3 to 5 min. The 10-s pre-CS period was used to calculate baseline responding.

Table 1

Outline of experimental procedures.

Group	Light conditioning	Reinforcer devaluation	Probe
Young	HL → pellet	Devalued: pellets → LiCl	HL
		Non-devalued: pellets; LiCl
Aged	HL → pellet	Devalued: pellets → LiCl	HL
		Non-devalued: pellets; LiCl

HL, house light; LiCl, lithium chloride.

Reinforcer devaluation

Following conditioning, rats were matched for performance and divided into devalued and non-devalued groups. On days 1 and 3, non-devalued groups were given 10 min access to a ceramic bowl containing 100 sucrose pellets in their home cage. On days 2 and 4, devalued groups were given 10 min access to a ceramic bowl containing 100 sucrose pellets, and immediately following this consumption period, they were given an intraperitoneal (i.p.) injection of 0.3 M LiCl. Non-devalued rats also received i.p. injections of 0.3 M LiCl on days 2 and 4 but this was not contiguously paired with sucrose pellets.

Probe test

Following reinforcer devaluation (but prior to the final consumption test), the rats were given a probe test that was exactly the same as Pavlovian conditioning outlined above, except that this test was run under extinction conditions (i.e., at the end of CS+ presentation, no pellets were delivered).

Response measures

We measured percent time spent in the food cup with an infrared photo beam positioned at the front of the food cup. For purposes of analysis, we examined the last 5 s of the CS+. Previous reports have demonstrated that responses are confined to this segment of the CS+ (Pickens et al., 2003b).

Data analysis

Data was collected using Graphic State 2 software from Coulbourn Instruments (Allentown, PA, USA). Then, the data was processed in Matlab to extract response rates and percent time spent in the food cup during CS presentations. Finally, the data was analyzed using Statistica, version 9.

Results

Aged (n = 17) and young (n = 24) rats were trained in a Pavlovian devaluation task. An experimental timeline is shown in Table 1. Training started with 10 days of Pavlovian conditioning. As illustrated in Figure 1A, conditioned responding in the last 5 s of the CS+ cue increased in both groups across sessions. In addition, aged rats showed a general reduction in responding, evident during the pre-cue baseline and during the cue. Consistent with this interpretation, a three-factor ANOVA (age × session × cue/pre) revealed significant main effects of session (F9,351 = 83.62, p < 0.001), age (F1,39 = 36.89, p < 0.0001) and cue/pre (F1,39 = 215.46, p < 0.0001), and significant interactions between session and cue/pre (F9,351 = 71.74, p < 0.0001) and between session and age (F9,351 = 2.15, p < 0.05). Notably, there was no interaction effect between age, session, and cue/pre (F9,351 = 1.49, p = 0.15), suggesting that the overall pattern of conditioning to the cue (versus pre-cue baseline) did not differ between the two groups. Accordingly post hoc testing revealed that young rats responded significantly more both during the last 5 s of the CS+ and during the 10-s pre-CS baseline period compared to aged rats (p < 0.01).

Figure 1

Comparison of performance of young and aged rats across Pavlovian conditioning (A), reinforcer devaluation (B), and conditioned responding after devaluation (C). Data presented are the mean and SE.

Following Pavlovian conditioning, young and aged rats were performance matched and divided into devalued and non-devalued groups. Rats then underwent reinforcer devaluation, which took place in the rats’ home cage. As illustrated in Figure 1B, pairing of the food with illness resulted in a decline in food consumption in both groups. A three-factor ANOVA (age × trial × pairing) confirmed this impression, revealing a significant main effect of trial (F2,74 = 66.36, p < 0.0001), age (F1,37 = 6.35, p < 0.05), and pairing (F1,37 = 131.37, p < 0.0001) and a significant trial by pairing interaction (F2,74 = 95.17, p < 0.0001). Post hoc testing revealed that, on average, aged rats ate fewer pellets than young rats (p < 0.05); however, paired rats in each group ate significantly fewer pellets than unpaired rats (p < 0.001).

After reinforcer devaluation (and prior to the final consumption test, which occurred after completion of all testing), all rats underwent a probe test in which they were exposed to the CS+ again under extinction conditions. As illustrated in Figure 1C, reinforcer devaluation had a significant effect on conditioned responding in both aged and young rats. A two-factor ANOVA (age × pairing) revealed significant main effects of age (F1,37 = 8.01, p < 0.01) and pairing (F1,37 = 11.15, p < 0.01. Planned comparisons demonstrated that both aged and young non-devalued rats demonstrated more conditioned responding than aged and young devalued rats (p < 0.05), and there was no significant interaction between age and pairing (F1,37 = 0.014, p = 0.91).

Discussion

Here, we tested the hypothesis that aging disrupts the ability to update cue-evoked information to guide behavior. We tested this hypothesis using a Pavlovian reinforcer devaluation task thought to depend on OFC in rats, monkeys, and humans (Gallagher et al., 1999; Gottfried et al., 2003; Izquierdo et al., 2004a; Machado and Bachevalier, 2007). We predicted, based on previous reports of OFC dysfunction in aged rats (Schoenbaum et al., 2002a, 2006b; Mizoguchi et al., 2010), that aged rats might be incapable of updating the value of the outcome to guide subsequent behavior.

Both young and aged rats underwent Pavlovian conditioning followed by devaluation of the food reinforcer by pairing with illness. Subsequently, responding to the cue was assessed in an extinction probe test. Young and aged rats conditioned to the cue and developed a conditioned taste aversion. Further, contrary to our hypothesis, both young and aged rats spontaneously attenuated conditioned responding to the cue as a result of reinforcer devaluation. These data show that normal aging does not affect the ability to use expected outcome value to appropriately guide Pavlovian responding. This result suggests that deficits in cognitive flexibility are dissociable from other known functions of prefrontal – and particularly orbitofrontal – cortex.

The current negative results are notable in light of prior reports by us and others that some aging disrupts reversal performance likely to depend on OFC (Schoenbaum et al., 2002a; Joly et al., 2006; Mizoguchi et al., 2010). In our own hands, these reversal deficits were observed in a significant proportion of a population of male Long-Evans rats at the same age as those in the current study (Schoenbaum et al., 2002a). This reversal deficit was orthogonal to declines in hippocampal function, assessed in the same rats in a well-established water maze task (Schoenbaum et al., 2002a), and later work showed that reversal deficits were associated with abnormally sparse and rigid representations of the cue-outcome associations in the OFC (Schoenbaum et al., 2006a). Specifically, reversal-impaired aged rats had fewer cue-selective neurons in the task, and the remaining cue-selective neurons were markedly less flexible than neurons of young rats. Furthermore, OFC neurons identified as outcome-expectant (defined by differential firing on positive and negative trials during the delay after a response but prior to outcome delivery; Schoenbaum et al., 1998, 2006b), were significantly less labile within the aged-impaired rats. Thus, outcome-expectant neurons in aged rats did not become selective for the appropriate odor cue after reversal learning. Based on these results, we expected to find that aged rats would also be impaired at using these cue-outcome associations to appropriately guide behavior in the Pavlovian reinforcer devaluation task used in the current study. However the results clearly show this is not the case.

There are a number of possible explanations for this apparent dissociation. One possible explanation is simply that our sample size and approach was not powerful enough to reveal the deficit we expected. In particular, a between subjects design is not optimal for revealing an effect on the performance of individual subjects. It is possible that a multiple-outcome within-subjects design would reveal deficits in some individual rats not evident here. We view this as unlikely given the robust overall response, but it cannot be excluded. Additionally, we did not characterize these particular rats on the reversal learning task used in the prior studies, thus it is possible that our population did not include reversal-impaired rats. Again we view this as unlikely, given our prior experience with the same age Long-Evans rats, which exhibited robust reversal impairments (and aberrant encoding in OFC) in two prior experiments with similar or smaller population sizes. However, it is not possible to fully rule out this explanation.

A second possible explanation is that the intact performance of the aged rats reflects an artifact due to differences in initial weights of the age groups. Aged rats were significantly larger at the time of testing (aged rats 760.2 g; young rats 270.5 g). Such radical differences in weight might have led to much higher motivation for the food in the aged rats after food restriction. However both groups were maintained at 85% of their free feeding weight, and the difference in food required to maintain this weight was negligible (3 pellets for young versus 4.5–5 pellets for aged). Further both groups showed similar (and quite robust) development of a conditioned taste aversion across trials. As a result it seems unlikely that the aged rats showed normal effects of devaluation on conditioned responding due to a higher motivation for the food. Notably the weight difference also yielded a significant difference in the volume of LiCl injection between the aged and young rats (1.37 ml for young versus 3.85 for aged), although the dose was the same for all rats (0.3 M LiCl). High volume could have induced acute pain in the aged rats, perhaps allowing neural circuits to support devaluation that were not engaged in young rats. If these circuits do not include OFC, this would produce the current pattern of result. However we did not observe evidence of overt pain or discomfort in the reaction of aged rats to the larger injections; both groups showed signs consistent with nausea. And the volume of LiCl infusion was uncorrelated with measures of devaluation in either age group (i.e., change in pellets consumed, total number of pellets consumed, or CS-evoked responding during probe testing).

A third possible explanation is that the normal response to devaluation here is a result of the strain of rat used. Militating against this criticism is the robust effects that any significant damage to OFC has on devaluation in this particular devaluation task (Gallagher et al., 1999; Pickens et al., 2003b, 2005). Additionally although effects of aging sometimes differ from what is observed in Long-Evans rats used here, the Long-Evans rats are arguably the most likely to show OFC dysfunction based on the literature. For example although Fischer 344 rats exhibit odor discrimination deficits (LaSarge et al., 2007), it is not clear that the deficits in odor discrimination learning necessarily reflect OFC dysfunction, given that the deficits involved simultaneously presented odors, which may require hippocampal function (Eichenbaum et al., 1988). Accordingly these learning deficits correlated with spatial learning deficits (LaSarge et al., 2007). This is not the case in Long-Evans rats in our serial odor discrimination task; in this setting aged Long-Evans rats show normal discrimination performance combined with reversal impairments (Schoenbaum et al., 2002a), and the reversal deficits are associated with changes in encoding in OFC and dissociable from hippocampal-dependent deficits in spatial learning (Schoenbaum et al., 2002a, 2006a). Further, similar age Long-Evans rats also show deficits in delayed-non-matching to sample in a task known to be sensitive to OFC damage (Otto and Eichenbaum, 1992; Zyzak et al., 1995). Thus Long-Evans would seem to be an excellent model for identifying general age-related decline in OFC-dependent function.

A fourth possible explanation is that reversal learning and devaluation are mediated by dissociable neural networks or even distinct regions within OFC, and that networks mediating reversal learning are impaired while those mediating devaluation are not. This would support accounts in primates suggesting that these two functions can be dissociated by lesions within different parts of the OFC (Kazama and Bachevalier, 2009). Again we view this as unlikely, at least in the rat, since rat OFC has not shown this type of heterogeneity at the regional level (i.e., the lesions causing devaluation and reversal deficits appear the same) and available evidence suggests that the type of encoding that mediates the two processes – and therefore the specific neural ensembles – are the same (Schoenbaum et al., 2009).

A fifth possible explanation is that the reversal deficit previously reported as reflecting dysfunction in OFC actually does not reflect localized dysfunction in the OFC itself – or at least reflects the loss of function across a much wider circuit than just OFC, a circuit which differs from that mediating devaluation effects. We have recently proposed that the OFC contributes to reversal learning by signaling outcome information acquired during initial learning to downstream areas to mediate normal error-signaling and updating of these associative representations (Schoenbaum et al., 2009). Accordingly reversal learning in normal rats is facilitated by slower recoding of ensembles in OFC (Stalnaker et al., 2006). This suggests that OFC facilitates reversal learning by signaling the old rather than the new information.

Clearly the network involved in this learning process, which likely involves midbrain dopamine error-signaling neurons (Calaminus and Hauber, 2008; Takahashi et al., 2009), must be substantially different from the one by which information about expected outcomes is used prospectively to influence ongoing decisions. This prospective network presumably depends on direct connections with other prefrontal and perhaps subcortical areas. The current results suggest that the functions of this prospective network may be largely intact even while learning related functions are disrupted.

This dissociation might occur if the network mediating learning was especially sensitive to OFC dysfunction and to the robustness of the cue-outcome or other representations encoded there. Alternatively, it is possible that the dissociation in the functions of these two networks reflects the fact that the primary deficit is not in OFC itself but rather lies elsewhere in some part of the network that is uniquely required for learning. An obvious candidate would be the midbrain dopamine system, which has been shown to signal critical error signals required for associative learning (Hollerman and Schultz, 1998; Waelti et al., 2001; D'Ardenne et al., 2008; Matsumoto and Hikosaka, 2009). Prefrontal cortical regions exhibit a significant loss of dopamine input with aging in humans (Ota et al., 2006), monkeys (Goldman-Rakic and Brown, 1981; Wenk et al., 1989), and rats (Allard et al., 2010). Further, decreased dopamine innervation of OFC has been associated with cognitive deficits in aged rats, including deficits in reversal learning (Mizoguchi et al., 2009, 2010). By this proposal, the abnormally sparse and rigid representations in OFC in reversal-impaired aged rats reported previously might not be causal but instead might be a side effect of diminished error-related feedback.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.