The impacts of acquisition and extinction cues on ABC renewal of voluntary behaviors.

In two instrumental conditioning experiments with rats, we examined the impacts of acquisition and extinction cues on ABC renewal of instrumental behavior. Animals were reinforced with food for lever pressing in one context, followed by extinction of the response in a second one. Presentations of a brief tone accompanied extinction in Experiment 1 (extinction cue), and acquisition in Experiment 2 (acquisition cue). A final test in a third context revealed that instrumental responding was decreased in the presence of the extinction cue, whereas it was increased in the presence of the acquisition cue. We discuss theoretical and clinical implications of our results.

Contextual cues play a fundamental role in instrumental conditioning (Bouton and Todd 2014; Bouton 2019). After an instrumental response has been successfully acquired due to repeated pairings of the response with a desirable outcome, context changes lead to disruption of acquisition performance (Bouton et al. 2011, 2014; Todd 2013; Thrailkill and Bouton 2015). Instrumental extinction, in which responding is decreased by the withdrawal of the reinforcing outcome, also depends on context; changes in the context of extinction can result in renewal of extinguished instrumental responding (Todd et al. 2014). For instance, when animals acquire an instrumental response in Context A that is subsequently extinguished in a second Context B, responding increases when tested in the initial acquisition Context A (ABA renewal; Nakajima et al. 2000; Bernal-Gamboa et al. 2017) or in a novel Context C (ABC renewal; Todd 2013; Bernal-Gamboa et al. 2017). Renewal of instrumental responding has also been demonstrated when acquisition and extinction took place in the same context, but testing in a second one (AAB renewal; Bouton et al. 2011; Bernal-Gamboa et al. 2014). The existence of ABC and AAB renewal reveals that changing the extinction context is sufficient for inducing response recovery indicating that extinction learning generalizes less readily across contexts than initial acquisition.

Renewal of instrumental behavior has been found to be modulated by discrete cues. Using ABA procedures, experiments on instrumental conditioning have demonstrated that responding can be attenuated if tested in the presence of cues that had accompanied extinction training (extinction cues; Willcocks and McNally 2014; Nieto et al. 2017). Given the context-specificity of instrumental acquisition learning, one might also expect that instrumental responding can be increased if tested in the presence of cues associated with initial acquisition training (acquisition cues). This assumption has been widely neglected in research; the only exception up to now is a study by Willcocks and McNally (2014), who found no evidence in support of the assumption. The authors used an ABA renewal procedure in which animals were initially trained to nose poke for alcoholic beer in Context A, followed by extinction treatment in Context B. In one of their experiments, an auditory cue appeared at regular intervals during the extinction sessions (extinction cue), while in another experiment, the auditory cue accompanied initial acquisition training (acquisition cue). During a final test in Context A, Willcocks and McNally found a reduction in instrumental behavior when testing occurred in the presence of the extinction cue, whereas responding was not affected by the presence of the acquisition cue.

The results reported by Willcocks and McNally (2014) are of particular interest as they might indicate a fundamental difference in the capacity of discrete cues associated with acquisition or extinction to modulate recovery of instrumental behavior after extinction. Given the theoretical importance of such a difference, the aim of the present study was to examine the generality of the findings by Willcocks and McNally. Therefore, we investigated the impact of discrete cues on ABC renewal of extinguished instrumental behavior using food as reinforcing outcome.

Previous research has revealed that ABC renewal procedures may result in weaker instrumental response recovery than ABA renewal protocols (Bernal-Gamboa et al. 2014, 2017). This can be explained by considering that ABA renewal may result from both omission of the extinction context's inhibitory impact that suppressed responding and excitatory properties of Context A that had been established during initial training. While in an ABC procedure, in which Context C is neutral with respect to the animal's learning history, responding may recover only due to removal of inhibitory control occasioned by the extinction context.

The stronger the response recovery supported by a renewal procedure, the harder it might be to induce further increments in responding due to an acquisition cue. Hence, an ABC renewal design, which supports weaker response recovery, may be better suited to detect a potential impact of an acquisition cue on renewal. Prior to this, we examined whether ABC renewal can be decreased by an extinction cue, which has not been investigated in previous research.

Table 1 illustrates the designs of the experiments. In each experiment, rats initially learned to perform two responses for food in separate contexts (response R1 in Context A and response R2 in Context B). Then, each response underwent extinction in the alternate context (R1 in Context B; R2 in Context A). Finally, responses were tested in their extinction context and in a third context, C (renewal context).

Table 1.

Experimental design

During extinction of R1 and R2 in Experiment 1, a discrete tone stimulus was repeatedly presented at variable intervals independently of animals’ performance (extinction cue). In the subsequent phase, presentations of the discrete tone accompanied testing in the renewal Context C for R1, but not R2. If the extinction cue is effective in reducing instrumental responding, animals should perform fewer R1 than R2 responses in Context C.

In Experiment 2, presentations of a discrete tone stimulus (acquisition cue) accompanied initial acquisition of R1 and R2. During the final phase, presentations of the tone accompanied testing in the renewal Context C for R2, but not R1. If the acquisition cue increases instrumental responding, animals should perform more R2 than R1 responses in Context C.

The 0.05 level of significance was used in all statistical tests. Stated probability levels were based on the Greenhouse and Geisser (1959) adjustment of degrees of freedom where appropriate. We used partial eta squared (ηp2) as the measure of effect size.

Figure 1A shows the mean responses per minute for R1 and R2 during acquisition and extinction sessions in Experiment 1. As can be seen, both responses were similarly acquired and extinguished. A 2 (Response) × 5 (Session) ANOVA on the acquisition data revealed a significant main effect of session, F(4,52) = 52.05, P < 0.001, ηp2 = 0.80. The main effect of response and the interaction did not reach significance, Fs < 1. A corresponding ANOVA on the extinction data found a significant main effect of session, F(3,39) = 72.45, P < 0.001, ηp2 = 0.85. The main effect of response and the interaction were not significant, Fs < 1.34, Ps > 0.26.

Figure 1.

Mean responding of R1 and R2 during acquisition and extinction (A) and the test sessions (B) in Experiment 1. Error bars denote standard errors of the mean.

Figure 1B shows the mean responses per minute for each response in either test context in Experiment 1. A 2 (Response) × 2 (Test Context) ANOVA found a significant main effect of response, F(1,13) = 65.98, P < 0.001, ηp2 = 0.84, and a significant main effect of test context, F(1,13) = 83.21, P < 0.001, ηp2 = 0.87. The Response × Test Context interaction was also significant, F(1,13) = 26.33, P < 0.001, ηp2 = 0.67.

We explored the interaction by simple effect analyses. The simple effect of test context was significant for R2, F(1,13) = 63.74, P < 0.001, ηp2 = 0.83, but not for R1, F(1,13) = 2.33, P = 0.15. Thus, rats performed higher levels of R2 in Context C than in the extinction context (ABC renewal), while the performance of R1 was similar across contexts. Moreover, the simple effect of response was not significant for the extinction context, F < 1, but reached significance for the renewal context, F(1,13) = 52.77, P < 0.001, ηp2 = 0.80. Thus, both responses occurred to a similar level in the extinction context, while in Context C, animals performed higher levels of R2 than R1, indicating that the presence of the extinction cue attenuated lever pressing.

Figure 2A depicts the mean responses per minute for R1 and R2 throughout acquisition and extinction in Experiment 2. As in the previous experiment, both responses were similarly acquired and extinguished. A 2 (Response) × 5 (Session) ANOVA on the acquisition data revealed a significant main effect of session, F(4,52) = 467.48, P < 0.001, ηp2 = 0.97, while the main effect of response was not significant, F < 1. The interaction reached significance, F(4,52) = 3.48, P = 0.03, ηp2 = 0.21. Subsequent simple effect analyses revealed no significant simple effect of response at any level of the factor session, all Fs < 1.53, all Ps > 0.23. A 2 (Response) × 4 (Session) ANOVA on the extinction data yielded a significant main effect of session, F(3,39) = 103.76, P < 0.001, ηp2 = 0.89. The main effect of response and the interaction were not significant, Fs < 1.58, Ps > 0.23.

Figure 2.

Mean responding of R1 and R2 during acquisition and extinction (A) and the test sessions (B) in Experiment 2. Error bars denote standard errors of the mean.

Figure 2B shows the mean responses per minute for each test session in Experiment 2. A 2 (Response) × 2 (Test Context) ANOVA found a significant main effect of response, F(1,13) = 9.27, P < 0.01, ηp2 = 0.41, and a significant main effect of test context, F(1,13) = 198.16, P < 0.001, ηp2 = 0.94. The Response × Test Context interaction was also significant, F(1,13) = 6.87, P = 0.02, ηp2 = 0.35. Simple effect analyses conducted to explore the interaction found that the simple effect of test context was significant for R1, F(1,13) = 55.10, P < 0.001, ηp2 = 0.81, and for R2, F(1,13) = 87.42, P < 0.001, ηp2 = 0.87. Thus, rats performed higher levels of each response in Context C than in the extinction context, which indicates ABC renewal for R1 and R2. Moreover, the simple effect of response was not significant for the extinction context, F < 1, but reached significance for the renewal context, F(1,13) = 8.58, P = 0.01, ηp2 = 0.40. Thus, both responses occurred to a similar level in the extinction context, while in Context C, animals performed higher levels of R2 than R1, indicating that the presence of the acquisition cue increased lever pressing.

Taken together, the results from Experiment 1 are, to the best of our knowledge, the first demonstration that an extinction cue can decrease ABC renewal of instrumental behavior. This finding is consistent with findings by Nieto et al. (2017) and Willcocks and McNally (2014), who have reported that an extinction cue attenuates ABA renewal of extinguished instrumental responding. Importantly, the present results extend their findings to an ABC renewal procedure. Moreover, our results also extend other reports that have demonstrated the effectiveness of extinction cues in different ABA renewal paradigms of Pavlovian conditioning (Brooks and Bouton 1994; Dibbets et al. 2008).

The results from Experiment 2 demonstrate for the first time that an acquisition cue can augment ABC renewal of instrumental behavior. In comparison, Willcocks and McNally (2014) have found no evidence for the effectiveness of an acquisition cue on ABA renewal of extinguished instrumental responding. However, their study and the present one differ in several procedural aspects that may have contributed to the diverging results, such as the renewal procedure (ABA vs. ABC), the nature of the instrumental response (nose poking vs. lever pressing), the type of reinforcer (drug vs. food), and the type of experimental design (between-subjects vs. within-subjects). Future research will be required to uncover procedural features that may influence the effectiveness of acquisition cues.

In the present experiments, we used a within-subjects design in which two behaviors (pressing the left and right levers) were counterbalanced as R1 and R2 responses across animals. This counterbalancing is an important procedural feature as it makes it unlikely that the observed differences between R1 and R2 were due to differences in response properties (for instance, behaviors might differ in their sensitivity to generalize across contexts). An alternative approach, which could also be interesting in future research, would be to test the same behavior in the extinction and renewal contexts each in the presence and absence of an acquisition or extinction cue.

The present findings suggest that acquisition and extinction cues are able in some way to modulate instrumental behavior. Furthermore, instrumental responding was controlled by contextual cues in the present study, which is indicated by the ABC renewal effect for response R1 in Experiment 2. Different possibilities have been proposed on how such cues (discrete and contextual) may operate (Bouton 2019). One possibility is that the cues act through direct excitatory or inhibitory associations with the reinforcer (Rescorla and Wagner 1972). According to a second view, the cues operate by exciting or inhibiting entire response-reinforcer associations (Bouton 2019). A third view assumes that cues modulate behavior by directly exciting or inhibiting a particular response (Rescorla 1993). With the present study, we are unable to differentiate between these theoretical approaches. However, the results from a series of experiments by Todd (2013) were most consistent with the view that instrumental extinction establishes an inhibitory association between the context and the response. For instance, Todd observed ABA renewal even though Contexts A and B had been equated with respect to conditioning and extinction (Experiment 1), which is inconsistent with an explanation in terms of direct context-reinforcer associations. Furthermore, his results revealed that the behavioral impact of an extinction context was not transferred to a response that had been extinguished in another context (Experiment 4), which indicates that the extinction context inhibited a particular response rather than entire response-reinforcer associations. A similar mechanism may underlie the effectiveness of discrete extinction cues, which has been suggested by Nieto et al. (2017) who found that the behavioral impact of an extinction cue was restricted to the response with which it was trained. Whether an acquisition cue acts in a corresponding way—through a direct excitatory cue-response association—is unclear yet and require future research.

The present study has promising and troubling clinical implications. Renewal has been proposed as a factor contributing to relapse after behavior therapy (Bouton et al. 2012; Craske et al. 2014). It implies that unwanted voluntary behaviors such as drug abuse, gambling, and overeating that were eliminated by behavioral treatments involving extinction (e.g., functional communication training) are prone to reappear outside the therapeutic environment (for review, see Podlesnik et al. 2017). The present study suggests that the utilization of extinction cues may be a beneficial therapeutic strategy for improving the long-term success of therapeutic interventions. Our study also directs attention toward discrete cues that accompanied a patient's performance of problematic behaviors. The present results indicate that these kind of cues may constitute a serious source for relapse after therapy. The likelihood of a relapse, which increases outside the therapeutic environment, may be further increased when the posttreatment context comprises cues that had been present during behavioral acquisition.

A group of 15 three-month-old and a group of 15 four-month-old female Wistar rats were used in Experiments 1 and 2, respectively. Rats were experimentally naïve and weighed on average 275.6 g in Experiment 1, and 282 g in Experiment 2. They were individually housed in methacrylate cages in a colony room, had ad libitum access to water, and were food-deprived to 83% of their initial body weight throughout the experiment. One rat was excluded because of sickness in each experiment yielding a final sample size of 14 animals for each experiment.

Twelve identical chambers manufactured by MED Associates (model ENV-008) were used for the experiments. The sidewalls and ceiling were made of clear acrylic plastic, and the front and rear walls of stainless steel. The floor consisted of sixteen 0.5 cm diameter stainless steel rods spaced 1.5 cm apart. A recessed 5 × 5 cm food magazine in which 45 mg Noyes A/I pellets could be delivered was centered on the front wall. Each chamber had two retractable levers, positioned to the right and left of the food tray. A 3000 Hz tone (80 dB) served as a discrete cue, which was delivered through a speaker at the ceiling of the sound attenuation chamber.

Chambers were set up to provide three sets of contexts. Four chambers provided one context in which one sidewall and the ceiling were covered with white and black horizontal lines (Context 1). Four chambers provided a second context where a white vinyl acetate sheet covered the floor, and dark dots the sidewalls (Context 2). Four chambers provided a third context where the sandpaper sheet covered the floor, and wide black and white diagonal lines one sidewall (Context 3). The three types of chambers (Contexts 1–3) were counterbalanced as Contexts A–C according to a Latin square design: for some animals, Contexts 1, 2, and 3 served as Contexts A, B, and C, respectively (five animals); other rats received Contexts 1, 2, and 3 as Contexts C, A, and B, respectively (five animals); for a third group of rats, Contexts 1, 2, and 3 represented Contexts B, C, and A, respectively (four animals). Table 2 illustrates the balancing schedule for Experiments 1 and 2.

Table 2.

Balancing schedule for experiments 1 and 2

The experimental protocol was approved by the Ethical Committee of the Faculty of Psychology of the National University of Mexico. Sessions were conducted on successive days at the same time each day. Prior to acquisition, rats were exposed to the contexts in the order A, B, and C in successive sessions on Day 1. Sessions were separated by approximately 1 h. During those sessions, approximately 30 food pellets were delivered at a variable-time 30 sec schedule. No levers were presented. Each session lasted 15 min.

During the next 5 d, rats received two daily sessions in which they were trained to press the right and left levers for food on a variable-interval 30 sec schedule. Only one lever was available in a session. For half of the animals, pressing the right and left levers served as R1 and R2 responses, respectively, whereas for the other half, the right and left levers represented R2 and R1, respectively. R1 was trained in Context A, and R2 in Context B. Each session lasted 30 min. Sessions on the same day were separated by approximately 3 h. For half of the animals, the first session per day was conducted in Context A, and the second session in Context B; the other half received reverse order (first session in Context B and second session in Context A). In each acquisition session of Experiment 2, rats received approximately 65 presentations of a 5-sec tone delivered at a variable-time 30 sec schedule (acquisition cue); the tone was presented for the acquisition of both responses.

Subsequently, rats received four extinction sessions for each response. No food pellets were delivered. R1 underwent extinction in Context B, and R2 in Context A. Each session lasted 30 min. Sessions on the same day were separated by 3 h. For half of the animals, the first session per day was conducted in Context B, and the second session in Context A; the other half received reverse order (first session in Context A and second session in Context B). In each extinction session of Experiment 1, rats received approximately 65 presentations of a 5-sec tone delivered at a variable-time 30 sec schedule (extinction cue); the tone was presented for extinction of both responses.

Finally, each response was tested in its extinction context (R1 in Context B and R2 in Context A) and in Context C (renewal context). Testing occurred in successive sessions, each lasting 10 min. Sessions were separated by 60 min. There was only one lever available during each test session and no reinforcers were delivered. For half of the animals, the order of test sessions was R1 in Context B, R2 in Context A, R1 in Context C, and R2 in Context C. For the other half, the test order was R1 in Context C, R2 in Context C, R1 in Context B, and R2 in Context A. In Experiment 1, R1 was tested in the presence of the extinction cue in both contexts, while R2 was tested with the extinction cue only in the extinction context. In Experiment 2, testing R2 in Context C occurred in the presence of the acquisition cue, while R1 was tested without the acquisition cue.