The effects of extinction-aroused attention on context conditioning.

Two experiments assessed the effects of extinguishing a conditioned cue on subsequent context conditioning. Each experiment used a different video-game method where sensors predicted attacking spaceships and participants responded to the sensor in a way that prepared them for the upcoming attack. In Experiment 1 extinction of a cue which signaled a spaceship-attack outcome facilitated subsequent learning when the attack occurred unsignaled. In Experiment 2 extinction of a cue facilitated subsequent learning, regardless of whether the spaceship outcome was the same or different as used in the earlier training. In neither experiment did the extinction context become inhibitory. Results are discussed in terms of current associative theories of attention and conditioning.

The present experiments were designed to determine the effect of extinction of a cue on subsequent contextual conditioning. During initial training a cue is paired with an outcome (e.g., conditioned stimulus–unconditioned stimulus) against a background of tonic stimuli typically conceptualized as the context. With respect to that context, the organism experiences pairings of it with the outcome in compound with the cue, as well as extended periods of time during the inter-trial interval (ITI) with neither the cue nor the outcome. Typically, with appropriate parameters, organisms come to respond when the cue is present, and not during the ITI when only the contextual stimuli are present.

During extinction trials, the cue is presented without the outcome and the acquired response to the cue declines and can ultimately disappear. Given that the organism no longer responds to anything, there is a certain intuitive appeal to the idea that attention should decline during extinction, as no important events are occurring. Nevertheless, formal associative models of attention provide reasons to expect extinction to increase or maintain attention, particularly to contexts.

Consider theories that incorporate the rule of attention developed by Mackintosh (1975), such as the theory articulated by Le Pelley (2004) and subsequent similar variations (Le Pelley 2004; Haselgrove et al. 2010; Pearce and Mackintosh 2010; George and Pearce 2012; for review, see Le Pelley et al. 2016). In these theories, attention to a stimulus changes as a function of prediction error—the difference between what is predicted by the stimulus and what occurs. Attention is reflected in a learning-rate parameter “α” or what Le Pelley also refers to as “attentional associability.” That attentional associability is determined by how much prediction error a cue produces in comparison to other stimuli. Stimuli producing less prediction error gain more attention.

Applying that idea to a conditioning-and-extinction sequence is relatively straightforward. During conditioning trials with the outcome, the cue and the context are present and their associations with the outcome should grow. During the ITI when the context is present and the outcome is absent, the outcome is overexpected based on the excitatory associative strength accrued by the context on the previous trial. In the case of this over expectation, it is assumed that the context acquires an anti-association with the outcome (i.e., an inhibitory mechanism) to counter any excitatory potential it may have. The net effect of acquiring this inhibition is that the context will continue to produce more prediction error than the cue on trials with an outcome. Thus, attention to the cue will grow on these trials, and attention to the context will decrease. During extinction, the outcome is absent and overpredicted by the cue. The context had acquired less excitation than the cue, and possibly some inhibition, thus prediction error produced by the context during extinction trials would be less than the cue. In extinction the cue produces substantial prediction error and attention to it will decrease while the context, whose prediction is more in line with the current contingencies relative to the cue, will gain in attention.

The idea that extinction could increase attention to contextual stimuli can also be inferred from Kruschke's theorizing (e.g., Kruschke and Johansen 1999; Kruschke 2001; for review, see Kruschke 2011,). Prediction error again drives attention in these models, and much like what occurs in the Mackintosh-based models, attention is shifted away from stimuli that produce error to stimuli that reduce it. Therefore, during extinction attention should shift toward the context which has an overall weaker net association with the overexpected outcome than the cue.

The theory of Le Pelley (2004) is known as a “hybrid” theory in that it also incorporates an additional attention construct he refers to as “salience associability.” Changes in this type of attention generally follow the rules laid out by Pearce and Hall (1980). Rather than increasing, this type of attention decreases as outcomes become well predicted. As prediction error increases, attention is said to increase. The relevant prediction error, however, is based on an aggregate prediction by all stimuli present. Each stimulus’ individual contribution to that error is not considered. This intuitive idea, expressed in its current form by Hall and Rodriguez (2010), also predicts that attention to contexts should increase during extinction. As in the earlier models, conditioning yields excitatory associations between the cue and outcome, as well as between the context and outcome. During the ITI, inhibition forms between the context and the outcome. As discussed before, the cue becomes a good predictor of the outcome, while the context acquires enough inhibition to counter its acquired excitation, and correctly predicts the absence of the outcome. At the end of this training the states of the outcome are well predicted and attention to all stimuli should be low. During extinction, however, the outcome is overpredicted, and attention to all stimuli should increase. At the end of extinction, if the absence of the outcome is correctly predicted, attention to all stimuli should again be low. Nevertheless, there should be attention to the contextual stimuli aroused at some point during extinction. A similar type of mechanism is also present in the model of Schmajuk et al. (1996), though the full details of the implementation are considerably different. We will have more to say about this model in the discussion.

The attentional theory of context processing (ATCP) introduced in Rosas et al. (2006) and Rosas and Callejas (2006) also predicts that attention to contexts should be aroused by extinction. In the prior theories, the arousal of attention is a function of either absolute, or relative, prediction error produced by the context. ATCP, however, does not describe how attention changes to discrete cues in any formal way. ATCP describes a set of conditions in which attention to contexts should be aroused; one of which is an adoption of Bouton's (1997) assumption that the presentation of ambiguous stimuli in a context arouses attention to the context to resolve the ambiguity. During extinction, where the cue is assumed to acquire an inhibitory association (e.g., Wagner 1981), counteracting its initial excitatory one, it becomes functionally ambiguous. Unspecified mechanisms detect this ambiguity and arouse attention to the context to resolve it. That is, extinction performance requires attending to the extinction context for the expression of the cue's inhibition acquired in extinction. Rosas et al. extend this analysis to suggest that once such attention is aroused, it is maintained during periods of time when the ambiguous cue is not presented and can subsequently affect the contextual control of another cue (e.g., Rosas and Callejas-Aguilera 2006, 2007; but see Nelson et al. 2011a; Nelson and Lamoureux 2015). Regarding the effects of attention on context conditioning the prediction is the same as that made by the other theories discussed above. Extinction should arouse attention to contextual stimuli and facilitate subsequent context conditioning.

Some evidence for extinction increasing attention to contexts during extinction exists. Using a video game task, Nelson et al. (2013) conditioned a visual “sensor” cue to evoke an expectation of an attack. Then, half the participants underwent extinction of that cue while the other half was exposed to the context alone. In the following test phase each group was divided into three subgroups and received a bi-conditional discrimination that required participants to attend to the joint configuration of stimuli (Red, Green, Blue, and Yellow sensors) in order to correctly predict the attack (i.e., RG+, BG−, RY−, BY+). In a third of the sub groups the context (A: or B:) was relevant to solving the discrimination. The authors arranged for all trials involving G to be in one context (A:RG+, BG−) and all the trials involving Y in another (B:RY−, BY+), reasoning that any manipulation that enhances attention to contexts should facilitate the discrimination when the contexts are relevant. Two control groups received trials where the context was unrelated to the types of trials occurring within them. The results clearly showed that the discrimination was easier when the context was relevant to solving it, and prior extinction facilitated learning the discrimination, but only when the context was relevant.

Using discrete stimuli in an arrangement where one element of a compound has properties similar to contexts, Vadillo et al. (2016), showed that a discrimination reversal can shift attention to those context-like elements. Vadillo et al. trained participants in a category-learning task where cues X and A combined (XA) indicated membership in category 1 and cues X and B combined (XB) indicated membership in category 2. Much like a tonic background contextual stimulus, X occurred both with and without A and B, and was not useful in predicting the category. After the first phase where the category learning was established, the cue-category relationships were reversed. On this task the authors inserted probe trials where a dot briefly appeared on one of the stimuli and participants were required to report where the dot appeared. The rationale underlying those probes was that participants would be more accurate and faster at reporting the dot when it appeared on a stimulus to which they were attending. The results showed that participants attended to the predictive cue more than the redundant one, but that difference was reduced when the cue-category relationships were reversed. That is, an interference-producing manipulation led to a shift in attention away from the predictive cues to the redundant X.

Lucke et al. (2013) used a predictive learning task and trained participants in an explicit discrimination where cue X was predictive of an outcome (e.g., stomach trouble) in Context A (e.g., a particular restaurant), but not in Context B. Eye tracking showed that participants directed their overt visual attention to the contextual stimuli. However, the training used there was an explicit conditional discrimination where the types of trials alternated in blocks between trials with X+ Outcome in Context A and X− in Context B. Whether or not such an explicit discrimination engages the same mechanisms as a conditioning-then-extinction sequence is not certain, but it suggests that overt attention to contexts is aroused when they are informative regarding the events occurring within them.

In sum, evidence is consistent with the idea that attention to contexts can be enhanced by extinction, but only one report (Nelson et al. 2013) used an actual extinction manipulation. The present experiments sought to corroborate that finding by determining whether extinction-produced attention could enhance context's ability to be conditioned. Such a demonstration would also reflect a clear change in attentional associability.

The possibility that extinction could enhance context conditioning is also important in its practical considerations. If extinction enhances context conditioning, caution should be used in treatments involving extinction. Consider, for example, the treatment of anxiety. Extinguishing discrete anxiety-related cues may leave the context especially susceptible to conditioning when an anxiety-provoking outcome is inadvertently encountered. Context conditioning has been discussed as a model of persistent anxiety (e.g., Grillon 2002). Thus, encountering unintentional aversive outcomes during an extinction-based therapy could serve to shift the elicitation of anxiety from discrete cues to contextual ones, which could lead to a more generalized anxiety state.

In two experiments participants received conditioning with a cue in one context. Then, to ensure that the contexts were unaffected by any accrued associative strength from conditioning, the participants received extinction and testing in a different context which was neutral at the start of conditioning. Though the context was different at the start of extinction, it had no association with the outcome and it produced no prediction error. In extinction attention should have still shifted toward the context and away from the cue as described earlier.

The first experiment used the same method as Nelson et al. (2013) where participants earned points by clicking a computer-mouse button to fire torpedoes at an on-screen spaceship which would sometimes attack the participant. Those attacks were signaled by visual “sensors.” Participants learned to conserve power by suppressing their firing in the presence of the sensor to cope with the attack. Contexts were provided by different space backgrounds where the gameplay took place. The design is shown in Table 1.

Table 1.

Design of Experiment 1

Participants first received eight trials where a red sensor was illuminated for 5-sec and its termination was paired with an attack from the spaceship on each trial. These trials were followed by eight trials in Phase 2 where the context was changed to the Phase-2 context and the cue appeared without the outcome in the After-Extinction group while the No-Extinction group simply received context exposure. In the final Testing phase all participants received eight unsignaled outcomes, one every 20 sec. If extinction arouses attention to contexts, then conditioning should proceed more rapidly in the After-Extinction condition than in the No-Extinction condition. There is also the possibility that the context of extinction could become inhibitory, leading to less rapid conditioning following extinction. We consider this possibility in more detail shortly.

A second experiment was conducted using a different group of participants and a different space-based video game developed by Nelson et al. (2014). Rather than a task involving suppression of responding, conditioning in this task is evidenced by an increase in behavior. Participants were initially trained to make rapid keyboard responses to charge particular weapons to repel the appearance of specific corresponding spaceships. After these different responses were associated with their respective spaceships, the participants were transported to a new galaxy (context) for patrol. Here, colored sensors appeared that predicted particular corresponding spaceships and participants came to emit the response associated with the specific spaceship prior to its arrival in the presence of the sensor. Sensors were present for 20 sec, and the spaceship appeared after 5 sec and remained for 15 sec, exiting with the termination of the sensor cue. Contexts were again provided by background space-galaxies where the gameplay takes place, but unlike the previous method these contexts were completely three-dimensional and rendered, along with other game elements, in a quality comparable to modern video games. Contexts were further differentiated by different custom music soundtracks playing in the background.

The design of the second experiment is shown in Table 2. All participants received conditioning with a red sensor predicting one specific spaceship (+). The second phase, followed in a different context with groups either receiving extinction of the red sensor, or exposure to the context. Throughout these two phases, trials were included where a blue sensor predicted another spaceship (*), simply to break the monotony of the training, particularly in the extinction phase for the No Extinction condition. Next all participants received 12 trials of context conditioning where a spaceship appeared every 26 sec in the absence of the sensor. The spaceship used was either the same (+) or different spaceship (!), as used in conditioning (spaceship identity counterbalanced). The 13th trial was a probe trial where the outcome did not appear and responding was measured for 65 sec. The design was thus a 2×2 factorial design where context conditioning was conducted either after extinction or context exposure, either with the same outcome as was used in conditioning, or a different one.

Table 2.

Design of Experiment 2

In the theories derived from Mackintosh (1975) and/or Pearce and Hall (1980), cited earlier, it is assumed that inhibition is acquired when an outcome is overpredicted. During extinction trials where the cue is presented without the outcome, the overprediction can be substantial enough that the inhibition acquired on those trials makes the net value of the context inhibitory. That is, the context can be predicted to acquire the properties of a conditioned inhibitor, explicitly signaling the absence of the outcome. In some situations, a context can indeed become inhibitory for an outcome during extinction (Cunningham, 1979; Polack et al., 2012; Glautier et al. 2013) but that is not a universal finding (Bouton and King 1983; Bouton and Swartzentruber 1986, 1989; Grahame et al. 1990; Nelson et al. 2011b).

If inhibition is conditioned to the context, then conditioning of that context should proceed slowly as such conditioning would represent the well-known retardation test for conditioned inhibition (Rescorla, 1969). Such a retardation could also occur in Experiment 1. After extinction, conditioning of, or within, the context would be expected to proceed slowly if the context was an explicit signal for the absence of the outcome. It is possible that attention could still be aroused to the context that might only serve to increase its expression of inhibition (e.g., Larrauri and Schmajuck 2008). To briefly foreshadow, no evidence of context inhibition was found in the first experiment. Yet, the necessary conditions to consistently produce one or the other result are not currently known. With the possibility that the method of Experiment 2 might produce inhibition, we examined the prospect in more detail there.

To separate any possible context inhibition from attention we used two different spaceships in the context-conditioning test of Experiment 2, the one previously used in cue conditioning for which the context might be inhibitory, or a different one with no relationship to the context. If attention to the contexts' changes, it should affect conditioning to the context regardless of the spaceship used. If the context becomes inhibitory, context conditioning should be slow only for the spaceship used in extinction. The different spaceships have considerable visual differences and are associated with different responses; thus, they should be easily recognized as different and inhibition has been shown to be outcome specific (Rescorla and Holland 1977). If both contextual inhibition and an increase in attention are obtained we should observe retarded conditioning for the same spaceship as was used during the initial conditioning and extinction phases and rapid excitatory context conditioning with the different spaceship.

For discussion, we conducted simulations of Experiment 2 using the models of LePelley (2004) and Hall and Rodriguez (2010). The details of the simulations are presented in the Materials and Methods. The final parameters used are reported in the results and the outcomes predicted by the simulations are presented and discussed in the discussion.

Results

Experiment 1

Seven participants were removed from the data set for failing to learn to suppress their responding to the sensor in the first phase (5 from group After Extinction and 2 from Group No Extinction).

Phase 1 and 2

Suppression of responding to the cue during phases 1 and 2, measured by standard suppression ratios (cue responding/(precue responding + cue responding)), is shown in Figure 1. Both groups equally acquired suppression, and that suppression was reduced in the group receiving extinction. An Extinction (to receive extinction or not) × Trials ANOVA of the eight conditioning trials revealed an effect of Trials F(7,217) = 32.22, P < 0.0001, η2 = 0.50, and no other effects, Fs < 1. An analysis of the eight extinction trials within group After Extinction showed an effect of Trials F(7,98) = 2.82, P = 0.01, η2 = 0.17.

Figure 1.

Data from phases 1 and 2 of Experiment 1. Open symbols represent the group to be tested after extinction, solid symbols represent the group to be tested without prior extinction. Bars represent the standard error of the mean.

Testing

During the context-conditioning phase the outcome occurred every 20-sec and we monitored responding on each second between outcome presentations. The “cue” was any time the context was present, which was continual, so there was no “precue” period with which to compute a suppression ratio. Therefore, we first analyzed the number of clicks emitted during bins of 5 sec between outcomes, shown in Figure 2, with an Extinction (After Extinction or No Extinction) × 5-sec Bin × Trials ANOVA. All effects were superseded by the presence of an Extinction × 5-sec Bin × Trials interaction, F(21,651) = 2.05, P = 0.0039, η2 = 0.06. Despite the three-way interaction, group variability was high and there were no reliable differences between the groups on any bin on any trial, Fs(1,74) ≤ 3.83, Ps ≥ 0.054.

Figure 2.

The number of mouse-clicks per 5-sec bin from testing in Experiment 1. Open symbols represent the group tested after extinction, solid symbols represent the group tested without prior extinction. Bars represent the standard error of the mean.

Notwithstanding the lack of between-subject differences on each bin, the effect of 5-sec bin is a within-subjects effect, which experiments typically have more power to detect. The interaction also tests whether the within-subjects change is different between subjects, a pattern that is not always reflected by the simple-effect between-subject comparisons. The figure suggests that the between-trial increase in the within-trial decrease was greater in the After- Extinction group than in the No-Extinction group. To make the between-subject differences in that pattern more apparent, we used a single number to represent the within-trial decrease on each trial. That number was a suppression ratio constructed out of the first and fourth bins (bin 4/(bin1 + bin4)). Small numbers reflect the within-subject decline across the trial, while numbers near 0.5 indicate no change in responding. Changes between trials represent what Figure 2 suggests, and the interaction supports, that the within-trial suppression increased across trials, and that the increase was greater in group After Extinction than in group No Extinction. These transformed data are shown in Figure 3. An Extinction × Trials ANOVA of these data showed effects of Trials F(7,217) = 4.16, P = 0.0003, η2 = 0.12, an effect of Extinction, F(1,31) = 5.10, P = 0.03, η2 = 0.14, and an Extinction × Trials interaction, F(7,217) = 2.47, P = 0.019, η2 = 0.07. Simple effects of Extinction on each trial revealed several low-probability differences on trials 3–6, where the increase in suppression at the end of the trial relative to the beginning observed group After Extinction was greater than in group No Extinction Fs(1,174)range 4.81–7.92, P = 0.005–0.03, drange 0.64–1.16, though only trial 5 (P = 0.005) would be considered reliable with a Bonferroni correction for the eight comparisons.

Figure 3.

Suppression across bins (Bin4/((bin1+bin4)) across trials in Experiment 1. Open symbols represent the group tested after extinction, Solid symbols represent the group tested without prior extinction. Bars represent the standard error of the mean.

Experiment 2

Data during filler trials with B* were not collected as they were irrelevant to the questions at hand and their occurrence was not manipulated between groups. Responding in the absence of cues in the first two phases was practically absent, averaging 0.16 responses per second across phases 1 and 2, and will not be reported further. Five participants were removed due to failing to learn to respond to the cue in the conditioning phase (two from No Extinction—Same outcome, and one from each of the other three groups).

Phase 1

The data from conditioning are shown in Figure 4 which shows the responses per second during the cue on each second of each trial prior to the outcome. Responding increased both within and between trials. An Extinction (to receive extinction or not) × Outcome-Type (to be tested with the same or different outcome as used in conditioning) × Trials × Seconds ANOVA confirmed that description revealing effects of Trials, F(9,828) = 129.21, P < 0.0001, η2 = 0.58, Seconds, F(4,368) = 585.01, P < 0.0001, η2 = 0.86, and a Trials × Seconds interaction, F(36,3312) = 43.05, P < 0.0001, η2 = 0.31. There were no effects of, or involving, the grouping variables, Ps ≥ 0.19.

Figure 4.

Data from Phase 1 of Experiment 2. Y-axis shows responses per second. X-axis represents each second on each trial of conditioning with Red. Open symbols indicate groups that received extinction in Phase 2. Closed symbols indicate groups that received no extinction. Circles represent groups tested with the same spaceship as Phase 1 and triangles received testing with the different spaceship. Bars represent the standard error of the mean.

Phase 2

Data from the After-Extinction groups during extinction are shown in Figure 5. As there was no outcome, the figure shows responses per second during the entire 20-sec of the cue presentation on each trial. Responding increased up until the time the spaceship outcome normally occurred (at the end of second 5) and then began to decrease, with overall responding decreasing between trials. An Outcome-Type × Trials × Seconds ANOVA reveals what would be expected by that pattern, showing effects of Trials, F(7,322) = 49.90, P < 0.0001, η2 = 0.52, reflecting the overall decrease, Seconds, F(19,874) = 28.79, η2 = 0.38, reflecting the changes within a trial, and Trials × Seconds, F(133,6118) = 5.72, P < 0.0001, η2 = 0.11, reflecting the change in the pattern of seconds over trials. There were no effects involving the grouping variable, Ps ≥ 0.08, though sampling differences resulted in a slight trend for those to be tested with the same outcome to respond insignificantly more on some trials (e.g., trials 5 and 6) than those to be tested with a different outcome, a trend we mention here as it persisted throughout the experiment.

Figure 5.

Data from Phase 2 extinction of Experiment 2. Y-axis shows responses per second. X-axis represents each second on each trial of Extinction with Red. Open symbols indicate groups that received extinction in Phase 2. Closed symbols indicate groups that received no extinction. Circles represent groups tested with the same spaceship as Phase 1 and triangles received testing with the different spaceship. Bars represent the standard error of the mean.

The computer began recording response data 1 sec after the outcome on each trial, leading to 25 sec of recording during the 26-sec interval between outcomes. Responding prior to the outcome during testing is shown in Figures 6A and B (note that the Y-axis scale has been reduced). Figure 6A shows responding in all groups, Figure 6B collapses across the Outcome Type variable. Both figures show responding in 5-sec blocks to reduce clutter. An analysis of the 5-sec blocks, or of all 25 sec, across trials supports the same conclusions. In short, conditioning was obtained as responses increased within and between trials. Responding in the After-Extinction conditions (open symbols) increased more quickly than in the No Extinction conditions (closed symbols), independently of the outcome type. In the After-Extinction conditions there was an unreliable trend for responding to the Same outcome (circles) to be greater than to the different outcome (triangles), as was observed between those groups in the previous phase.

Figure 6.

Data from the context-conditioning test of Experiment 2. Y-axis shows responses per second. X-axis represents each 5-sec block on each trial of conditioning with Red. Open symbols indicate groups that received extinction in Phase 2. Closed symbols indicate groups that received no extinction. Circles represent groups tested with the same spaceship as Phase 1 and triangles receive testing with the different spaceship. Bars represent the standard error of the mean. A shows all four groups, B collapses across the outcome types.

These impressions were confirmed by an Extinction × Outcome type × Trials x Blocks ANOVA of the 12 test trials. The analysis revealed a main effect of Extinction, F(1,92) = 6.49, P = 0.01, η2 = 0.07, and effects of Trials, F(11,1012) = 20.06, P < 0.0001, η2 = 0.18, Blocks, F(4,368) = 40.41, P < 0.0001, η2 = 0.31, Trials × Blocks, F(44,4048) = 6.997, P < 0.0001, η2 = 0.07, and Extinction × Blocks, F(4,368) = 4.10, P = 0.003, η2 = 0.04. No other effects were reliable, Ps ≥ 0.199. Figure 6B shows the data collapsed across the outcome type variable to show the effect of extinction more clearly.

Simple effect tests of Extinction on each block revealed 20 places where potential differences exist, Fs(1,674)range = 4.72–10.2, P = 0.001–0.03, drange = 0.42–0.65, appearing on trials 3 (blocks 3–5), 4 (blocks 2–5), 5 (blocks 4–5), 6 (blocks 3–5), and on trials 7,9,11, and 12 in blocks 4–5. With a 0.05 criterion and 60 comparisons we would expect 3 chance rejections. Some of the remaining 17 comparisons may be errors, but the probability that all the remaining comparisons are errors is infinitesimally small. Thus, there are clearly differences enough to support that there was greater responding in the After Extinction condition.

Data from the probe test are presented in Figure 7A and B. Panel A shows data from all four groups on each second of the probe test while Panel B shows the data collapsed across the Outcome-type variable. The average standard error was 0.46 and 0.33, for panels A and B, respectively. Error bars have been omitted so as to not clutter the figure. The effect of Extinction was less apparent by the time the probe test was conducted. Nevertheless, an analysis (Extinction × Outcome Type × Seconds) of the 25-sec of the probe test prior to the time the spaceship had been expected, revealed effects of Seconds, F(24,2208) = 15.73, P < 0.0001, η2 = 0.15, and a Seconds × Extinction interaction, F(24,2208) = 1.59, P = 0.03, η2 = 0.02. There were no other significant effects, Ps ≥ 0.27. The collapsed data in panel B suggests that the within-subject rate of increase across the trial was greater in the After-Extinction group, but simple effect tests found no significant effect of Extinction on any second, Ps ≥ 0.099. The same analysis of the last 40 sec of the probe test, after the expected spaceship did not appear, revealed only an effect of seconds, F(39,3588) = 6.82, P < 0.0001, η2 = 0.07, as responding decreased. There were no other effects in this final analysis, Ps ≥ 0.19.

Figure 7.

Data from the context-conditioning test of Experiment 2. Y-axis shows responses per second. X-axis represents each second on each trial of testing. The break in the symbols above the 25 sec of measurement represents where the outcome would ordinarily be expected. At left, open symbols were participants tested after receiving extinction. Solid symbols represent participants that received no extinction. Circles represent groups tested with the same spaceship as used in Phase1 and triangles represent groups tested with a different spaceship. At bottom, open symbols show groups tested after extinction averaged across outcome type, and closed symbols show groups tested without extinction, averaged across outcome type. Error bars are removed to prevent clutter. The average error in the phase was 0.46 and 0.33 for the left and right panels, respectively.

Simulations

The final parameters (see the Materials and Methods for their derivation) used in the model of Le Pelley (2004) were as follows. All pairs of values are for the context and cue, respectively. The values were 0.073 and 0.9 for α and 0.550 and 0.9 for ς. γ was set at 0.1. θ was set to 0.8 and 0.1 for reinforced and nonreinforced trials, respectively. β was set to 0.6 and 0.22 for reinforced and nonreinforced trials, respectively. The sum of the associations for all stimuli present on each trial are shown in Figure 8 and changes in the α attention parameter are shown in Figure 9.

Figure 8.

Responses predicted by the model of Le Pelley (2004) for groups After- and No Extinction with the same outcome in Experiment 2.

Figure 9.

Changes in attention predicted by the model of Le Pelley (2004) for groups After- and No Extinction with the same outcome in Experiment 2.

Discussion

In both experiments extinction of a cue led to more rapid conditioning in the contexts after extinction had occurred than when extinction had not. Responding in the context appeared to show temporal specificity. In the first experiment suppression increased within each trial on test. Such an increase could be due to either learning when the outcome occurs, or learning a period of safety immediately after an outcome presentation, or both (e.g., Rosas and Alonso 1996, 1997). In the probe trial of Experiment 2 we have clearer evidence that participants did time the occurrence of the outcome. Responding increased up until the time the outcome was expected, and then began to decrease. Thus, at least one component of the context which was learned about was temporal, and that component seemed to be especially salient after extinction.

The more rapid conditioning following extinction is consistent with an increase in attention, and corroborates other work showing that attention to contexts increases when cues acquire conflicting information about an outcome (e.g., Lucke et al. 2013; Nelson et al., 2013; Vadillo et al. 2016). An increase in attention to contexts during extinction is predicted by the models of Mackintosh (1975), Pearce and Hall (1980), Hall and Rodriguez (2010), and models that incorporate rules similar to one (e.g., Schmajuk et al. 1996; Kruschke 2011), both (e.g., Le Pelley 2004), or neither (Rosas et al. 2006) of these theories. Despite that all of these theories predict an increase in attention to the context, not all necessarily predict the exact behavioral results obtained.

Figure 8 shows a simulation of the No- and After-Extinction groups tested with the same outcome in Experiment 2 using the model of Le Pelley (2004). The figure shows the sum of the associative strengths of all the stimuli present on a trial, thus reflecting the predicted average behavior. During conditioning responding to the cue and context rises, while responding to the context alone remains low. During extinction responding to the cue decreases and the context is predicted to become slightly inhibitory. In the group that receives exposure to the context in the absence of extinction, the associative strength remains at its starting level of zero. When unsignaled outcomes occur, the model correctly predicts that the group which had received extinction will acquire the response more rapidly, despite the context being slightly inhibitory at the beginning of training.

The model successfully predicts these results because it predicts an increase in attention to the context, represented by the learning-rate, during extinction. That increase more than compensates for the different starting associative strengths predicted between the two groups. Figure 9 shows the predicted α between the conditions. During conditioning the cue is a better predictor of the outcome than the context and rapidly gains attention, while the context loses attention. During extinction, when the outcome is absent, the context is a better predictor of that absence than the cue, so attention to the context rapidly grows while that to the cue declines. These values will ultimately stop changing when the associative strengths are adjusted to match the actual outcome, thus attention accrued to the context will be maintained even once the absence of the outcome is correctly predicted. In the group that does not receive extinction, very little outcome is predicted by the context, no outcome occurs, and thus no changes in attention are effected, leaving α at its low staring level. During context conditioning, attention to the contexts increases in both conditions, but the Extinction conditions begin the phase with a considerable attentional advantage.

In our simulations using the rules developed by Pearce and Hall (1980) and implemented in their present form by Hall and Rodriguez (2010) we were unable to find parameters such that the data correctly matched that from our conditioning and extinction phases. Nevertheless, the model did predict conditioning and extinction, simply not in the relative proportional levels we obtained to the cues and contexts in these phases. The model also predicted that the extinction context would become inhibitory and that attention to the context would be higher with extinction than with simple exposure. The error in prediction produced by the cue and the context during extinction is higher than that produced by the context alone in the no extinction condition, thus attention will remain higher in the extinction condition than in the control where the context is simply exposed.

In both experiments, particularly the second, there was evidence of temporal conditioning. Participants were able to use temporal cues to predict when the outcome would occur, and they were better able to do this after extinction than without. This particular result is somewhat at odds with theories of attention that have been developed to deal with exteroceptive cues. The data from extinction suggest that temporal cues were present during the first two phases, as responding began to decline after the time the outcome was expected, even though the cue was still present. To the extent that attention to temporal cues obeys the same rules, then attention to these cues should have declined as shown in the simulations with the discrete cues.

Other possibilities emerge when considering that timing cues need an initiating event. To the extent that timing initiated by the cue is a different signal than timing initiated by other events, then extinction-induced changes in attention to one temporal cue may not affect other, different, temporal cues. On test, the event which undoubtedly starts the timing is the outcome, either its onset or offset. In the first phase, like the exteroceptive contextual cues, timing initiated by the outcome was irrelevant to predicting the next outcome as the inter-trial intervals were variable. Of interest, the outcome was absent during extinction, and thus any temporal cues that might be initiated by its occurrence were likewise absent when changes in attention should be occurring. In that regard, the increase in attention to these temporal cues is consistent with the ATCP[3] model of Rosas et al. (2006), to the extent that these former background temporal cues can be viewed as contextual. Rosas et al., have shown that extinction can lead to an increase in attention, not only to contextual cues present in extinction, but also to future contextual cues not present during extinction. Thus, the extinction manipulation may have produced a general arousal that facilitated the use of the outcome and subsequent 26-sec interval as a signal for the next outcome, even though that exact set of cues was not present in training.

In neither experiment did evidence of inhibition appear. The failure to find inhibition is not a null result that is inconclusive due to a lack of power. Rather, the opposite result was obtained in both experiments. Conditioning in the potentially inhibitory context was more rapid. Experiment 2 allowed for the greatest possibility of observing both an increase in attention and the presence of contextual inhibition predicted by these theories. During the context conditioning test either the same or different outcome was used. When the same outcome was used, we could observe the effects of the context becoming inhibitory. There, we would expect conditioning to be slower in the extinction condition than in the control. When a different outcome was used, one for which the context should not be inhibitory, we would simply observe the effect of enhanced attention as manifest by more rapid conditioning following extinction. In our data we observed more effective conditioning following extinction regardless of the outcome used.

The lack of detectable inhibition is a result that is inconsistent with the model of Schmajuk et al. (1996). Larrauri and Schmajuk (2008) have shown that the model predicts that extinction should make a context inhibitory. In their model, changes in attention follow a Pearce and Hall (1980) type of rule where attention should initially increase with the prediction error at the start of extinction, and then decrease as extinction proceeds (but still remain above the control condition that is simply exposed to the context). Importantly, attention is more than a learning-rate parameter in their model, it also affects performance (see also Craddock and Miller 2014). Thus, even though the context is predicted to be inhibitory at the end of extinction, that inhibition might not be immediately evident because attention to the context at the end of extinction could also be low, decreasing performance associated with that inhibition. Nevertheless, the unsignaled presence of the outcome during the test phase would serve to produce prediction error and be detected as “novelty” in the terms of the model, enhancing attention, and thus increasing the expression of inhibition conditioned to the context. In short, the model of Schmajuk et al. (1996) predicts that context conditioning will be slower following extinction of a cue due to the presence of attention-enhanced inhibition, a prediction which is not supported by the present findings.

As mentioned in the introduction, extinction sometimes produces demonstrable contextual inhibition (Cunningham 1979; Polack et al. 2012; Glautier et al. 2013), and other times not (Bouton and King 1983; Bouton and Swartzentruber 1986, 1989; Grahame et al. 1990; Nelson et al. 2011) and the parameters required to reliably produce one or the other result are unknown. In the present experiment, the level of inhibition predicted to have accrued to the context during extinction was very small, and predicted by the model of LePelley (2004) to only be present on the first trial (see Fig. 8). Tests of inhibition require that there be an expectation of the outcome against which the inhibition acts. Very early in testing, when no expectation of the outcome has been established, there is no expectation to be reduced by the inhibition. Nevertheless, that inhibition must be overcome during conditioning for the expectation of the outcome to be expressed during the retardation testing. Given that attention was aroused after extinction, any inhibition that may have been present might simply have been too little to offset the effect of the increase in attention produced by the extinction manipulation, allowing that group to still condition faster as predicted by the model of Le Pelley (2004).

Materials and Methods

Participants

Forty college students (44.5% male, mean age 19.32) volunteered for the present study and received course credit for their participation. Participants were randomly assigned to conditions.

Apparatus

The apparatus is thoroughly described in Nelson and Sanjuan (2006) and will be only briefly summarized here. The apparatus was a first-person video game where the participant viewed a space background through a viewscreen that contained a series of round sensor ovals at the bottom and a point counter at the top. Through this screen a green bird-like spaceship was flying about in a random path. Two different backgrounds could be viewed through the screen. One was a predominately red star-field image of the Crab nebulae and the other was a predominately blue image of the Pillars of Creation area of the Eagle 1 nebulae. The sensor used was the illumination of the center oval for 5-sec with the color red. The game was played on four laptop computers, two with 14.1-in screens and two with 15.6-in screens, separated by cubicles. Assignment to conditions was random across computers.

Procedure

After obtaining informed consent, participants read instructions silently which were then read aloud to the participant by the attending researcher. The instructions were the same as those reported in Nelson and Sanjuan (2006) and informed the participant that he or she was to earn points by firing torpedoes at the enemy spaceship. They were informed that they would be attacked by the spaceship and that attacks would drain their power leaving them unable to play until their power was recharged. To avoid the effects of the attack participants were told to suppress their torpedo firing to conserve energy in preparation for the attack. They were told that sensors might appear which might, or might not, be useful.

Upon pressing the “S” key to start, they played the game for 60 sec before the first trial. Torpedoes were fired by clicking the mouse on a variable-ratio 3 schedule where a torpedo was launched, on average, after every three clicks. A random half of those impacted the spaceship and incremented the points in the point counter throughout the game. Each of the following eight trials consisted of the 5-sec illumination of the center oval with red followed by an attack from the spaceship. The inter-trial interval was variable with a mean of 9 sec. Phase 2 began with a context change where the participant was informed that he or she and the enemy spacecraft were being transported to new galaxy and the background screen was changed. Participants in group After Extinction then received eight trials where the red sensor was illuminated without the attack. Participants in group No extinction simply played the game for the same amount of time with no events occurring. Following the final trial, participants then received eight trials where the spaceship attacked without warning with a fixed ITI of 20 sec.

Data analysis

The computer recorded the number of mouse clicks 5-sec before the cue (precue) and during the 5-S cue (cue). These data were converted into standard suppression ratios (cue/(pre_cue + cue)) for analysis during phase 1 and 2. During the test the number of clicks per second across the 20-sec intervals between outcomes, in 5-sec bins, were analyzed as well as a suppression ratio designed to reflect suppression across a trial in the form of 5-sec bin 1/(5-sec bin 1 + 5-sec bin 4). Simple effects were conducted using error terms appropriately derived from the appropriate pooling of terms from overall analysis (e.g., Howell 1987) and degrees of freedom pooled and reduced using the Welch (1947) and Satterhwaite (1946) procedure. Effect sizes are reported as partial-eta squared η2 and Cohen's d, as appropriate.

Participants were 101 college-student volunteers (47% male, mean age 19.81), randomly assigned to conditions with 25 in each group and 26 in group No Extinction–Different outcome.

Participants played a video game (the game and all stimuli used here are thoroughly described in Nelson et al., 2014, and also described in the video located at: https://www.youtube.com/watch?v=Yl9Q8ThMN40). The same computers used in Experiment 1 were used here. Conditions were random between computers and participants. Through the screen participants viewed three-dimensional colored and animated space environments with spinning space stations. The environment used in “response training,” described below, was the inside of a green wireframe-gridded box-shaped chamber. Two other environments were used. One environment (“Boutonia”) contained blue nebula and a blue ringed planet along with a spinning sphinx-like space station. Another (“Wagneria”) contained green and yellow nebulae, no planets, and a round spinning space station. All environments were viewed though a view screen that had four different gun-like weapons attached to each corner and a crescent-shaped panel of eight round sensors at the bottom center of the screen. Four spaceships were used, and each appeared from a corner of the screen and was repelled by activating the weapon located in that corner. Weapons were activated by accumulating 5 sec of responding at the rate of 3 per second and began firing with every other keypress so long as a rate greater than 1 press per second was maintained. One ship was the “Learian” which was blue, saucer shaped, and appeared from the top right of the screen and was repelled with the backspace key. The “Juk Destoyer” was a longer red spaceship appearing from the bottom right and repelled by pressing the number-pad zero key. The “Luckonian” was a seahorse-shaped ship appearing from the bottom left of the screen and repelled by pressing the left shift key. The fourth ship was the “Stellarian,” a white airplane-like ship that appeared from the top left of the screen and was repelled by pressing the left tab key.

Referring to the design in Table 2, “+” and “!” were the appearances of the Learian or Juk Destroyer spaceships, counterbalanced. “*” was the appearance of the Luckonian spaceship. The background contexts A: and B: were the Boutonia and Wagneria galaxy contexts, respectively. These were not counterbalanced as no comparisons between contexts were made.

Response training

After obtaining informed consent, participants were seated at the computer and headphones were placed on them. They were instructed to press the “B” key on the keyboard and all instructions were subsequently delivered by the software. They were instructed that they must learn to activate weapons by pressing keys on the keyboard to repel invading spaceships, and received practice trials with four different ships. On the first trial with a particular ship the instruction informed the participant of the name of the ship, the weapon used to repel the ship, and the key to press to activate the weapon. They were instructed that the key must be pressed rapidly and repeatedly. The participant then pressed the key until the weapon was activated. The ship was repelled after firing eight shots. On subsequent appearances of a ship, no further instructions were provided. Participants were trained to respond to four different ships (five trials each) in the manner described in the “response training” phase of Experiment 1 in Nelson et al. (2014).

Experimental phases

At the end of response training, Phase 1 began. Participants underwent a context change to a different galaxy referred to as “Boutonia” (described and pictured in Nelson et al. 2014) and received 13 conditioning trials where the illumination of a 20-sec flashing (3-Hz) sensor light, in a panel at the bottom of the screen, was paired with the appearance of an attacking spaceship. The spaceship arrived after the light had been flashing for 5 sec and remained on the screen for the remaining 15-sec of the light. There were 10 trials where a red light was paired with the appearance of either the Learian or the Juk Destroyer, counterbalanced. There were three trials (trials 4, 8, and 11) where a blue sensor was flashed and paired with the Luckonian. The ITI was variable and averaged 20 sec.

Phase 2 began 5-sec following the final trial of phase 1. The participant was transported to another galaxy and then received 10 trials. Trials 1 and 4 were filler trials where the blue sensor was paired with the Luckonian. The remainder of the trials varied by group. Those in the After Extinction conditions received eight trials where the red sensor was illuminated for 20 sec but the spaceship appearing in Phase 1 did not appear. The No Extinction condition received trials with a black sensor that produced no functional change in the visuals of the game, thus they were simply exposed to the context during those trials. The ITI was again variable and averaged 16 sec.

Testing began after Phase 2. Twenty-six seconds after the last Phase 2 trial the first trial of testing began. On each of 12 trials of testing a spaceship appeared for 15 sec without being signaled. Trials were 26-sec apart (spaceship offset to onset). Those in the Same outcome groups received trials where the same spaceship as was used in conditioning appeared. Those in the Different outcome received trials with the different spaceship. After the 12th trial, no spaceship appeared and responding was measured for 65 sec.

The computer recorded the number of responses made on the backspace and number-pad zero keys during each second 5-sec before a sensor was illuminated (precue), and for the 20-sec during the sensor on trials where sensors were present. Responding on the left shift key, used on the filler trials, was not recorded. During each trial of testing, measuring of responding began 1-sec after the outcome exited on the previous trial and continued for the next 25 sec. Data from periods where the spaceship was absent were analyzed with mixed (between-within) analysis of variance (ANOVA). The remaining details are the same as Experiment 1.

Simulations

Simulations of Experiment 2 were conducted with the model of Le Pelley (2004) implemented in Microsoft Excel 2016 with the equations programmed in Visual Basic for Applications. The simulations of the Hall and Rodriguez (2010) model were done with the Pearce & Hall Simulator 1.0 (Grikietis et al. 2016) available at the Centre for Computational and Animal Learning Research Webpage (Cal-R.org). The filler trials were the same between conditions and ignored as they did not change the ordinal between-group predictions. In the remaining trials the average ITI was the same duration as a cue. Thus, we modeled each trial as two trials. First, we presented a stimulus designated as a context without the outcome, and that was followed with a presentation of the context stimulus along with another representing the cue. These were paired with the outcome.

We adjusted the parameters of the models so that the predictions ordinally matched the behavior we observed during conditioning and extinction. First, with respect to the model of Le Pelley (2004), θ and β for nonreinforced trials were adjusted so that 10 extinction trials reduced responding by approximately 72% from the first to the last extinction trial, matching what occurred in the extinction phase of Experiment II. Then, the starting value for the attentional parameters (α and ς) for the context were adjusted until responding to cue and context was 17 times greater than that of the context alone at the end of conditioning, as was observed in conditioning. λ was set to 0.8 following Le Pelley (2004). In the Pearce-Hall simulator we modified β for reinforced and nonreinforced trials, as well as γ. λ was set to 1.