Context-gated statistical learning and its role in visual-saccadic decisions.

Adaptive behavior in a nonstationary world requires humans to learn and track the statistics of the environment. We examined the mechanisms of adaptation in a nonstationary environment in the context of visual-saccadic inhibition of return (IOR). IOR is adapted to the likelihood that return locations will be refixated in the near future. We examined 2 potential learning mechanisms underlying adaptation: (a) a local tracking or priming mechanism that facilitates behavior that is consistent with recent experience and (b) a mechanism that supports retrieval of knowledge of the environmental statistics based on the contextual features of the environment. Participants generated sequences of 2 saccadic eye movements in conditions where the probability that the 2nd saccade was directed back to the previously fixated location varied from low (.17) to high (.50). In some conditions, the contingency was signaled by a contextual cue (the shape of the movement cue). Adaptation occurred in the absence of contextual signals but was more pronounced in the presence of contextual cues. Adaptation even occurred when different contingencies were randomly intermixed, showing the parallel formation of multiple associations between context and statistics. These findings are accounted for by an evidence accumulation framework in which the resting baseline of decision alternatives is adjusted on a trial-by-trial basis. This baseline tracks the subjective prior beliefs about the behavioral relevance of the different alternatives and is updated on the basis of the history of recent events and the contextual features of the current environment.

Learning about the statistical regularities of the environment offers significant behavioral advantages to an organism. It enables more accurate prediction of events in the near future, which potentially allows for faster and more accurate behavioral responses (Nissen & Bullemer, 1987). Conversely, when the environment changes, it will be important to know that such a change has taken place so that behavior can be adjusted accordingly (Brown & Steyvers, 2009; Redish, Jensen, Johnson, & Kurth-Nelson, 2007). Indeed, inappropriate perseveration of behavior that is no longer relevant to a task is a defining characteristic of several neurological and psychiatric conditions (Lezak, 1995; Luria, 1966).

That humans and other animals are highly sensitive to statistical regularities has been demonstrated in a wide variety of domains, from simple conditioning (Dickinson & Mackintosh, 1978) to learning of artificial grammars (Reber, 1967; Redington & Chater, 1996) and extended sequences of spatial targets (Kinder, Rolfs, & Kliegl, 2008; Nissen & Bullemer, 1987; Reed & Johnson, 1994; Shanks & Johnstone, 1999). Across these various domains, the relations between events in the environment and behavior can be learned, even when these relations are probabilistic and subject to noise (Courville, Daw, & Touretzky, 2006; Shanks, Wilkinson, & Channon, 2003). In many studies learning appears to be implicit, although whether learning can occur in the absence of awareness is contentious (Cleeremans, Destrebecqz, & Boyer, 1998; Shanks, 2010).

In recent years, the role of statistical learning in the exploration and interpretation of the visual world has become more widely recognized. Through the acquisition of knowledge of the (typical) spatiotemporal relations between visual objects, object recognition (Fiser & Aslin, 2001; Turk-Browne, Jungé, & Scholl, 2005) and visual search (Chun & Jiang, 1998; Eckstein, Drescher, & Shimozaki, 2006; Geng & Behrmann, 2002, 2005; Jiang & Chun, 2003; A. D. Smith, Hood, & Gilchrist, 2010) are facilitated. For instance, in contextual cuing experiments (Chun & Jiang, 1998; Jiang & Chun, 2003), the spatial configuration of a set of distractor items provides information to the observer as to where the target may be found. In more naturalistic scenes, the surrounding configuration of “nontarget” objects may constrain the likely location of some task-relevant target (Eckstein et al., 2006; Torralba, Oliva, Castelhano, & Henderson, 2006). One interpretation of these effects is that the context elicits a prior expectation that interacts with the sensory evidence (likelihood) to guide covert and overt attention to the task-relevant information. For example, cars are typically found on the ground, rather than in the sky, and chimneys are situated on rooftops. Long-term learning of such relations allows the observer to restrict his or her search space and thereby speed up localization and identification of a target.

Another type of statistical dependency that aids the allocation of covert and overt attention concerns the temporal order of visual events. One straightforward demonstration of learning such temporal relations is the standard cuing effect observed in simple reaction time experiments (Posner, 1980). If a cue is predictive of where the upcoming target is likely to appear, observers are quicker to respond to the target onset. The flip side of this facilitation occurs when the interval between cue and target onset is prolonged (Maylor & Hockey, 1985; Posner & Cohen, 1984; Samuel & Kat, 2003; Taylor & Klein, 2000): under these conditions, reaction times are increased.

This inhibitory effect is thought to reflect a bias against returning attention to locations that have been recently inspected either overtly or covertly (Klein, 2000; Klein & MacInnes, 1999; Maylor & Hockey, 1985; Posner & Cohen, 1984; Samuel & Kat, 2003; Taylor & Klein, 2000). Inhibition of return (IOR) has been argued to be a useful and adaptive mechanism through appeal to a statistical regularity that is assumed to be operative in the natural world: locations that have been inspected recently are unlikely to become informative in the near future, because the information that could be collected from that location will not have changed in the intervening time period. Efficient information gathering is aided by directing cognitive resources at locations that have not yet been explored (Klein & MacInnes, 1999; Posner & Cohen, 1984).

Though intuitively appealing, the idea that IOR acts as a “foraging facilitator” has recently come under closer scrutiny. Dodd, Van der Stigchel, and Hollingworth (2009) reported evidence of a pronounced IOR effect in visual search, but not in a range of other visual tasks that were performed on the same stimuli (e.g., memorization). The effect manifested itself in longer fixation durations preceding saccades to recently fixated locations, as well as a reduced probability of refixation in search, compared with the other tasks. These findings suggest a significant degree of task dependence and flexibility in IOR. Adding to this evidence, T. J. Smith and Henderson (2009) found only an effect on fixation duration before return saccades in participants who were viewing natural scenes for a memorization task. However, in this instance, the latency effect was not accompanied by a choice bias against returning to previously fixated locations (see also Hooge, Over, van Wezel, & Frens, 2005).

Moreover, Farrell, Ludwig, Ellis, and Gilchrist (2010) have pointed out that the extent to which the temporal stability assumption underlying IOR holds may vary from one local environment and task to the other. For instance, return locations (i.e., those locations that have been recently fixated) are much more likely to contain relevant information when one is viewing an array of closed-circuit television screens showing dynamic environments compared with the case of scanning a bookshelf in search of a particular title. To determine whether IOR is adjusted to such variations in the environmental statistics, Farrell et al. (2010) examined oculomotor IOR in sequences of saccadic eye movements generated under conditions in which the likelihood that a target would require a saccade to location that had been recently fixated was varied between groups of participants. The inhibitory effect was amplified when participants were rarely directed to look back at the previously fixated location and was abolished altogether when they frequently had to look back to the return location.

Such findings raise a number of fundamental questions about behavior. How do individuals learn about the statistics of their environment and use those statistics to guide predictions of future events? What happens when the statistics in the environment change? In the specific case of visual exploration, how is it that participants are able to adapt their behavior to the statistics of one local environment (e.g., the lab setting in Farrell et al., 2010), but then presumably do not generalize those statistics to different environments (e.g., outside the lab), where they would arguably be maladaptive? Finally, if individuals can restrict the use of statistical knowledge to appropriate environments, can they learn about the statistics of different environments in parallel? Although a wealth of research on statistical learning has comprehensively described the behavior of participants in stationary environments, it rarely speaks to these more challenging cases that are arguably more representative of natural behavior (though see Speekenbrink & Shanks, 2010). We are concerned with how individuals learn about their environmental statistics under nonstationary conditions and how they apply this acquired knowledge in their oculomotor behavior.

In this article, we examine two potential mechanisms of statistical learning. A simple recency mechanism might rely on monitoring a short history of recent events in order to set up the expectations for the near future. Additionally, the contextual features of an environment might be used to retrieve knowledge of the statistical structure of the local environment, including, for example, where behaviorally relevant information may be and what type of events are likely to occur. The critical distinction between these mechanisms is whether the environmental statistics are estimated from the recent history alone or whether an internal representation or memory of these statistics is invoked when a relevant contextual cue is presented. We, respectively, refer to these two mechanisms as local tracking and contextual learning.

Local Tracking

Experimental psychology has revealed many examples where recently processed information and events exert a stronger influence over behavior compared with information presented longer ago (Bertelson, 1961; Jarvik, 1951; Jones, Love, & Maddox, 2006; Speekenbrink & Shanks, 2010). It is well known that in laboratory experiments, behavior on a given trial is frequently influenced by the nature of the previous trial(s). These sequential effects or dependencies occur even in stationary, randomized experimental paradigms (Bertelson, 1961; Kirby, 1972; Remington, 1969; Soetens, Boer, & Hueting, 1985). That is, even though the underlying statistics of an environment remain constant and temporal correlations in the input sequence have been purposefully removed, participants often develop expectations that stimuli will alternate from trial to trial or stay the same (in simple two-choice reaction time tasks: Bertelson, 1961; Cho et al., 2002; Kirby, 1972; Remington, 1969; Soetens et al., 1985; in visual search: Maljkovic & Nakayama, 1994, 1996; McPeek, Maljkovic, & Nakayama, 1999; Walthew & Gilchrist, 2006; and in visually guided saccades: Anderson, Yadav, & Carpenter, 2008; Carpenter, 2001; for a recent review on sequential effects in saccadic behavior, see Fecteau & Munoz, 2003). Such biases manifest themselves as faster and more accurate responses when the actual stimulus and response on trial n match the expectation derived from trial n − 1.

These sequential effects might appear puzzling given that there is rarely any actual relationship between successive trials. However, several models (Cho et al., 2002; Wilder, Jones, & Mozer, 2009; Yu & Cohen, 2008) posit that sequential effects are the manifestation of a mechanism that is responsible for adaptation to time-varying statistics in the nonstationary environments that are typical of the world outside the lab. In nonstationary environments, the statistics that govern behavior (e.g., stimulus presentation in Anderson & Carpenter, 2006; reward delivery in Gallistel, Mark, King, & Latham, 2001; Sugrue, Corrado, & Newsome, 2004) change periodically, typically without any external marker to signal the change. Conceptually, the challenge for the subjects is two-fold. First, they must identify a genuine change as distinct from the random variability that may occur in the environment. In case of unsignaled changes, the only way to identify a genuine contingency change is to monitor the history of recent instances (i.e., trials in an experimental context; Brown & Steyvers, 2009). If new observations are inconsistent with the expectations built up from recent instances, this would suggest that the environment has changed. Second, having detected a contingency change, behavior needs to be adapted to the new statistical structure.

Both challenges can be met through some fairly simple updating algorithms. An illustration of this point may be found in a study by Anderson and Carpenter (2006). In that study, observers simply generated visually guided saccades to a single target that appeared either to the left or right of a central fixation point. The probability of a leftward or rightward target was changed abruptly at irregular intervals. Following a change in the probability of each saccade direction, latency gradually changed in line with the direction of the probability variation. For example, if leftward targets became more likely, then latency for leftward saccades gradually decreased.

Anderson and Carpenter (2006) modeled their data using a simple trial-by-trial updating rule of the expectation of leftward and rightward targets. This expectation was assumed to be a combination of the expectation in place on the previous trial and an additional input that depended on the nature of the previous trial. That is, the expectation was strengthened if the previous trial was congruent with the expectation but not if the previous trial was incongruent. In other words, recent experience was weighted most heavily, and experiences further in the past were progressively discounted. When there are genuine temporal correlations in the input, this simple form of one-trial facilitation or priming can mediate adaptation to changing statistical regimes.

Note that such adaptation is not limited to this simplest of situations in which the relevant feature (e.g., target location) directly maps on to the behavioral response. Behavior is still facilitated if the tracked feature does not form the basis of the response. In many visual search studies, the “reported” feature is different from the one that is repeated across trials. For instance, repetitions of target color and target location in visual search facilitate responses to some unrelated feature of the target item, such as the orientation of a small symbol inside it (Walthew & Gilchrist, 2006) or on which side of the target is notched (Maljkovic & Nakayama, 1994, 1996; McPeek et al., 1999). Such findings would indicate that facilitation through recency does not act at the level of the motor response itself but rather helps the observer find the task-relevant item more quickly so that the required motor response can be decided upon sooner.

Moreover, even the facilitation of localization may be fairly sophisticated. For instance, Kristjánsson, Mackeben, and Nakayama (2001) had observers perform a discrimination task at a cued location. The target location within the cue was repeated across trials with some probability (a “streak” condition), alternated between two locations within the cue from trial to trial, or randomly varied between these two locations across trials. Critically, the spatial location of the cue–target compound itself was randomized. Discrimination performance was best in the streak condition, suggesting that the facilitation of localization can even operate in relatively abstract, object-centered co-ordinates. Similar object-based facilitation was obtained with streaks of other cue–target relations. For instance, consistently pairing the target with a distinctly colored or shaped region of the cue also facilitated discrimination (Kristjánsson & Nakayama, 2003).

In general, evidence from randomized, stationary environments, environments with temporal correlation in the trial structure, and environments with transitions in statistical contingencies suggests that individuals form expectations about the future, based on some memory for recent events. One intuitive explanation for the strong influence of recent experience is that the behaviorally relevant statistics of an environment will have a characteristic time scale. More recent experience is likely to have been acquired under the same environmental statistics and will form a more solid basis for making predictions about the near future. As time goes by, it is likely that a different set of statistical regularities apply. As a result, experience acquired a longer time ago is a less reliable predictor.

Contextual Learning

The utility of a local tracking mechanism is obvious when the statistics of some process change without any external indicator. However, this mechanism provides no opportunity to apply long-term knowledge in the formation of expectations about the future. The capacity to use stored knowledge over longer time scales is desirable when changes in statistics are correlated with other changes in the environment. We refer to such external markers of a change in the environment as contextual signals. Note that many authors use the term context to refer to the statistical structure of the environment itself, so that an unsignaled change in the statistics is regarded as a change in context (Yu & Dayan, 2005). In this study, we reserve the term context for the surface features of the environment, independent of the underlying statistics. Accordingly, the statistics may change without a change in context or with a concomitant change in context.

If features of the environment are reliable predictors of the spatial or temporal statistics of that environment, an adaptive system would be expected to take advantage of this predictive power and learn such relationships. The context may activate a plethora of expectations, for instance concerning where important information will be and what kinds of events may occur in that context. Expectations about the near future may then be derived not only from the immediately preceding events but also from a memory representation of the statistics that accompany the multisensory cues that define a particular context. As a result, a change in context may lead to a rapid adjustment of expectations without the need for a period of tracking the relevant behavioral variable(s).

The modulating role of context in setting expectations about future events is widely recognized in the literature on conditioning. A good example is the phenomenon of renewal after extinction (see review by Bouton, 2004). Conditioning experiments start by forming a link between stimulus and response, through reinforcement or pairing with an unconditioned stimulus. Extinction refers to the reduction in responding when the stimulus is no longer predictive of the event with which it was paired. However, extinction appears to be context specific: a stimulus–response association formed in one context, but abolished in a different context (typically a completely different experimental apparatus), is reinstated rapidly when the animal is returned to the original context in which learning occurred (Bouton & King, 1983). In other words, the animal does not have to “relearn” the predictive relation from scratch, implying that extinction did not cause the original association to be “unlearned.”

In order to explain the dependence of extinction on context, several authors have argued that the animal has acquired multiple associations involving the stimulus (Bouton, 2004; Gershman, Blei, & Niv, 2010; Redish et al., 2007): one that involves an expectation of a future event and one that does not. The context determines which of these memory representations is retrieved. That is, if the current context is classified as the same as during the acquisition phase, delivery of the stimulus produces a response. If the current trial is seen as belonging to the extinction context, the stimulus does not lead to a response.

An important feature of this proposal is that context itself does not directly determine the behavioral response. Rather, the context effectively gates the associations between stimuli and responses. For such gating to be effective, it is vital that the agent categorizes the current context correctly as a situation that has been experienced before. In other words, the cues that make up a current context need to be mapped on to an internal state representation that matches these cues and contains relevant information about the statistics operating in that context (Gershman et al., 2010; Redish et al., 2007). If the cues do not provide a good match with the bank of internal state representations, a new state is created for which learning proceeds afresh. Redish et al. (2007) refer to this process as state-splitting.

In the conditioning experiments just discussed, the context acts to retrieve a representation of the statistical structure of the environment, which is then used to govern how the animal responds to the stimulus. A similar suggestion has been made in the domain of categorization. Here, a stimulus is defined by a set of values on multiple dimensions, and the category structure partitions the multidimensional space into a small set of discrete categories (Jones & Sieck, 2003; Nosofsky, 1986; Shepard, Hovland, & Jenkins, 1961). Lewandowsky and colleagues (Lewandowsky & Kirsner, 2000; Lewandowsky, Roberts, & Yang, 2006) have shown that subjects can learn to associate different contextual cues (e.g., the color of a stimulus) with different category structures. Again, the context itself does not produce a behavioral response (the signaled category) but rather dictates how a subject processes and responds to an incoming stimulus.

The examples reviewed in this section suggest that in addition to simple recency, knowledge of different environments may be associated with different contextual cues. The context does not control behavior directly, but indirectly through an internal representation of the environmental structure. Once the current context has been recognized and categorized, expectations about future events and the consequences of particular actions can be formed and tested against experience. The critical issue we focus on here is whether such contextual learning gates associations that do not involve particular stimuli or objects, but instead embody more abstract temporal regularities in the world. In particular, we ask whether contextual learning can inform people's expectations about the relevance of previously visited locations, irrespective of the identity of those locations or the information they contain.

Mechanisms for Adaptation of Oculomotor IOR

Given the limited resolution of human vision, gaze shifts are critical for the uptake of visual information from the environment (Findlay & Gilchrist, 2003). Moreover, shifting gaze to an object or region of interest frequently forms the starting point of purposeful behavior (e.g., when catching a ball, we first tend to fixate it). As such, it is clearly of some importance that the mechanisms involved in gaze shifting can be flexibly tuned to the statistical properties of the environment. IOR is a particularly instructive phenomenon in this regard. It may be viewed as an expression of knowledge of the “average” statistics of the environment, which can be modulated by the temporally local statistics (as in Farrell et al., 2010). Oculomotor IOR is therefore a good candidate behavior to inform the broader issue of the role of learning and context in the adaptation of gaze shifting mechanisms in nonstationary environments.

Consider how a local tracking mechanism could give rise to adaptation of IOR. In the paradigm of Farrell et al. (2010), subjects generate short sequences of two saccadic eye movements. The second movement may direct the eyes back to the previously fixated location or to one of two locations that have not yet been fixated in that sequence. Suppose that individuals have an expectation that the current sequence will be of a similar type as the previous sequence. If the previous sequence required a return saccade, the response associated with the return location on the current trial may be selectively facilitated. Likewise, if the previous sequence required a saccade to a new location, responses to the two new locations may be selectively facilitated on the current trial. If the cue triggering the second saccade matches the expectation, the target-directed response will be initiated more rapidly, resulting in a latency advantage. In this way, local tracking could lead to average matching to the statistical properties of the environment (e.g., Walthew & Gilchrist, 2006).

Now consider the influence of context. It may be that participants learn to associate the experimental context (e.g., the lab setting: the eye tracker, the types of displays used, the experimenter, etc.) with the relevant temporal statistics operating in that environment. As a consequence, presentation of the cues that define this particular context could enable participants to retrieve an internal model of the associated environmental statistics. This internal model may then be used to instantiate a certain amount of bias toward or away from certain locations depending on whether they had been recently visited.

We studied the contribution of both types of mechanism to the formation of expectations about future events in a nonstationary environment. Specifically, we introduced step-changes in the prior probability that participants were directed to look back at the previously fixated location. To assess the contribution of a local mechanism (Experiment 1), we examined whether IOR is adapted to the local statistics when the changes in prior probability were unsignaled or “hidden” (Anderson & Carpenter, 2006; Brown & Steyvers, 2009; Sugrue et al., 2004). Under such conditions, any adaptation would presumably be mediated by a pure recency effect.

We then explored the contribution of contextual cues in learning about the environmental statistics, first in concert with a local tracking mechanism (Experiment 2) and then in isolation (Experiment 3). In both these experiments, the different levels of prior probability were marked with a surface feature of the visual environment: the shape of the cue that signaled the target for the upcoming saccadic eye movement. Provided that this shape change was sufficiently salient for our subjects to differentiate the contexts internally (Gershman et al., 2010; Redish et al., 2007), it might be possible for them to retrieve the relevant statistical knowledge as and when they encounter a particular contextual cue.

General Method

Materials and Procedures

All experiments reported in this article were performed with variants of the gaze-contingent saccade sequencing paradigm used in our previous work (Farrell et al., 2010; Ludwig, Farrell, Ellis, & Gilchrist, 2009), illustrated in Figure 1. At the start of each sequence, observers were presented with a single ring on the screen (1.5° radius). Each of the four possible ring locations was used equally often as the starting point but randomly intermixed. Subjects were asked to fixate in the center of the ring. Following accurate fixation, the experimenter launched the trial. Three additional rings appeared in the periphery, with a 7°separation between adjacent rings.

Figure 1

Schematic trial sequence examples with two types of movement cues. Time runs from top to bottom. A. Line cues: The orientation of the line tells the subject where to look next. The eyes are directed to a new location within the sequence on the second saccade. B. Arrowhead cues: The eyes are directed to a return location on the second saccade.

A cue presented at fixation signaled to the observer where to look next. An invisible “tolerance” region (3° radius) around the target ring was used to classify the accuracy of the saccade online. Provided the saccade landed within this tolerance region, the second cue was presented after a delay of 390 ms, and the participant was to direct his or her gaze to the indicated target circle. If the first saccade was inaccurate, the sequence was abandoned. If either the first or the second saccade was inaccurate, a 100-ms low tone (500 Hz.) was played. Upon successful completion of a sequence, a higher tone (750 Hz) sounded. Each combination of starting position and the three possible first target rings (i.e., 12 combinations) was repeated six times in a block, randomly intermixed. The choice of the second saccade target was primarily determined by the experimental condition (i.e., return or new, which varied probabilistically).

Two types of movement cues were used across the experiments: a single line segment (measuring 0.5°; Figure 1A) or an arrowhead (Figure 1B), made up of two line segments half the size of the single one. Cues and rings were black and presented on a mid-gray background. Stimuli were viewed from a distance of 57 cm on a 21-in. CRT monitor (either an Eizo Flexscan T965 [Eizo Nanao Corp., Hakusan, Japan] or a LaCie Blue Electron [LaCie Group, Paris, France). Eye movements were monitored with either the EyeLink II or the EyeLink 1000 systems (SR Research, Kanata, ON, Canada), the former sampling at 500 Hz and the latter at 1,000 Hz. Only the position of the dominant eye was tracked.

All experiments involved a variation in the prior probability of refixation on the second saccade. The return location is one of three potential target locations for the next saccade. The return probability in any one sequence was either low (1/6) or high (3/6). A block of trials contained an equal number of low and high return probability sequences. As a consequence, the overall return probability in a block was equal (2/6): across a block, the return location was just as likely to be the target for the next saccade as either of the two new locations.

In Experiments 2 and 3, the movement cue (arrowhead vs. line) covaried with the return probability; in these cases, the cue-probability mapping was counterbalanced across observers. If there was no cue variation (Experiment 1), only line segment cues were used, as in our previous work (Farrell et al., 2010; Ludwig et al., 2009). Participants were never informed about the possibility of statistical variation or the mapping between the different cues and return probabilities.

Only second saccades were analyzed, as a function of whether they were directed to return or new locations and whether the return probability was low or high. The latency of the second saccade was defined with respect to the presentation of the second movement cue. In the offline analyses, we adopted a somewhat more liberal accuracy criterion than in the online measurement (Farrell et al., 2010). A movement was classified as accurate if it started within the acceptance region of the currently fixated ring and ended within the quadrant of the target ring. Errors were coded as saccades that started within the acceptance region of the currently fixated ring but landed in the quadrant of a nontarget ring. Errors can be of different types, depending on the target location. When the target of the movement is a return location, erroneous movements can only be directed to new locations. However, when the movement should be directed to a new location, an error may be directed to the other new location or to the return location.

The average frequencies of the different types of errors across all experimental conditions are listed in Table 1, but these were generally too low for meaningful analysis. Latencies were trimmed to exclude any observations shorter than 80 ms and longer than 1,000 ms. The overall number of correct trials included in the latency analyses reported below ranged from 102 to 279 across all participants and experiments. Median saccade latencies were analyzed throughout, given the small number of observations per participant in some conditions (see Footnote 1).

Table 1

Mean Proportion of Errors in Experiments 1–3

Clearly under these conditions the number of sequence we could collect from each individual participant was relatively limited. For instance, the maximum number of valid trials is 216 in this experiment. For half of these trials, the return probability would have been low and under these conditions the number of return trials is, by definition, small (maximum of 18 across four blocks).

Participants

We tested 30 participants in each experiment, so in total data are reported from 90 participants (65 female, 25 male; 17–48 years age range). Participants were either paid or received course credits. All reported normal or corrected-to-normal vision. Each participant performed five blocks of 72 trials. The first block was discarded as practice, and in the first two experiments, the return probability for this practice block was equal throughout (with a constant, single line segment cue). In the final experiment, the first block was indistinguishable from the remaining blocks but nevertheless discarded as practice.

Experiment 1

In the first experiment, we asked whether human subjects learn about the environmental statistics when the statistics vary locally in time and these variations are unsignaled in any way to the subject (Anderson & Carpenter, 2006; Brown & Steyvers, 2009; Sugrue et al., 2004). As reviewed earlier, the only way to tell that the statistics have changed is by keeping track of a sufficiently short-term history of past trials. If such local tracking occurs, we would expect to observe adaptation of oculomotor IOR in response to the statistical variation. Recall that the overall return probability in a block of trials was equal. Therefore, if subjects do not keep track of the recent history of trials or have a temporal weighting window that stretches too far back in time, we would not expect to see any adaptation at all.

Method

Participants performed sequences of two saccadic eye movements as illustrated in Figure 1A. Only line cues were used. After an initial practice block in which the return probability was equal (i.e., 2/6) throughout, participants performed four experimental blocks of 72 trials. Every 18 trials, the return probability was changed from low to high, or vice versa.1 These changes were not signaled to the participant.

Table 2 shows the structure of the four experimental blocks when the first experimental block started with a low return probability (for half of the participants, the first experimental block started with a high return probability). We were predominantly interested in behavior after a transition in the return probability. The first 18 trials of each block were discarded, because these were preceded by the same contingency at the end of the previous block and a short break in between blocks to rest the participant and recalibrate the eye tracker.

Table 2

Design of Experiment 1

Results and Discussion

Figure 2A shows the mean (of median) saccade latency for each of the four cells of the experimental design (target location: return, new; return probability: low, high). These data were subjected to a 2 × 2 analysis of variance (ANOVA). The main effect of target location was significant F(1, 29) = 17.91, p < .01, η2 = .38, replicating the standard oculomotor IOR effect in saccade sequences (e.g., Farrell et al., 2010; Hooge & Frens, 2000; Ludwig et al., 2009; Vaughan, 1984). There was no main effect of return probability, F(1, 29) = 1.16, p = .29, η2 = .04, or interaction between the two factors, F(1, 29) = 0.90, p = .35, η2 = .03.

Figure 2

Inhibition of return (IOR) effects under low and high return probability conditions in Experiment 1. A. Mean of the median latencies of saccades to return and new locations. B. Temporal evolution of IOR after a switch from one contingency to the other. All error bars are within-subject standard errors of the mean.

The lack of an interaction between target location and return probability might seem to suggest that no adaptation took place in this situation where the statistical variation was unmarked. However, participants will necessarily have had to experience a certain number of trials for adaptation to the new contingency to occur, as a single switch (i.e., a return trial after a run of new trials) does not necessarily indicate a change in the statistics generating the individual trials. Accordingly, when the previous contingency involved a low return probability, we would have expected that IOR should be large to begin with and gradually decrease as the observer is exposed to more trials under the high return probability. Conversely, when the previous contingency involved a high return probability, IOR should be small to begin with, but increase with exposure to the new contingency. At the gross level of the overall data, it is possible that these opposite changes cancel each other out.

The critical question then is how IOR evolves over the course of the 18 trials within a single contingency. To address this question, we analyzed the saccade latencies at this finer time scale, as follows. Over the course of four blocks, observers were presented with 16 series of 18 trials. The first series of each block was discarded as explained earlier. For each participant, this left six high return probability series that followed a period of low return probability and six low return probability series that followed a high return probability series. These series were divided in bins of six trials, and for each bin, we computed an IOR score (defined as the difference in latency between return and new saccades), separately for the low-and high return probability conditions. Note that we could, in principle, have looked at the change in IOR over time using smaller bins. The reason we chose bins of six trials was to ensure that even in the low return probability condition, the expected frequency of a return trial per bin would have been one. Across six series of 18 trials then, the maximum number of return trials for any bin was six, in the unlikely event of a perfect observer who made no errors. Given this constraint, it was not possible to perform this particular analysis on a finer time scale without there being missing values for a large number of participants.

Figure 2B shows the result of this analysis. The figure shows a gradual divergence over time between the IOR effects under the two contingencies. A 2 (return probability) × 3 (bin) ANOVA indicated a marginal effect of return probability, F(1, 29) = 3.18, p = .09, η2 = .10. There was no main effect of bin, F(2, 58) = 0.35, p = .71, η2 = .01, and no interaction between return probability and bin, F(2, 58) = 0.99, p = .38, η2 = .03.

Although it is difficult to draw firm conclusions from a marginal effect in a single experiment, we believe it is appropriate to conclude that either modest adaptation to the different contingencies occurred or that at least some participants must have produced a clear adaptation effect (a sufficient number to obtain a marginal effect). Certainly toward the end of any one contingency, IOR appears clearly dependent on the return probability. The rationale of the fine-scale analysis shown in Figure 2B suggests that we should see a crossover in the magnitude of IOR over the course of the 18 trials from one contingency. However, the IOR effect is identical for the two contingencies in the first bin of six trials. This does not necessarily mean that there is no crossover but rather indicates that the crossover may have happened relatively rapidly over a time scale that we do not have the resolution to detect. To the extent that learning occurred then, the data suggest that the local mechanism operates on a relatively rapid time scale.

To determine the time scale over which the recent history of actual trial events influenced behavior, we examined how the saccade latency depends on previous trials as a function of the recency of those trials, as is typically done in the analysis of sequential effects in conventional reaction time paradigms (Bertelson, 1961; Cho et al., 2002; Kirby, 1972; Remington, 1969; Soetens et al., 1985). In the following analysis, we conditionalized the saccade latency on the previous two trials, irrespective of the underlying return probability.

We examined unbroken runs of three consecutive correct trials. If the sequence was broken by an erroneous or anticipatory movement, the counting of correct sequences started afresh from the next correct trial. Because each of the three trials may be an R or N trial (where return and new trials are denoted as R and N, respectively), there are 23 possible sequences to examine. We code these sequences as character triplets such as RRR, RRN, and so on. The final character indicates the “current” trial, the middle character corresponds to the immediately preceding trial (n − 1), and the first character corresponds to the trial before that (n − 2).

Figure 3 shows the eight different sequence types along the abscissa. Data are shown separately for return and new trials (solid and dashed lines, respectively; data from Experiment 1 are shown by the square symbols). Going from the outer data points to the innermost data points, the current trial is increasingly inconsistent with the recent trial history. Therefore, the signature of sequential dependencies is a latency peak near the middle of the graph (e.g., NNR and RRN sequences; see Cho et al., 2002; Soetens et al., 1985).

Figure 3

Sequential effect analyses for all three experiments. R and N refer to return and new trials, respectively, indicated by the solid and dashed lines. In the character triplets, the final character refers to the current trial (bold typeface), the middle character to the previous trial (1-back), and the first character to trial before the previous one (2-back). Error bars are within-subject standard errors of the mean.

The functions in Figure 3 are broadly consistent with this signature pattern, particularly for new trials. The peak is modified here by the overall longer latencies of return trials (i.e., the basic IOR effect), which elevates the left-hand function and produces an asymmetry in the data. Additionally, disregarding the actual return probability means that the number of new trials will be approximately double the number of return trials. The inevitable consequence is that the function relating mean latency on return trials to the trial history are more noisy. Nonetheless, a trend for latencies to increase as the previous trials become less consistent with the current saccade is evident.

To statistically assess this trend, we subjected the data from Figure 3 to an ANOVA with the following repeated factors: target location (return vs. new), 1-back similarity (same or different movement type as required on the current trial), and 2-back similarity (same or different movement type as the current trial). For instance, the 1-back effect can be gauged by comparing the two outermost data points with the two innermost data points for each of the two subsets of data. Only those observers who contributed to all eight points in the figure were included (29/30 participants).

The main effect of target location was significant, F(1, 28) = 17.36, p < .01, η2 = .38. This simply reflects a robust overall oculomotor IOR effect. More important, the main effect of 1-back similarity was also significant, F(1, 28) = 7.27,p < .05, η2 = .21. Saccade latency was consistently increased if the current trial was of a different type than the previous trial (e.g., if the target appeared at a new location on the current trial, but appeared at the return location on the previous trial). There was no effect of 2-back similarity, F = 0.83, p = .37, η2 = .03. The two-way interaction between 1-back and 2-back similarity was significant, F(1, 28) = 9.81,p < .01, η2 = .26. This interaction is most likely attributable to the elevated latency for RRR trials. As it turned out that this was the only experiment in which this interaction was reliable, we will not focus on it in detail.

To conclude, this sequential effects analysis suggests a robust facilitation effect when the immediately preceding trial is of the same type as the current sequence. This finding does not imply that any adaptation to a switch in the statistical contingency should be complete after a single trial. However, Figure 2B suggests that it only takes a relatively small number of trials dominated by one type (e.g., new) or the other (e.g., return) to (a) overcome the preceding contingency and (b) reach a new steady state of adaptation to the new contingency. Indeed, visual inspection of Figure 2B suggests that a steady state may already be reached within 7–12 trials of a change in contingency. The sequential effects analysis suggests that this state may be achieved through the operation of a local facilitatory mechanism that looks back to the immediately preceding trial.

The finding that we get some degree of adaptation at all in this experiment is surprising for a number of reasons. In most previous studies that required subjects to track the hidden state of some variable (Anderson & Carpenter, 2006; Gallistel et al., 2001; Sugrue et al., 2004), the environment changed relatively infrequently. That is, each contingency was presented for at least 50 trials in Sugrue et al. (2004; monkey observers), 70 trials in Anderson and Carpenter (2006; humans), and ~40 min in Gallistel et al. (2001; rats). In contrast, in our experiment, the environment changed much more frequently (every 18 trials). Moreover, in our experiment, the statistical variation concerned a quantity that was, presumably, not as behaviorally salient as something like the rate of reward in previous studies (Baum & Davison, 2004; Davison & Baum, 2000; Gallistel et al., 2001; Sugrue et al., 2004). Like Anderson and Carpenter (2006), we “only” varied the likelihood that a particular location would become the target for the next saccadic eye movement. Unlike Anderson and Carpenter (2006), however, this location did not correspond to a fixed spatial point in head-centered or retinotopic co-ordinates, because the starting point and first target location were completely counterbalanced in our experiment.

The results of Experiment 1 provide evidence for the involvement of a recency mechanism that facilitates responses congruent with recently experienced trials. Indeed, a sequential effects analysis suggested that this mechanism was predominantly driven by the immediately preceding trial. As reviewed in the Introduction, sequential effects involving the previous trial are ubiquitous in experimental psychology, both in randomized and correlated experimental designs, and in stationary and nonstationary environments.

Our findings are novel in that they concern facilitation of a correlation between two consecutive movements, primed by a previous sequence, rather than a single motor response. Moreover, facilitation is not bound simply to a fixed location in space or a fixed movement vector. Instead, the facilitation of different movement programs has to occur “on the fly,” once the first eye movement of the sequence has at least been specified and probably even completed. Although a local mechanism may appear as a simple, repetitive priming mechanism, what is being primed is clearly quite an abstract variable that has to be matched to specific movement vectors online.

Experiment 2

Although local tracking is a useful mechanism to adapt to hidden changes in statistics, a number of theories from the animal learning literature suggest that contextual cues may be used to make predictions about future events (Bouton, 2004; Gershman et al., 2010; Redish et al., 2007). In the next two experiments, we examined the role of context by consistently pairing the environmental statistics with a particular cue. In Experiment 2 we asked whether—and to what extent—the adaptation to the statistical variation observed in Experiment 1 is aided when this variation is consistently signaled by a change in a surface property of the environment, namely, the shape of the cue. As in Experiment 1, the overall return probability across a block of trials was equal. If the context was ignored, we would expect to see the same effects as in Experiment 1. To the extent that subjects used the context, we would expect to see more pronounced adaptation.

Experiment 2 closely mirrored the design of Experiment 1, except that the two contingencies were paired with different types of movement cues. The two types of cues are illustrated in Figure 1. For half of the participants, a low return probability was signaled by the oriented line segment and a high return probability was associated with the arrowhead cue. This mapping was reversed for the other half of the participants. The mapping remained constant throughout the four experimental blocks of 72 trials. Accordingly, the cue switched every 18 trials in synchrony with the change in the return probability. Participants were not told about the statistical variation and its mapping onto the different cues.

Table 3 shows the structure of the four experimental blocks in the same format as shown for Experiment 1 in Table 2. For the purpose of illustration, the table shows a session that starts with a low return probability in the very first experimental block, which was paired with a line cue. However, both the order of the two contingencies and their mapping with the different contextual cues were counterbalanced across participants. As in Experiment 1, the first 18 trials of each block were discarded.

Table 3

Example of the Sequence of Trials for a Single Participant in Experiment 2

Figure 4A shows the mean saccade latencies in the four critical conditions. It is immediately obvious that the interaction between target location and return probability is more pronounced in the presence of the contextual cue (compare with Figure 2A): substantial IOR is observed in the low return probability conditions, but is almost abolished when the return probability is high. We found a large main effect of target location (return vs. new), F(1, 29) = 15.09, p < .01, η2 = .34; in contrast, the main effect of return probability was not significant, F(1, 29) = 1.24, p = .27, η2 = .04. Critically, the interaction between these two factors was reliable, F(1, 29) = 9.33, p < .01, η2 = .24.

Figure 4

Inhibition of return (IOR) effects under low and high return probability conditions in Experiment 2. A. Mean of the median latencies of saccades to return and new locations. B. Temporal evolution of IOR after a switch from one contingency to the other. All error bars are within-subject standard errors of the mean.

As before, we examined how IOR evolved over the course of presentation of a given contingency in bins of six trials. Figure 4B shows a clear separation between the two contingencies, essentially right from the start of a new contingency. The figure and statistical analysis are based on the 29 participants who contributed to all six cells in this breakdown of the data. Unsurprisingly, we obtained a clear main effect of return probability, F(1, 28) = 13.23, p < .01, η2 = .32. There was no main effect of bin, F(2, 56) = 1.24, p = .30, η2 = .04, and no interaction between return probability and bin, F(2, 56) = 0.10, p = .91, η2 = .00.

One implication of the pattern of results in Figure 4B is that the change in context allows participants to immediately adopt the appropriate “task set” in the form of a bias toward return or new locations. As a result, it is tempting to conclude that the influence of a local tracking mechanism was eliminated—or at least, reduced—under these conditions. Once again, analysis of the sequential effects at a finer time scale can shed light on this issue. For the purpose of this analysis, we focused on the local sequences of trials that participants actually experienced, ignoring both the return probability as before and the changing contextual cues.

Figure 3 (circles) shows the outcome of this analysis. The increasing latencies going from outside inward once more suggest the slowing of response times when the current trial does not match the trials that preceded it. The data from the 29 (of 30) participants who contributed to all eight data points were subjected to the same statistical analysis as in Experiment 1. The main effect of target location was again significant, F(1, 28) = 9.10, p < .01, η2 = .25, reflecting the standard oculomotor IOR effect. The main effect of 1-back similarity was near the significance threshold, F(1, 28) = 3.89, p = .06, η2 = .12. There was no main effect of 2-back similarity, F(1, 28) = 1.37, p = .25, η2 = .05, and the interaction between 1-back and 2-back similarity was not significant, F(1, 28) < 1, η2 = .02.

The contrast between the results from the first two experiments demonstrates the contribution of context above and beyond the local tracking demonstrated in Experiment 1. Although some adaptation appeared to occur when abrupt changes in the return probability were unsignaled to the participant (Experiment 1), adaptation was more pronounced and immediate when these changes were paired with an external change in the subject's environment (Experiment 2). The local sequential analysis suggested that a local mechanism was still operative.

The results from Experiment 2 show that subjects adapted almost immediately to a switch in contingency. In line with contextual influences observed in animal learning (Bouton, 2004; Gershman et al., 2010; Redish et al., 2007), this would suggest that the context acts to retrieve a representation of the statistics that operate in the temporally local environment. As reviewed earlier, such gating by context presupposes that subjects have categorized the different contexts as distinct situations and have internally represented the statistics that apply in these situations. Before accepting these inferences from our data, we sought to assess the importance of the short-term stability and periodic alternation between contexts (and their associated statistics).

Experiment 3

In the final experiment, we asked whether some short-term stability (i.e., over a number of trials) in the context and statistics is necessary for the pronounced effect of context on adaptation. Although context and contingency switched fairly rapidly in Experiment 2, the periodic nature of the switches between cues may have made the changes in context more salient, which could have facilitated contextual learning in different ways.

For instance, the “blocked” presentation of the different contingencies means that a context-blind local mechanism will, eventually, match the environmental statistics (cf. Experiment 1). As a result, when the context switches, the output of a local mechanism could be used to update the association between the previous context and the internal estimate of the return probability at that point.

Alternatively, it is possible that no genuine associations between cues and statistics are formed at all. The change in context may simply signal to the subject to reset the internal estimate of the return probability to some neutral baseline expectation and to start local tracking afresh. As a result, when the context changes, adaptation to the new contingency no longer involves “overcoming” the previously built-up bias, but can instead proceed from a more neutral starting position.

Note that both of these processes could have been operative to account for the strong adaptation seen in Experiment 2. The periodic switches in contextual cues may have encouraged expectations to be reset. An episode of local tracking could then be used to update the association between context and statistics. Indeed, this may be how associations are initially acquired. Over time, the influence of resetting expectations and local tracking may diminish, and expectations are set in accordance with the learned mapping between context and statistics. Clearly, the data from Experiment 2 alone do not allow us to identify very precisely whether—and how—a genuine association is formed.

In this regard, there is one straightforward manipulation that should be very informative. If the different contingencies and their contextual cues are randomly intermixed, the temporal correlation between trials is removed. That is, a randomly mixed sequence of low and high return probability trials is indistinguishable from a constant, equal return probability sequence of trials. As a result, one trial is no longer predictive of the likely nature of the upcoming trial(s), thereby removing the correlation on which adaptation under a context-blind local mechanism would be dependent. If clear evidence of adaptation is obtained under these conditions, we can be confident that subjects have formed a genuine association between contextual cues and the statistical structure of the environment defined by those cues.

Experiment 3 consisted of four experimental blocks in which return probability varied between low and high, and this variation was consistently paired with a different context (line or arrowhead cues; counterbalanced between participants). However, the different contingencies now no longer alternated in series of 18 trials but were randomly intermixed throughout the four blocks. As the different contingencies were no longer presented in alternating series, there was no need to discard the first 18 trials of each block; accordingly, data from all four of the experimental blocks were included in the analyses that follow.

Participants performed one practice block as before, but the practice block was now indistinguishable from the experimental blocks. That is, both cue types were used, and they mapped on to the return probability variation in exactly the same way as the remaining blocks that were used in the following analyses. We changed the nature of the practice block from previous experiments on the basis of an a priori expectation that participants would find it difficult to adapt to the different contingencies under these challenging conditions. Therefore, we considered it prudent to give the participants as much exposure to the variation in context and statistics as possible.

The overall latency results are shown in Figure 5A, and are very similar to the results from Experiment 2. There were main effects of target location and return probability, F(1, 29) = 22.02, p < .01, η2 = .43, and F(1, 29) = 5.48, p < .05, η2 = .16, which, respectively, revealed an IOR effect and an overall latency reduction when the return probability was high. Most important, the interaction between target location and return probability was significant, F(1, 29) = 21.39, p < .01, η2 = .43.

Figure 5

Inhibition of return (IOR) effects under low and high return probability conditions in Experiment 3. A. Mean of the median latencies of saccades to return and new locations under low and high return probability conditions. B. IOR over the course of the four experimental blocks. All error bars are within-subject standard errors of the mean.

Without the periodic structure present in Experiments 1 and 2, we were not able to perform the binned analysis of IOR shown in Figures 2B and 4B. However, we did assess the magnitude of IOR under the two contingencies on a blockwise basis. Figure 5B shows the evolution of IOR across the four experimental blocks (based on the 28/30 participants who contributed to all eight cells of this analysis). The low and high return probability conditions were already well separated in their IOR effect by the second experimental block. The main effect of return probability was significant, F(1, 27) = 13.51, p < .01, η2 = .33, and a small interaction effect approached significance, F(1, 27) = 2.31, p = .08, η2 = .08. These results show that participants can acquire the association between context and environmental statistics relatively rapidly (i.e., after one or two blocks of exposure, if the practice block is included as relevant exposure), with some slight suggestion of further changes with more extended practice.

Experiment 3 was designed to prevent the contribution of a context-blind local tracking mechanism to adaptation to the different contingencies. Nevertheless, it is possible that such a mechanism was still in operation, as was found in Experiment 2. Figure 3 (diamonds) displays the results of the sequential effects analysis for Experiment 3, alongside the conditionalized latencies from the other two experiments. The visual similarity in the pattern of sequential effects was confirmed statistically (based on the 28/30 participants who contributed to all data points). Once again, there were main effects of target location and 1-back similarity, F(1, 27) = 11.03, p < .01, η2 = .29, and F(1, 27) = 6.88, p < .05, η2 = .20, respectively, with an advantage for a repeated trial type (return or new). As in Experiments 1 and 2, there was no main effect of 2-back similarity, F(1, 27) = 0.00, p = .97, η2 = .00, and the interaction between 1-back and 2-back similarity was not significant, F(1, 27) = 0.22, p = .64, η2 = .01. Even under these conditions then, a local recency mechanism seemed to be operating, looking back to the immediately preceding trial only.

It is remarkable that participants were able to learn about the statistics of the two contexts (see Figure 5), considering that the contingency changes were very frequent in comparison with the previous experiments and that the temporal correlations in the input trial sequence were deliberately removed. These results provide clear evidence that participants formed genuine associations between the contextual cues and the different contingencies. A context-blind, local tracking mechanism is unlikely to have supported associative learning between context and statistics in the absence of temporal structure in the input sequences. The finding that there was nonetheless a dependence in saccade latency across successive trials may seem puzzling, but agrees with other demonstrations of sequential effects in uncorrelated environments (Bertelson, 1961; Cho et al., 2002; Kirby, 1972; Remington, 1969; Soetens et al., 1985).

Up to this stage, we have sought to characterize the contributions of local and contextual mechanisms empirically. We can now use these data to understand more precisely how the outputs of these mechanisms are utilized by the oculomotor machinery responsible for making decisions about when and where to move the eyes next. To this end, we need to incorporate local and contextual influences in an overall model of saccadic choice and latency.

An Evidence Accumulation Account of IOR and Its Adaptation to the Local Statistics

In our previous work (Farrell et al., 2010; Ludwig et al., 2009), we have successfully used an evidence integration framework to account for the inhibitory effect in both saccade latency distributions and in choice accuracy. Various sequential sampling and accumulator models form part of this general class of models (reviewed in Luce, 1986; Ratcliff & Smith, 2004; P. L. Smith & Ratcliff, 2004). They all assume that evidence in favor of discrete response alternatives is accrued over time, and a choice is made when the integrated evidence exceeds some criterion level. Variability in choice and latency is accounted for by assuming one or, typically, more sources of variability, either within or between trials.

As a particular instantiation of this general class, we (Farrell et al., 2010; Ludwig et al., 2009) have adopted the linear ballistic accumulator model (LBA; Brown & Heathcote, 2008) to account for IOR and its variation with the environmental statistics. The LBA is a conceptually and computationally simple model of decision making. It assumes that choice is governed by a race between deterministic accumulators, with variability in choice and latency following from variability in the rate and starting point of accumulation (this variability applying separately and independently to each accumulator). Each accumulator represents one response alternative and the one that crosses a decision criterion first determines choice. The response latency is a combination of the decision time (i.e., the time it takes for an accumulator's activation to exceed the criterion) and a nondecisional component reflecting peripheral sensory and motor processes.

In previous fits of the standard LBA model to empirical data, we have shown that the basic IOR effect is best accounted for as a slowing in the (mean) rate of accumulation associated with the return location (Farrell et al., 2010; Ludwig et al., 2009). Critically, the variation in IOR with environmental statistics is not expressed in the accumulation rate, but is entirely accounted for by a change in the amount of evidence that needs to be accumulated for a decision to be made (Farrell et al., 2010). This effect is consistent with the psychological interpretation of the starting point as an index of the prior belief that a particular response will be called for. That is, starting points are higher for accumulators that represent decision alternatives with a greater (subjective) prior probability (Basso & Wurtz, 1998; Carpenter & Williams, 1995; Churchland, Kiani, & Shadlen, 2008). When the return probability is high, the baseline of the return accumulator is moved closer to criterion; when the return probability is low, the baseline is moved further away from criterion. Given the results of the present set of experiments, the critical question is how the starting point moves toward or away from criterion as a result of exposure to different statistical contingencies. In particular, we ask by what mechanism the starting point tracks beliefs about which response alternatives are likely to become relevant.

There are various candidate mechanisms that could be implemented to update the subjective beliefs encoded by the starting points of accumulation: for example, stored exemplar representations (Logan, 1988, 2002; Nosofsky, 1986; Speekenbrink & Shanks, 2010), iterative Bayesian belief updating (Speekenbrink & Shanks, 2010; Wilder et al., 2009; Yu & Cohen, 2008), and sequential Monte Carlo integration (Brown & Steyvers, 2009). One major aim of the modeling presented here is to explore what the critical ingredients are for any model to capture our results. For the purpose of demonstrating the overall viability of our proposal, a very simple form of associative learning provides an effective and parsimonious account of the complete pattern of data across all three experiments reported in this article.

A Model of Belief Updating

Basics of the model

In this section, we specify a “front end” to the accumulation of evidence to threshold. This front end is a simple belief updating model that specifies how the starting points of accumulation vary from trial to trial. Given the starting-point adjustments produced by the model, the saccadic decision is modeled as a standard race to threshold, with IOR built-in as an attenuation of the mean accumulation rate of the accumulator representing the return location(Farrell et al., 2010; Ludwig et al., 2009). This attenuation remains constant and unaffected by the changes in the statistical contingencies. Without it, no IOR would be predicted. Any change in the magnitude of IOR is solely mediated by the changes in the starting points.

Like the experimental data presented here, the model focus is restricted to the second saccade of a sequence. The updating of the starting points is assumed to occur sometime after the offset of the first saccade of the sequence and before the presentation of the second movement cue. At that point in time, there are three response alternatives, and the participant does not yet know which one will be cued next. However, from previous experience and the context of the current trial, he or she will have some expectation of having to go back to the previously fixated location or having to move on to a new location.

In our experiments, the response alternatives, in terms of the specific vectors involved, and their mapping onto response types (return vs. new) change from one trial to the next. The different accumulators may be regarded as coding these more abstract response types. Alternatively, they may be tied more closely to the oculomotor machinery and regarded as specific movement vectors that have been assigned either return or new status. This assignment will need to occur on every trial, presumably after completing the first saccade of the sequence. For the present purposes, we need not commit to one view or the other.

Model specification

The starting point associated with the return location is denoted R. The starting points of the accumulators representing the new locations are N1 = N2 = N; they are treated identically because from the model's perspective, the two locations are not differentiated. We define a vector, y, to indicate the nature of a given trial: y = {1, 0} and y = {0, 1} for return and new sequence types, respectively. We also define a context vector: x = {1, 0} for line cues and x = {0, 1} for arrowhead cues.

The context vector, x, is associated with the different trial types, y, through a set of weights, wij, where i = 1, 2 indexes the context variable and j = 1, 2 indexes the different types of sequences. Standard learning rules may be now used to update these weights from one trial to the next. We used a simple learning rule (Sutton & Barto, 1981) with weight decay: where n indexes trial. The left-hand section of Figure 6—labeled weight update—illustrates this simple associative network. The figure shows an example in which the context of the trial just completed was an arrowhead cue, and the sequence called for a return movement: in other words: x2 = 1 and y1 = 1. Equation 1 states that only the weight connecting arrowheads with return movements, w21, will be strengthened. None of the other weights are strengthened because the product xiyj will be 0 for these weights. The extent of facilitation is encoded by the learning rate, cj, which depends on the nature of the preceding trial. That is, facilitation of a return movement can only be attributed to one accumulator, whereas facilitation of movements to new locations need to be distributed across two locations. For convenience, we simply set c2 = c1/2.

Figure 6

Schematic of a belief updating model. Events in the previous trial (the context and whether the cue directed a saccade toward a return or new location) are used to adjust the association between context and trial types (weight update). The weights are used to adjust the starting points of three accumulators representing the three response alternatives on the current trial (belief update), given the current context. Once the starting points are set and the second movement cue is presented, evidence in favor of the alternative responses, based on the stimulus directing the current saccade, is accumulated toward a decision criterion (evidence integration).

Although only one weight will be strengthened at a time, all weights are updated according to Equation 1. As such, they are all subject to a certain amount of decay, which is controlled by parameter α. This parameter is constrained to lie between 0 and 1, which will determine the time constant of the system's memory. When α = 1, there is no forgetting: any facilitation is simply added to the previous weight value, and all other weights retain their previous value. When α = 0, there is no memory at all, and weights will simply fluctuate between 0 and cj solely on the basis of the immediately preceding trial. Intermediate values of α make the temporal evolution of the weights “leaky” and prevent the weights from growing without bound, while still retaining some memory of the previous trial history.

Although we have predefined different context representations, it is clear that these are probably created online as the system is exposed to different contexts. For instance, in Experiment 1, there is only ever one context: using the mapping defined previously, x(n) = {1, 0} for all n. As a result, the weights projecting from unit x2 are never updated by Equation 1, that is, w2. (n) = 0 for all n. In this situation, the second context essentially does not exist, and there is no internal state representing this context (Gershman et al., 2010; Redish et al., 2007). In Experiments 2 and 3, the second context unit and the weights projecting from that unit may be created as soon as, or shortly after, a new context is encountered. The creation of a new context unit and associated weights corresponds to the process of state splitting. We have not formally implemented this process but envisage it as occurring along the lines put forward by Redish et al. (2007): a new state is created as soon as the cues that make up the current situation cannot be matched to any of the existing internal states.

Once the first saccade on the current trial has been completed, the model is in a position to adjust the starting points of the three accumulators for the second saccade based on the current context: and for the return accumulator and two new accumulators, respectively. Note that the influence from the previous trial is mediated by the current values of the weights.

The middle section of Figure 6—labeled belief update—illustrates how the weight update carried over from the previous trial impacts on the starting points of the three accumulators on the current trial. In the example shown, the context (arrow cue) remains constant across the two trials. Based on the presentation of the arrow cue on trial n, the return starting point is strengthened, whereas the remaining starting points are subject only to passive decay (shown as a reduction in contrast in the figure).

The right-hand portion of Figure 6—labeled evidence integration—shows how the updated starting points feed through into the temporal dynamics of the race to threshold. The adjustment of the starting point for the accumulator representing the return location is illustrated by the upward arrow on the left of the panel. Assuming a return target is presented on the current trial, the return accumulator wins the race easily. Clearly, the more such trials occur in close succession, the shorter the latency of return movements will be, thereby reducing the magnitude of IOR when the return probability is high.

One important point to make is that the model for belief updating does not contain separate mechanisms for local tracking and contextual learning, despite our initially separate treatment of these mechanisms. The weights carry both sources of information, with context-gated learning reflected in the second term of Equation 1, and an additional dependence on the previous trial through the weight decay component. In other words, tracking the statistics of the environment is inherently local and contextual. As will be shown next, this feature of our model allows it to capture the data from all three experiments reported here, despite the simplicity of the updating rule.

Simulations

Having formalized a model for the updating of subjective prior probabilities of alternative responses, we set out to determine whether the model could account for the qualitative patterns of data obtained in our experiments. We therefore simulated the model, coupled with the evidence accumulation assumptions of the LBA model (Brown & Heathcote, 2008; Farrell et al., 2010; Ludwig et al., 2009).

In these simulations, we aimed for a satisfactory qualitative fit, rather than a precise quantitative fit on the data from individual participants. These simulations represent a demonstration of sufficiency. In doing so, they provide a compelling demonstration of how a relatively simple updating scheme can give rise to what appears to be quite sophisticated behavior, namely, adaptation to locally varying temporal statistics, gated by surface features of the context.

In the simplest form described here—assuming initial weights of 0 and setting c2 = c1/2—the updating model is characterized by just two free parameters: weight decay, α, and the learning rate associated with the return location, c1. The simulations reported were performed with a common set of parameters, chosen to give a satisfactory qualitative fit to our data from all three experiments: α = 0.72 and c1 = 0.1.

In order to generate (mean) latencies, and in particular produce IOR, the complete system also requires specification of the parameters of the accumulation process, as implemented by the LBA model. The following LBA parameters required specification: mean accumulation rates for target and nontarget locations, attenuation in the mean accumulation rate for return locations, variability in accumulation rate, and the nondecisional delay. Following Ludwig et al. (2009) and Farrell et al. (2010), IOR was implemented by attenuating the mean accumulation rate associated with a return location via a multiplicative constant. In contrast to standard implementations of the LBA (particularly those in Ludwig et al., 2009 and Farrell et al., 2010), we did not include variability in the starting point, precisely because this is what the updating model was designed to account for. The decision criterion, used as a scaling parameter, was set to 1 (Farrell et al., 2010; Ludwig et al., 2009).

These accumulation parameters bring the total number of parameters up to seven for the complete system. However, in the simulations reported below, the LBA parameters were not free but were obtained by averaging parameter estimates across observers in the baseline—equal return probability—condition of Farrell et al. (2010). These five values are listed in the caption of Figure 7, along with the detailed simulation procedure.

Figure 7

Simulation results. A. Temporal evolution of inhibition of return (IOR) after an unmarked switch from one contingency to the other (constant context). B. Same as A, but the contingency is signaled by a contextual cue. C. Mean simulated latency under randomly intermixed contingencies. D. Sequential effects analysis examining simulated latency contingent upon the preceding two trials, analogous to Figure 3. The free parameters of these simulations are given in the text. The parameters of the accumulation process are fixed to the mean parameter estimates from the baseline condition of Farrell et al. (2010). These are mean target accumulation rate (vT = 0.0057), mean nontarget accumulation rate (vD = 0.0013), standard deviation around the mean accumulation rate (σv = 0.0013), inhibitory rate attenuation for return locations (m = 0.90), nondecision time (Ter = 115.43), and θ = 1 (decision criterion). Each simulation involved 72,000 trials. On each trial, accumulation rates were drawn from Gaussian distributions. The means of these distributions were determined by whether the accumulator coded a target (vT) or nontarget (vd) location and was then further adjusted by the multiplicative constant representing IOR if that location corresponded to a return location (m). The standard deviation of the distributions was kept the same (σv). With the starting points and the accumulation rates in place for a given trial, the outcome of the race is known: (θ − b)/a specifies the time to reach threshold for an accumulator, with θ = 1 as the decision criterion, b the starting point of an accumulator, and a its accumulation rate. The saccade latency is the time to threshold of the winning accumulator, plus the additional nondecisional delay (Ter).

Panels A–C in Figure 7 show the model's predictions for Experiments 1–3. We examine each experiment in turn. As explained before, in Experiment 1, there is only one context, and only the weights projecting from that context are updated. As the context does not change, every time a return movement is required, the return accumulator is facilitated on the subsequent trial. Likewise, every time a movement to a new location is required, the two accumulators representing these new locations are boosted for the subsequent trial. Under these circumstances, the model behaves in accordance with the way the local mechanism was characterized informally at the start of this article.

For the simulated data, we computed the temporal evolution of IOR across a series of 18 trials in any one contingency, in exactly the same way as we did for our empirical data shown in Figures 2B. In line with the informal description of a local mechanism presented earlier, the initially small IOR effect grows rapidly when switching from a high to low return probability, and a large IOR effect decreases rapidly when switching in the opposite direction (Panel A).

Figure 7B shows a simulation of Experiment 2 when there is more than one context, and both the context and the return probability switch every 18 trials. The simulated data pattern now shows much less of a temporal dependency in that the effect of contingency on mean IOR is present very early on after a switch, as was found in Experiment 2 (see Figure 4). Because the parameters of this simulation were identical to those used in the generation of Panel A, the difference between the two panels is entirely due to the contextual variation that was present in the experiment and the simulation.

Figure 7C shows the simulation results when both the context and return probability vary from trial to trial, but remain yoked, mirroring the design of Experiment 3. As in the experimental data (see Figure 5), the model produces the observed interaction between return probability and trial type (return vs. new), such that a larger IOR effect is found when the return probability is low (compared with the high return probability condition).

Finally, the data from all three simulations were subjected to the same sequential effects analysis performed on the empirical data. Figure 7D demonstrates a strong dependence on the preceding trials in all three simulated experiments, as was obtained empirically at least for the nonreturn trials (the empirical data for return sequences are much more noisy, as pointed out in the discussion of the sequential effects of Experiment 1). This finding is important and may be related to a point made in relation to the sequential effects analyses shown in Figure 3. When considering the data alone, it seemed puzzling that even when a purely local mechanism could not mediate adaptation to intermixed contingencies, a strong dependency on the immediately preceding trial was nevertheless observed. The modeling suggests an explanation for why this should be and indeed generates a prediction that may be assessed against the data.

The model, as implemented here, assumes that the facilitation from the previous trial is context selective. The starting points are updated in a manner that is contingent on the current context. If the context is repeated, the facilitation of the response congruent with the previous sequence is carried through in the starting point of the accumulator representing that response. However, if the context is not repeated, the facilitation derived from the previous trial will be expressed in the starting point only when its context is presented again. By that time the weight under consideration will have decayed to some degree. As a result, the model predicts a stronger sequential effect from the previous trial when the context is repeated, compared with when the context switches. This is a prediction that can be tested with the data from Experiment 3.2

We thank Mike Mozer for pointing out this prediction of the model.

For the purpose of this test, we examined the 1-back effect because this was the only reliable sequential effect to emerge from Experiment 3. As this effect did not interact with the nature of the current sequence, we pooled across sequence type (i.e., whether the current trial was return vs. new) and simply conditionalized saccade latency on whether the preceding (n − 1) trial was of the same or different type as the current trial. The other conditional constraint—not included in the previous analyses—was whether the context remained the same or switched. The 1-back effect (different–same) was 24 ms (± 5 ms, standard error of the mean [SEM]) when the context remained the same, but only 8 ms (± 3 ms, SEM) when the context switched. The model predictions for these two contrasts came to 17ms and 4 ms, respectively (recall that no attempt was made to obtain a precise quantitative fit to these data).

To assess whether this difference was reliable, we performed a 2 × 2 ANOVA on the observed median saccade latencies, with 1-back similarity and context switch as the two repeated measures factors. All 30 participants contributed to this analysis. There was a main effect of 1-back similarity, which is essentially the same as that reported in the original sequential effects analysis, F(1, 29) = 20.22, p < .001, η2 = .41. There was no main effect of context switch, F(1, 29) = 0.43, p = .52, η2 = .02. Most important, the interaction between context switch and 1-back similarity was significant, F(1, 29) = 15.21, p < .001, η2 = .34. The prediction of the model of a stronger 1-back effect when the context remains the same is clearly supported by the empirical data.3

There is a corollary prediction that can be made if this logic is followed further. Given that the analysis just described shows that switching contexts dilutes the 1-back effect, and given that the frequency of the context switches was larger in Experiment 3 than Experiments 1 and 2, we would predict the magnitude of the sequential effect to decrease across the three experiments. This prediction is expressed in the slopes relating local sequence to mean latency in Figure 7D. The slopes of each pair of functions get shallower in going from Experiment 1 to Experiment 3. The same effect appears to be present in Figure 3. To assess this prediction, we conditionalized the data from all three experiments only on the nature of the previous trial (pooling over sequence type). We performed a mixed-factor ANOVA with 1-back similarity as the repeated measures factor and experiment as a three-level between-subjects factor. The critical interaction between 1-back similarity and experiment did not reach significance, F(1, 29) = 0.25, p = .78, η2 = .01. Therefore, there was no statistical evidence for the prediction that the 1-back effect should decrease as context switches become more frequent. Unfortunately, however, the power of detecting this interaction involving a between-subjects factor was very limited (the observed power with α = .05 was estimated as .09).

To conclude, our model of belief updating is inherently local and context-selective. If only ever one context is encountered, the model simply implements a form of 1-trial priming with memory decay (as reflected in weight decay). When a situation is classified as a new context, the effect of the previous trial is context dependent and only emerges in behavior if the matching context is presented before decay has undone the facilitation. With only two free parameters that were kept constant across simulations of all three experiments, this simple model provided a good account for (a) the overall patterns of mean saccade latency, (b) the temporal evolution of IOR, (c) the nature of the sequential effects, and (d) the context dependency of these sequential effects.

General Discussion

Adaptation to the statistical structure of the environment is clearly beneficial for any organism. Such adaptation will be more challenging under nonstationary conditions, that is, when the statistical structure varies (Speekenbrink & Shanks, 2010). Given the frequency of gaze shifts and their importance for visually guided behavior in general, a key question is how the underlying mechanisms involved can be flexibly tuned to the local environmental statistics. We have targeted oculomotor IOR in particular, because IOR itself may be seen as an assumption about the statistical structure of the world. However, our results should be seen in the broader context of the role of learning and context in adaptive visual-saccadic behavior.

In this article, we have shown that learning operates over a rapid time scale and in a context-selective manner to enable adaptation of oculomotor behavior to the statistical properties of the environment. Moreover, we have developed an extremely simple, yet powerful, model of how such context-gated adaptation could occur.

Adaptation in our experiments essentially involves learning the correlation between two successive movements. Our subjects were able to acquire this correlation even when any one statistical contingency was presented for a relatively short period of time (≤ 18 trials). This is all the more impressive because this correlation cannot be characterized in terms of specific spatiotopic locations or movement vectors but instead refers to a more abstract temporal relationship between successive events. Moreover, the expression of this knowledge has to take place on the fly, in a short time window (i.e., between the offset of the first saccade and the appearance of the second movement cue).

When the statistics of the environment changed, subjects learned to match the new contingency. Adaptation was particularly pronounced and rapid when a surface feature of the environment, a contextual cue, changed along with the statistics. Indeed, subjects were able to acquire the mapping between context and statistics and use it to guide their behavior even when different contingencies were randomly intermixed. As a result, they could retrieve a representation of the statistical structure of the environment when confronted with a particular contextual cue. The context did not directly dictate the required response, but rather influenced how an incoming stimulus was processed: that is, the context gated which correlation structure was brought to bear on current experience.

Our general proposal is that expectations about the current properties of the environment are encoded in the baseline activity of units that represent the evidence in favor of different decision alternatives (Farrell et al., 2010). These subjective beliefs about the structure of the environment are assumed to be continually updated in light of experience. The importance of our findings is that they provide compelling evidence that this updating operation is inherently local—driven by recency—and context-dependent. That is, the carryover from recent experience manifested in one-trial priming effects and sequential effects in general is modulated by the context. If the context changes, the influence of recent experience is much diminished. This would seem ecologically adaptive: when the cues that make up a particular context change, recent history may be a less reliable guide to what is likely to happen in the near future.

We have presented one particularly simple instantiation of a context-selective belief-updating mechanism. When coupled with standard assumptions about the dynamics of the visual-saccadic decision process, this model provides an excellent and parsimonious account of all major patterns in the data.

Belief Updating and Its Impact on Choice and Latency

There are two parts to the general theory outlined here. One is a mechanism of belief updating; the other is an evidence-integration mechanism. Belief updating sets the starting points of accumulation (Anderson & Carpenter, 2006; Cho et al., 2002). We opted for a simple associative learning network for the belief updating component and ballistic accumulation for the subsequent evidence integration process. We are not wedded to these specific mechanisms and alternatives could have been used, possibly with at least as much success. However, we believe both components are well justified from both theoretical and practical perspectives.

Evidence integration

With regard to the integration mechanism, the choice for a “race-to-threshold” model generally, and the LBA model (Brown & Heathcote, 2008) in particular, was motivated by the following considerations. First, accumulator and sequential sampling models have a long history of success in accounting for both choice probabilities and the latency distributions of correct and error decisions (among many others: Brown & Heathcote, 2005; LaBerge, 1962; Link & Heath, 1975; Luce, 1986; Ratcliff, 1978, 1988; P. L. Smith & Ratcliff, 2004; P. L. Smith & Vickers, 1988; Usher & McClelland, 2001; Vickers, 1970). Alternative methods for deriving the latency predictions in the present study would involve either arbitrarily scaling the expectations or perhaps mapping the expectations onto parameters of some descriptive latency distribution. Neither of these approaches would be theoretically principled or desirable, and both would involve more free parameters than we have adopted here.

Second, many of these models assume a certain amount of variability in the starting point of accumulation, which is typically considered a source of internal noise. The updating model presented in this article, as well as previous work by Cho et al. (2002) and Anderson and Carpenter (2006), instantiate an important and natural hypothesis about the origin of starting point variability: at least part of it reflects the trial-to-trial variability in subjects' expectations about which response alternatives are likely to be relevant based on the recent history of experiences.

Third, the idea of integrating evidence to a decision threshold has received strong neurophysiological support in the domain of visual-saccadic decision making. Recordings from a variety of neural circuits involved in eye-movement programming support the notion of accumulating evidence from the stimulus (i.e., likelihoods) from a baseline level of activity that indexes the prior probability of a movement alternative (for reviews, see Gold & Shadlen, 2001, 2007; P. L. Smith & Ratcliff, 2004; Schall, 2003).

Finally, the LBA model was used in our previous work to account for latency distributions and choice accuracy in our saccade-sequencing paradigm (Farrell et al., 2010; Ludwig et al., 2009). By placing the belief updating mechanism in the context of the same model, we can demonstrate the plausibility of the proposed front end by accounting for our previously reported finding that the environmental statistics uniquely affect the starting point of accumulation.

Belief updating

Moving on to the belief updating mechanism itself, our simple rule for updating associative weights is equivalent to exponential filtering models. Such models have been applied very successfully in accounting for adaptation in nonstationary reward environments (Corrado, Sugrue, Seung, & Newsome, 2005; Sugrue et al., 2004), sequential effects in two-choice reaction time tasks (Yu & Cohen, 2008), and attentional priming in visual search (Mozer, Shettel, & Vecera, 2006). Moreover, Yu and Cohen (2008; see also Wilder et al., 2009) have shown that, in the context of accounting for sequential effects in randomized paradigms, exponential filtering approximates the optimal, normative Bayesian solution for iteratively updating expectations on the basis of past experience, while being more biologically and psychologically realistic than the full-blown Bayesian solution.

To make the link with exponential filtering more explicit, for a given context i and unit representing sequence type j, there will be a history of inputs. This history forms a time series of instances of either zero or positive (0 or cj) amounts of facilitation, depending on the nature of the previous sequence and context. Now define a set of temporal weights, described by an exponential function of the form φ(k)=e−k⧸τ, where k denotes the number of trials in the past—that is, the current trial is k = 0 and gets a weighting of 1, and trial n − 1 gets a weighting of φ(1). Exponential filtering involves weighting the current and previous inputs with φ and then summing them. This weighted sum corresponds to the strength of association between context i and sequence type j, encoded by wij in our model.

This formulation in terms of an exponential moving window over past and current inputs is equivalent to our learning rule with weight decay, specified by Equation 1 (Sutton & Barto, 1981). Specifically, the decay parameter α relates to τ through τ = −ln(α)−1. In our simulations, we set α = .72. Using the exponential filtering formulation, this value corresponds to a time constant of around three trials. This limited memory demand is well within the realm of biological possibility (e.g., Sugrue et al. (2004) reported time constants of nine trials for their choice data in monkeys). Note that a time constant of approximately three trials does not necessarily imply that we should have obtained n-back effects beyond the immediately preceding trial. Apart from the issue of whether there is the statistical sensitivity to detect such longer reaching sequential effects, the weight decay formulation demonstrates that the influence of a trial n-back is absorbed by the trials that follow it. The time constant merely provides an estimate of how long it takes for the influence of any one trial to decay to a certain level: φ(3)≈0.37 (i.e., 37% of the maximum weight).

Several other forms of belief updating could have been implemented. We have already briefly mentioned the optimal Bayesian approach, which is particularly well suited to the iterative nature of the belief updating mechanism (prior expectations are combined with current observation to set the expectation for the subsequent trial or episode). Another possible candidate is an exemplar-based scheme (Logan, 1988, 2002; Nosofsky, 1986; Speekenbrink & Shanks, 2010), with each combination of context and sequence type laying down a distinct memory trace. Upon encountering a specific context, subjects may retrieve a subset of memory traces on the basis of similarity of the traces to the current context. Expectations may then be set in accordance with the frequency of return and nonreturn traces in the subset. Finally, a class of very promising methods is based on sequential Monte Carlo integration or particle-filtering models (Brown & Steyvers, 2009; Sanborn, Griffiths, & Navarro, 2006). These models offer a more psychologically realistic approximation to the normative Bayesian solution. Discrete particles correspond to possible “world states” (here: combinations of the two contextual cues and the two different sequence types). Every time one state is experienced, particles corresponding to that state are propagated while other particles are replaced. The distribution of particles across the different states then approximates the probability distribution of these distinct states, which may be translated into subjective expectations. The number of particles available determines the memory (i.e., time constant) of the system: with a psychologically plausible, limited set of particles, their distribution will be most heavily influenced by recent experience.

The important issue is why these different schemes can account for the way that expectations are updated with experience. The reason is that, like our model, they all have the capacity to track a variable that closely reflects the frequency of the two different sequences (return, new), in a context-selective manner. In addition, this variable may be computed over some limited time window that is weighted toward more recent events. Finally, the models allow for some flexible adjustment in the size of this time window, so that the extent of past discounting may be adapted to the volatility of the environment. These are the critical ingredients to enable context-gated adaptation in a nonstationary world.

Adaptation Cued by Context Alone

When changes to the structure of the environment are coupled with changes in some of the (surface) features of that environment, the context may act to retrieve a representation of what kind of events are likely to occur in the “new” environment. This description of the contextual mechanism appears to imply that assumptions about the statistical structure of the world should be relatively independent on recent experience acquired under a different context. Indeed, for the contextual mechanism to have any ecological utility, it would seem desirable to be able to set expectations on the basis of the context alone, regardless of the immediately preceding history.

However, our model relies on the continual updating of the association between contextual cues and the learned correlation between two eye movements in a sequence. Contextual weights necessarily decay when not topped up with trials under the context they represent (see Equation 1). Suppose the context switches. As the number of intervening trials grows, the subject will be less able to appropriately set the starting points when switched back to the previous context. Indeed, this is the reason why the simulation of Experiment 2 (Figure 7B) still shows some gradual build-up and decrease in the magnitude of IOR after a context and contingency switch. This reliance on recency in setting the contextual weights drives the sequential effects in the model (and, we argue, in our participants).

It is therefore reasonable to ask whether we have in fact implemented the contextual mechanism as conceptualized at the outset. That is, can our model adjust expectations on the basis of contextual signals alone, even after a paucity of experience in that context?

This capacity is easily incorporated in the model. It may be that subjects acquire the relevant associations by continually updating the contextual weights, but over time come to treat these weights increasingly as fixed. In our formulation, such a change could be instantiated by letting α grow to 1 and cj decrease to 0 in Equation 1. As a result, subjects may immediately retrieve the appropriate representation of the likelihood of refixation, given a certain contextual cue. In this state, the system would be much more robust to intervening trials from different contexts and contingencies.

The model makes a clear prediction of the empirical signature of such a transition: robust and more immediate adaptation with a much smaller dependency on the recent trial history. We believe the data sets presented here are not sufficiently diagnostic to really allow us to detect such a transition, for instance, through a blockwise sequential effects analysis. The assumption of continual updating (and decay inherent in the updating operation) appears to work for the empirical data reported here, so that additional assumptions about changes in the parameters controlling belief updating did not seem warranted.

Context-Based Adaptation in Natural Environments

An important feature of the modeling is that it addresses the way in which memory for the statistics in the world can guide future behavior. One natural question is then to what extent the described contextual association mechanism is operative in more complex, naturalistic environments. In our experiments, there were only two contingencies, and their associated contextual cues were the only features of the test displays that varied. It is clear that under these rather restricted conditions, the statistics of two environments can be learned in parallel effectively. However, in naturalistic settings, it is not entirely clear what counts as a context.

It is likely that humans will have representations of the statistical regularities that operate in different environments (e.g., office environment, street scene, and so on; Tversky & Hemenway, 1983). These representations may be exemplar-based, with the relevant representation retrieved on the basis of similarity matching (Nosofsky, 1986; Speekenbrink & Shanks, 2010). The capacity to store and retrieve such representations presumably will be subject to the same constraints that limit storage in and retrieval from long-term memory in general (Shiffrin & Atkinson, 1969). Moreover, which representation is retrieved may be subject to similar constraints as categorization in general (Rosch, 1978). For example, if a precise match to a current environment is unavailable, the representation from a superordinate category may be used. Alternatively, a new category may be created (Love, Medin, & Gureckis, 2004), a process described by Redish et al. (2007) as state splitting.

A related issue is what triggers an episode of tracking the relation between a contextual cue and behaviorally relevant variable. One possible mechanism is prediction error feedback (Redish et al., 2007). When presented with a certain context and having “chosen” what variable(s) to predict, behavior will be governed by expectations retrieved from memory. As long as those expectations are matched with a frequency that is appropriate given the natural uncertainty in an environment, there is no need to alter the associative weights any further. However, if expectations are violated more frequently, a new episode of learning may be triggered (see also Love et al., 2004).

The learning of relatively arbitrary contextual associations, as demonstrated in our experiments, suggests that there will be considerable flexibility in what counts as a separate context or situation. In addition, it is likely that there also will be flexibility in the choice of what behaviorally relevant variable to predict in a given context. This choice in itself will be governed by the task demands. In a relatively simple saccade-sequencing task such as the one used here, any information that is predictive of the upcoming saccade may be used to perform the task more efficiently. Presumably, this is why the distinction between return and new locations mattered to our subjects. However, that is not to say that the likelihood of refixation will always be an important variable for subjects try to track and predict (Dodd et al., 2009; T. J. Smith & Henderson, 2009). Had we varied the likelihood of, say, second saccade targets being on the left-hand side of our display configuration, it is likely that subjects would have learned and used this statistical regularity to their advantage (Geng & Behrmann, 2002; Walthew & Gilchrist, 2006). Indeed, it is likely that several variables can and will be tracked simultaneously, in order to adjust behavior to the demands of the temporally and spatially local environment and task. A challenge for the future is to identify what characteristics of the trial sequence and contextual features are used by subjects to update their expectations and the generality of the learning mechanisms involved.