Outcome probability modulates anticipatory behavior to signals that are equally reliable.

A stimulus is a reliable signal of an outcome when the probability that the outcome occurs in its presence is different from in its absence. Reliable signals of important outcomes are responsible for triggering critical anticipatory or preparatory behavior, which is any form of behavior that prepares the organism to receive a biologically significant event. Previous research has shown that humans and other animals prepare more for outcomes that occur in the presence of highly reliable (i.e., highly contingent) signals, that is, those for which that difference is larger. However, it seems reasonable to expect that, all other things being equal, the probability with which the outcome follows the signal should also affect preparatory behavior. In the present experiment with humans, we used two signals. They were differentially followed by the outcome, but they were equally (and relatively weakly) reliable. The dependent variable was preparatory behavior in a Martians video game. Participants prepared more for the outcome (a Martians' invasion) when the outcome was most probable. These results indicate that the probability of the outcome can bias preparatory behavior to occur with different intensities despite identical outcome signaling.

1 Introduction

Learning to predict important events is critical for the survival of any animal species. Not surprisingly, organisms have evolved the ability to detect signals in the environment that may help them in this endeavor. However, very few of those critical events that organisms need to predict are clearly signaled by reliable and infallible signals. Most signal–outcome relationships are probabilistic and organisms are constantly surrounded by a myriad of stimuli that may or may not signal anything. To complicate things even further, many of those potential signals are only weakly to moderately reliable. Thus, one of the most serious challenges organisms face in their daily life consists of accurately estimating the reliability of the countless stimuli around them, or in other words, the predictive value of those signals. If they succeed, they will prepare for important events in the presence of reliable signals and not in the presence of confusing ones, which would be a tremendous waste of energy and resources. Their probability of survival crucially depends on this ability.

Reliable signals of important outcomes are responsible for triggering critical anticipatory or preparatory behavior, which is any form of behavior that prepares the organism to receive a biologically significant event. The salivation that occurs in anticipation of food, documented in Pavlov’s (1927) experiments, is a good example of this kind of behavior. Another example of anticipatory behavior is Garcia’s food-aversion learning (Garcia & Koelling, 1966), which produces anticipatory nausea in response to flavors that signal that a meal might be poisonous and in this way prevents the organism from ingesting it. Importantly, anticipatory behavior may include different response systems and may refer to either appetitive or aversive events. Moreover, it may involve different degrees of intentionality. Anticipatory behavior may consist of automatic Pavlovian responses, such as salivation or nausea, or it can be intentional and instrumental behavior, such as pressing a lever to avoid foot-shock (e.g., Brogden, Lipman, & Culler, 1938; Sidman, 1953).

Ideally, anticipatory behavior should only be triggered by stimuli that are reliable signals of significant outcomes. If organisms lacked the ability to tell reliable from unreliable signals, they would have to be constantly preparing for the imminent occurrence of potentially critical events. The cost for such strategy, in terms of energy and stress, would make it unfeasible. Alternatively, organisms might have adopted the strategy of not preparing at all in the absence of perfectly reliable signals. This would certainly have been lethal as well. Thus, erring from time to time in predicting the important events and preparing for the wrong ones might be as lethal as not preparing at all. However, errors will often occur in the world of uncertain information in which we all live. Thus, natural selection must have biased this process of detecting and responding to reliable signals in one particular direction, that in which behavioral errors are less costly. From this, it follows that in addition to reliability triggering anticipatory behavior, several other variables must be affecting (and thus, biasing) this process. The present research will look at one of these variables.

Previous research has already dealt with the question of to what degree humans and other animals are sensitive to what has been called cue–outcome contingency, that is, the reliability with which a signal predicts the occurrence of an outcome. A widely accepted statistical measure of contingency is Δp (Allan, 1980; Jenkins & Ward, 1965; Rescorla, 1968), which is defined as the difference between the probability of the outcome (O) in the presence and the absence of a signal (S). That is, Δp=P(O|S)−P(O|¬S). If the outcome occurs with the same probability in the presence and in the absence of the signal, contingency is zero, and thus, the signal holds no predictive value.

Many studies with non-human animals have shown that contingency modulates the degree with which animals prepare for a given outcome in the presence of a signal (e.g., Hallam, Grahame, & Miller, 1992; Murphy & Baker, 2004; Rescorla, 1968). Animals prepare less when the outcome is signaled by a weakly reliable signal. Thus, among other factors that are also known to influence learning and behavior (such as saliency or attention), all current models of animal learning and behavior we are aware of assume that anticipatory behavior should be guided by contingency (e.g., Stout & Miller, 2007; Rescorla, 1968; Rescorla & Wagner, 1972; Van Hamme & Wasserman, 1994).

The potential influence of signal–outcome contingency has also been central in the study of causal, predictive and contingency learning with humans. In this area, sensitivity to signal–outcome contingency has often been reported when using dependent variables such as subjective estimations of causality, of contingency, or of predictive value (e.g., De Houwer, Vandorpe, & Beckers, 2007; Vadillo, Miller, & Matute, 2005; Vadillo & Matute, 2007; Wasserman, Kao, Van Hamme, Katagari, & Young, 1996). However, systematic biases of these processes have often been reported as well. A typical experiment in this literature may present participants with a series of fictitious patients, one per trial. The fictitious patients are sometimes taking a pill (S) which may or may not be followed by relief from back-pain (O). One group of participants observes a condition in which the probability of recovery from back-pain is 80% regardless of whether the fictitious patient has or has not taken the pill, that is, P(O|S)=P(O|¬S)=.80; Δp=0. Another group of participants receives a condition in which the probability of the outcome (i.e., recovery) is 20% regardless of whether the patient has or has not taken the pill, that is P(O|S)=P(O|¬S)=.20. Thus, contingency is zero in both groups but the probability of the outcome differs. What has been observed in these cases is that the higher the probability of the outcome, the higher the degree of contingency that participants estimate between the signal and the outcome. That is, if contingency is held constant, judgments of contingency and of causal relationships increase as P(O|S) increases. This is known as the outcome–density bias and has been reported in a wide variety of procedures, conditions, and laboratories (e.g., Allan & Jenkins, 1983; Allan, Siegel, & Tangen, 2005; Alloy & Abramson, 1979; Blanco, Matute, & Vadillo, 2013; Buehner, Cheng, & Clifford, 2003; Matute, 1995; Msetfi, Murphy, Simpson, & Kornbrot, 2005; Musca, Vadillo, Blanco, & Matute, 2010; Shanks, López, Darby, & Dickinson, 1996; Wasserman et al., 1996).

Thus, human experiments have shown that contingency judgments are often biased by P(O|S). However, it is unclear to what extent the causal and contingency judgments collected in these latter experiments are comparable with the preparatory responses measured in animal conditioning experiments. Quite interestingly, there are some studies in which human participants are not asked to give a judgment of contingency or a judgment about the predictive value of the signal. Instead, they are simply asked to predict the outcome or to estimate how important would it be to prepare for the outcome when the signal is present. These tasks are more similar to the ones faced by animals in conditioning experiments, where animals do not have to judge the signal–outcome contingency, but just prepare for the incoming outcomes. The results of these human experiments suggest that, if anything, the influence of P(O|S) on these judgments of prediction and of importance of preparation is even greater than on judgments of contingency and causality (De Houwer et al., 2007; Vadillo & Matute, 2007; Vadillo et al., 2005). Nevertheless, it is well known that what people say that they would do is often very different from what they really do, and there have been numerous demonstrations of dissociations between judgments and behavior (Perales, Catena, Shanks, & González, 2005; Perruchet, Cleeremans, & Destrebecqz, 2006). Thus, it might still be the case that, as predicted by learning theories, preparatory behavior in humans as in other animals is actually guided by contingency.

The only experiment that, as far as we know, has been conducted with humans using anticipatory behavior per se as the dependent variable was conducted by Blanco, Matute, and Vadillo (2010). They reported that anticipatory behavior in humans was driven by contingency. That study used the Martians paradigm (Arcediano, Ortega, & Matute, 1996; see also Costa & Boakes, 2011; Franssen, Clarysse, Beckers, van Vooren, & Baeyens, 2010; Matute, Vadillo, & Bárcena, 2007). It is a simple videogame in which the task of the participants is to prevent Martians from landing the Earth. There are various cues that might signal a Martian’s invasion (the outcome) or nothing. Thus, this task assesses how people learn to predict the invasions and how they prepare for them. In other words, it assesses anticipatory behavior directly rather than asking participants provide a judgment. The results of Blanco et al. showed that participants actually prepared more for the outcome in the presence of a reliable than unreliable signals. However, because the probability of the outcome was held constant in that experiment, it provides no hints on how this variable might have biased behavior. In the current experiment, we will explore whether the probability of the outcome may bias non-verbal anticipatory behavior when contingency is kept constant.

2 Method

2.1 Participants and ethical statement

Thirty-six students from Deusto University volunteered to take part in the experiment and were paid €5 for their collaboration. Their mean age was 22.22 years old (SD=1.514). They were run in individual cubicles containing a chair, a desk and a computer.

2.2 Procedure and design

Following Blanco et al.’s (2010) experiment we used the Martians task (Arcediano et al., 1996; Costa & Boakes, 2011; Franssen et al., 2010; Matute et al., 2007). This allows us to assess how P(O|S) affects the participants’ anticipatory behavior directly. In this task, the goal of the participant is to prevent Martian spaceships from landing the earth. Every 250 ms a Martian spaceship appears on the screen and participants can destroy it by pressing the spacebar, which causes a laser gun to fire. Participants usually develop a stable baseline behavior of approximately four responses per second. Once this behavior is established, the actual experiment begins. From this moment the Martians will occasionally activate an anti-laser shield, which appears as a series of light flashes accompanied by a distinctive sound for 500 ms. If the participant fires the laser gun during the activation of a shield, an invasion of Martian spaceships will follow. During an invasion, the laser gun becomes deactivated and the Martian spaceships appear at a very high rate (one spaceship every 50 ms). The activation of the shield is the outcome the participant has to prepare for. Given that the shield activation lasts for only 500 ms, there is no time to cancel ongoing behavior and prevent the invasion unless the activation of the shields is predicted and ongoing behavior is suppressed in anticipation of the shield’s activation. Thus, the suppression of ongoing behavior is our dependent variable. It indicates the degree to which anticipatory behavior for the outcome occurs in response to a given signal. In order to effectively anticipate the outcome, the participant can learn to make use of signals that appear on the screen that might predict the activation of the shield.

A within-participants design was used. The manipulation of different outcome probabilities was between two equally (and moderately) reliable signals of the outcome. Signals A and B were abstract visual symbols.[1] Participants were told that these symbols were communication signals intercepted between Martian spaceships. The two Martian signs differed in their shape and colors, and were counterbalanced in their role as Signal A and Signal B. They appeared on the screen for 1500 ms each time and could or could not be followed by the outcome (the activation of the shield). As explained below, the probability of the outcome given each signal was different, although the signal–outcome contingency was the same for both.

In line with earlier research (e.g., Alloy & Abramson, 1979; Msetfi et al., 2005; Vallée-Tourangeau, Baker, & Mercier, 1994), and because in this experiment the two conditions were manipulated within subjects, we used two different tones to mark the different trial types belonging to each condition. Note that this is particularly important in trials in which neither signal is present, which could be in principle computed for any of the two conditions, so they need to be clearly separated. A low and a high tone were counterbalanced in their role as Context 1 and Context 2. Therefore, there were two different communication signals, A and B, which always occurred in different contexts and were characterized by different background tones. There were a total of 80 training trials. Of these, 40 trials occurred in Context 1. They consisted of the presence or the absence of target Signal A, and comprised the low outcome probability condition. The other 40 trials occurred in Context 2 and consisted of the presence or the absence of Signal B, and comprised the high outcome condition. The tones lasted for 1500 ms, so whenever signals A and B were presented they occurred simultaneously with their respective contexts. For programming purposes, the 80 trials were divided in 10 blocks of eight trials each, so that all different trial types could be presented in random order within each block. The purpose of this programming strategy was to avoid concentration of a certain type of trial either at the initial or final part of the training sequence. The 10 blocks of trials were presented without interruption. The inter-trial intervals lasted between 6500 and 11,500 ms. During that time participants were pressing the space bar for Martians on a black (and silent) background.

Table 1 shows the number of trials for both conditions. As can be seen, the probability that the outcome followed the signal was .50 for Signal A and .90 for Signal B, which was the one for which we expected more preparatory behavior. Both signals had the same contingency with the outcome, with Δp held constant at .40. Thus, the two of them were equally (and moderately) reliable. It is important to note that the probability of the outcome cannot be manipulated without confounding the overall number of trials and the number of signal–outcome pairings in each condition. As can be seen in Table 1, in the present experiment we chose to keep the overall number of trials constant across conditions. This is standard practice in experiments that explore the effect of the probability of the outcome, among other things because it is known that outcome–probability effects are dramatically influenced by the length of the training phase (Shanks et al., 1996).

Table 1.

Number of trials of each type presented for each condition.

Context 1			Context 2
	O	noO		O	noO
A	10	10	B	18	2
noA	2	18	noB	10	10

Signals A and B were two different abstract visual symbols, counterbalanced. Signal A (low outcome condition) was always presented in Context 1. Signal B (high outcome condition) was always presented in Context 2. Context 1 and Context 2 were a low and a high tone, counterbalanced, which were used to mark the different conditions. O and noO represent the presence or absence of light flashes, which played the role of outcomes.

Given that the target behavior consists of suppressing ongoing behavior, a suppression ratio (SR) needs to be calculated so that the number of responses (i.e., in this case, presses of the space bar) that occur during the presentation of the signal can be compared to the base rate of responding (Arcediano et al., 1996). The dependent variable is thus computed by dividing the number of responses during the signal (Rsignal) by the number of responses during the signal plus the number of responses during an equivalent time period preceding the signal during the intertrial interval (Rpre): that is, SR=Rsignal/(Rsignal+Rpre). If this ratio has a value of 0.5, this means that the participant shows the same amount of bar-pressing behavior during and just before the signal presentation, and therefore he or she is not preparing for the outcome. A suppression ratio of 0, on the other hand, reflects that the participant completely suppresses bar-pressing behavior during the signal presentation, and thus he or she is strongly preparing for the outcome.

3 Results

Figure 1 shows the suppression ratios for Signals A and B and for Contexts 1 and 2 across training. Because trials at the end of training are the most likely to yield asymptotic results, we compared the mean suppression ratios of the last trial with each signal. Behavioral suppression was stronger when B, the signal that was associated to a greater probability of the outcome, was present, than when A, which was associated to a lower probability of the outcome, was present. A repeated measures t-test on the mean suppression ratios during these trials with signal (A vs. B) confirmed that, as expected, participants suppressed their behavior more for B (M=0.315, SD=0.078) than for A (M=0.356, SD=0.116), t(35)=2.503, p<.05, d=0.41. Therefore, although both signals were equally weak predictors of the outcome, the signal holding the highest P(O|S) influenced behavior by yielding more preparation for the outcome under that condition.

Figure 1.

Mean suppression ratios for Signals A and B, as well as for Contexts 1 and 2 when presented alone during the 10 blocks of eight trials in which the different trial types were presented. When a given signal (A or B) or context alone was presented more than once in the same block of trials the figure shows the average suppression ratio for that cue or context in that block.

Even though the use of trial markers, or contexts, is standard practice and should not be eliciting preparatory behavior (because of the way they are described in the instructions and the way they are used), it could be argued that they might not be as neutral as we intended. Indeed, it is possible they became the only informative cues in this experiment because the outcome occurred with greater frequency in Context 2 than in Context 1. Figure 1 shows that this was not the case. As expected, participants prepared for the outcome in response to signals but not in response to trial markers or contexts.

Nevertheless, and in order to make sure that the differences in responding between signals A and B were not due to any differences in responding to the trial markers (Context 1 vs. Context 2), we also analyzed the suppression ratios in the last trial in which the contexts were presented alone, that is, the latest trials in which Signal A and Signal B were absent. A repeated measures t-test on the mean suppression ratios during these context-alone trials revealed that suppression ratios did not differ depending on context (M=0.439, SD=0.061 for Context 1; M=0.427, SD=0.091 for Context 2); t(35)=0.704, p=.486, d=0.15. Thus, the observed differences cannot be attributed to differences in the contextual trial markers. Preparatory behavior in the absence of the signals was very week and did not differ between both contexts.

4 Discussion

Behavioral experiments with human (Blanco et al., 2010) and non-human animals (e.g., Hallam et al, 1992; Murphy & Baker, 2004; Rescorla, 1968) had already shown that preparatory behavior is influenced by the reliability of the signals or, in other words, by signal–outcome contingency. Those experiments had typically maintained P(O|S) constant, so that contingency between the signal and the outcome could be manipulated and its effect on behavior established.

By contrast, judgmental experiments with humans had often manipulated P(O|S) while holding contingency constant. In this case, the dependent variable had been subjective judgments of contingency rather than preparatory behavior. The results had shown that the higher the P(O|S), the higher the judgments. In addition, they had also shown that it was not only the subjective judgments of contingency that was affected by P(O|S). Experiments in which participants were asked to predict an outcome based on the presence of a signal showed that these prediction judgments are influenced mainly by the probability of the outcome given the signal, not by contingency (Vadillo & Matute, 2007; Vadillo et al., 2005). Moreover, when asked whether they would prepare for the outcome, participants judged that it was most important to prepare for an outcome when its occurrence was most probable (De Houwer et al., 2007). Thus, we predicted that if contingency was kept constant, preparatory behavior might be biased when the outcome’s frequency was manipulated. The results showed the predicted pattern. Although the two signals in the present experiment had the same contingency (i.e., Δp) with the outcome, participants prepared more for the outcome when it was most likely to occur.

In the context of human and animal research, the effects of P(O|S) on causal and contingency judgments have usually been considered a bias. If the participants’ goal is to detect whether two events are covariate, their base rate should be irrelevant. Therefore, any demonstration that the base rate probability of the outcome matters in those judgments can be considered a bias. However, the situation might be the opposite for experiments using outcome predictions, preparatory judgments and anticipatory responses as dependent variables. If an outcome is very likely to happen, it makes sense to predict it very strongly, to prepare for it and to adapt our behavior consequently, regardless of the reliability of the cues that signal it. In other words, it is not obvious that the effect of P(O|S) on predictive and preparatory judgments, and on anticipatory behavior, should be considered a bias.

This does not necessarily mean that the effect of contingency on anticipatory behavior should be considered a mistake or a bias. Indeed, the question of what is a bias is a hard question that might not have a simple response. It all depends on how we look at it and what aspects we pay attention to. Preparing for an outcome only in the presence of highly reliable signals may make sense because it is a waste of energy to prepare for outcomes when they do not occur, and preparing in the face of misleading and ambiguous signals would certainly lead to many cases in which the organism prepares in vain. However, many signals hold weak or just moderately reliable relationships with the important outcomes we need to predict for survival. Thus, there is no doubt we will often err. We may err in our estimations and in preparing for outcomes when they will not actually occur; we will err in not preparing for critical outcomes that will occur; and given that there will be many errors and some may be lethal, evolution must have favored the least costly error. On balance, not preparing for a predator, or for a prey, is certainly more costly than preparing in excess. Thus, any variable that, when in doubt, makes the organism prepare in excess, has probably developed the power to bias both perception and behavior toward this direction of overestimation of contingencies and over-preparing for the outcome.

Of course, it is not without problems to state that an evolutionary framework can predict the results observed in the present research. In a sense, anything to happen should be a consequence of evolution. As such, and according to some, evolution cannot be an explanation, because it could serve to account for whatever result is observed (Gould & Lewontin, 1979; Plotkin, 2004). However, we find that sometimes it is useful to think in generic evolutionary terms in addition to thinking on more focused predictions and on previous data, which might help discriminate among several more specific hypotheses. We believe the present line of research is one of those cases that should benefit from such combined approach.

In any case, and aside from the necessary speculative nature of the discussion on whether it constitutes a bias or an efficient strategy, our results are important in that they show that non-verbal preparatory behavior in humans does deviate from contingency, at least under certain conditions. As we already noted, a shared principle among theorists of learning has been that behavior is strongly guided by contingency. Thus, it has been typically assumed by most textbooks (though often implicitly) that the probability with which the outcome occurs should not significantly affect behavior (e.g., Bouton, 2007; Domjan, 2003). The main contribution of our experiment has been to uncover the effect of the probability of the outcome on human preparatory behavior. We can state that not only contingency but also the probability of the outcome has an effect on preparatory behavior, at least in humans. It seems that, when preparing for an outcome, people, and possibly other animals as well, tend to allocate resources as a function of both, the reliability of the signals (i.e., contingency) and the probability that the outcome will follow. This probably minimizes the potential errors that can take place in a world of high uncertainty, or at least biases behavior toward the least costly errors. That is, assuming two signals of similar reliability, it makes sense to prepare more in the presence of the signal that is most frequently followed by the outcome.

There are some results in the literature that seem contrary to the present ones, at least at first glance. Those studies suggest that learning is faster when P(O|S) is low than when P(O|S) is high, along the same level of contingency (Δp). Wasserman, Elek, Chatlosh, and Baker (1993) explored different levels of contingency in a causal judgment task with humans manipulating different values of P(O|S) and P(O|¬S). They compared, for instance, a condition in which P(O|S) was 0.25 and P(O|¬S) was 0 to a condition in which P(O|S) was 1 and P(O|¬S) was 0.75. These two cases involve a contingency of 0.25. Wasserman and his colleagues (1993) observed that contingency judgments were more accurate in the case in which the probabilities were lower. This might seem to contradict our results. However, it should be noted, first, that the signal S was not an external signal, as in the present research, but an action of the participant. Second, the dependent variable was a contingency judgment, which might also be a source for differences. And third, that what Wasserman et al. (1993) found was that participants were more accurate when the probabilities were lower. This means that their participants showed an overestimation of contingency when the probabilities where higher. This is perfectly compatible with our finding that participants prepared more for the outcome when the probability of the outcome was higher.

Hallam et al. (1992) showed a similar result with rats. In Experiment 1, carried out with a licking suppression paradigm, they measured the level of preparatory behavior when varying P(O|S) and keeping P(O|¬S) equal to zero. They found that preparatory behavior did not increase linearly with changes in Δp. They observed a larger increment between 0.0 and 0.25 than between 0.50, 0.75 and 1. Note, however, that these manipulations involve variations in Δp, from 0 to 0.25, 0.50, 0.75, 1. In our experiment, Δp was kept constant at 0.4, so Hallam et al.’s result might not be as related to ours as it might seem at first. Similarly, Morís, Carnero, and Loy (2012) showed that rats can anticipate the arrival of food to the magazine device using an auditory stimulus when P(O|S) equals 0.1 and P(O|¬S) equals 0, but not when P(O|S) is 0.9 and P(O|¬S) is 0.8.

The differences between those two studies and the present one are so many that it is difficult to conclude what the crucial variables might be. For instance, in Morís et al.’s study, the fact that in one condition P(O|¬S) equals 0 adds information to the experimental subjects by showing that the context-alone trials predict no food and therefore the signal is the only possible predictor for food. This does not occur in the other condition, in which, because the two probabilities are very high (0.9 and 0.8), food occurs at almost any trial regardless of whether the signal is present or absent, which means that the signal is not particularly relevant.

To sum up, we tested preparatory behavior in an instrumental avoidance task and have shown that it does not occur only in response to reliable signals. Instead, relatively weakly reliable signals can trigger preparatory avoidance behavior, and this behavior is enhanced, as the outcome is more likely to follow them. We used humans as subjects but given the enormous similarities between humans and other species in their detection of contingencies and conditioned behavior (for reviews see Escobar & Miller, 2012; López & Shanks, 2008; Miller & Matute, 1996; Wasserman et al., 1996) we assume that the results can possibly be extended to other animals as well. An interesting question for further research would be whether this finding can be extended to other, more autonomic response systems, such as salivation, nausea or immune reactions, to mention just a few. Do they occur only in response to highly reliable signals or are they also biased by the probability of the outcome? In principle, there are no grounds to suspect differential influences of these variables on different response systems or species. Our preparatory behavior results in the present experiment, along with the previously mentioned findings in the human judgmental literature, suggest that these more autonomous response systems could possibly be also biased by signals that are frequently followed by significant outcomes.