The Effects of Heuristics and Apophenia on Probabilistic Choice.

Given a repeated choice between two or more options with independent and identically distributed reward probabilities, overall pay-offs can be maximized by the exclusive selection of the option with the greatest likelihood of reward. The tendency to match response proportions to reward contingencies is suboptimal. Nevertheless, this behaviour is well documented. A number of explanatory accounts have been proposed for probability matching. These include failed pattern matching, driven by apophenia, and a heuristic-driven response that can be overruled with sufficient deliberation. We report two experiments that were designed to test the relative effects on choice behaviour of both an intuitive versus strategic approach to the task and belief that there was a predictable pattern in the reward sequence, through a combination of both direct experimental manipulation and post-experimental self-report. Mediation analysis was used to model the pathways of effects. Neither of two attempted experimental manipulations of apophenia, nor self-reported levels of apophenia, had a significant effect on proportions of maximizing choices. However, the use of strategy over intuition proved a consistent predictor of maximizing, across all experimental conditions. A parallel analysis was conducted to assess the effect of controlling for individual variance in perceptions of reward contingencies. Although this analysis suggested that apophenia did increase probability matching in the standard task preparation, this effect was found to result from an unforeseen relationship between self-reported apophenia and perceived reward probabilities. A Win-Stay Lose-Shift (WSLS ) analysis indicated no reliable relationship between WSLS and either intuition or strategy use.

INTRODUCTION

Multiple examples of apparently suboptimal human choice behaviour have been empirically documented over the last century (Kahneman, Slovic, & Tversky, 1982; Kahneman & Tversky, 2000; Shafir & LeBoeuf, 2002; Stanovich, 1999). One prominent example is probability matching, whereby people tend to select probabilistically rewarded responses in proportion to the relative likelihoods of reward, rather than the optimal strategy of always selecting the option with the highest reward probability (Shanks, Tunney, & McCarthy, 2002). Various explanatory accounts have been proposed for this behaviour. One possibility is that participants believe that the sequence of outcomes is determined not by chance, but by some relationship to their previous choices or previous outcomes. We refer to this category of model as the apophenia account. A second category of model suggests that the tendency to match response probabilities to reward probabilities is the result of a heuristic choice strategy that can be overcome by analysis and deliberation. We report two experiments in which we seek to assess the relative contribution of each in determining choice behaviour.

Probability Matching

Probability matching was first documented in rat behaviour under concurrent variable-ratio (VR) schedules of reinforcement (Brunswik, 1939). In humans, it was first reported in a verbal conditioning paradigm (Grant, Hake, & Hornseth, 1951; Jarvik, 1951) followed by a series of studies in which participants made repeated choices between probabilistic and differentially rewarded options (Goodnow & Postman, 1955; Neimark, 1956). This behavioural tendency is now supported by a comprehensive empirical literature in humans (Vulkan, 2000). In other animals the matching of choice proportions to the amount of reward obtained from each option has been consistently shown on concurrent variable-interval (VI) schedules. This phenomenon originated the now well-established matching law, although observed departures from strict matching have since required the inclusion of additional paremeters in what is termed the generalized matching law (Davison & McCarthy, 1988; Herrnstein, 1961; McDowell, 2013; Poling, Edwards, Weeden, & Foster, 2011). It is important to note that, on ratio schedules, the matching law predicts maximizing rather than probability matching, as discussed by Schulze, van Ravenzwaaij, and Newell (2017). This prediction has been supported in pigeons, which have been shown, under experimental conditions, to learn to approach maximizing behaviour on concurrent variable-ratio schedules (Herrnstein & Loveland, 1975). In the field of ecology however there is an overwhelming empirical consensus that, in natural environments, animals tend to allocate foraging time in proportion to the amount of resources available in different areas (Weber, 1998). Whether this situation is generally better approximated by a VR or VI schedule is open to question, though there are clearly other factors to take into account as well, such as reward persistence over time and the presence of competitors (see Schulze, van Ravenswaaij & Newell, 2015, 2017). This foraging pattern has been termed the ideal free distribution (IFD, Fretwell, 1972; Fretwell & Lucas, 1970) and is well documented across a multitude of animal species. These include mallards (Harper, 1982), sparrows (Gray, 1994), pigeons (Baum & Kraft, 1998; Bell & Baum, 2002), common cranes (Bautista, Alonso, & Alonso, 1995), rats (Tan et al., 2014), roe deer (Wahlstrom & Kjellander, 1995), white-tailed deer (Kohlmann & Risenhoover, 1997), guppy fish (Abrahams, 1989), cichlid fish (Godin & Keenleyside, 1984; Grand & Grant, 1994; Tregenza & Thompson, 1998), coho salmon (Grand, 1997), wood ants (Lamb & Ollason, 1993), dung flies (Blanckenhorn, Morf, & Reuter, 2000), and in bumblebees both directly in terms of IFD (Dreisig, 1995) and analogous “majoring and minoring” long-term foraging behaviour (Heinrich, 1979).

Apophenia

Apophenia is the tendency to perceive illusory patterns in random and unconnected events or stimuli (Conrad, 1958). In humans, this is an empirically well-documented phenomenon (Ayton & Fischer, 2004; Falk & Konold, 1997; Gilovich, Vallone, & Tversky, 1985). Moreover, a number of studies have reported that participants often perceive patterns in the sequences of outcomes in a standard probability learning task, even when none are present, and that this perception is associated with probability matching. Jarvik (1951) found that over half of participants believed a two-alternative forced choice verbal-conditioning sequence to derive from some form of pattern. Goodnow (1955) found overmatching in a task framed as gambling, but that participants’ behaviour reverted to probability matching when they were instructed to “search for a principle.” Both Rubinstein (1959) and Peterson and Ulehla (1965) found that participants made significantly more maximizing choices when the randomness of the series had been clearly demonstrated, on card and dice based alternatives of the task respectively. Yellott (1969) found that in a “noncontingent success” condition, introduced in the final block of a standard task, participants generated deterministic and patterned “superstitious” response sequences, with several participants spontaneously reporting having identified a pattern.

Of course, if the likelihoods of reward were not independent and identically distributed (IID), then the discovery of a predictable pattern in the reward sequence could allow for accurate prediction of future outcomes, leading to greater success than the exclusive selection of the more likely option. In this case, pattern matching could be the optimal response. This line of reasoning has led to the development of “smart” accounts of probability matching. Gaissmaier and Schooler (2008) found that participants with a higher working memory capacity, who also probability matched in the first half of their task (in which there was no pattern), were more likely to discover a pattern that was inserted into the second half of the task. In the majority of studies, however, rewards on the standard task are serially independent, so any attempt at pattern matching is detrimental to task performance. Nevertheless, participants are not usually told whether or not there is such a pattern in the sequence before the experiment. This means that although pattern-search behaviour is inevitably unsuccessful, it is based upon a lack or misappraisal of stimulus information, rather than being an inherently irrational choice strategy. It may therefore be considered an overextension of an otherwise normative behaviour.

Of the relatively few recent studies that have tested the contribution of apophenia to probability matching, results have been somewhat inconsistent. Unturbe and Corominas (2007) found that participants who reported more complex rules in the sequence were more likely to probability match than those who did not, while Wolford, Newman, Miller, and Wig (2004) observed that increasing the probability of alternation between rewarded stimuli led to the perception of greater randomness and, in turn, increased levels of maximizing. By contrast, on a two stage task, Koehler and James (2009) found that even when participants had no exposure to the specific reward sequence in the learning phase, probability matching was equally prevalent on the subsequent test phase. We test this by contrasting the choice behaviour, along with self-reported levels of apophenia, in participants who are explicitly informed that there are no predictable patterns to be found with participants who are left to discover this for themselves.

Heuristics

The second category of model assumes that probability matching is a heuristic-driven behaviour that can be overruled through deliberative reasoning. The heuristic account can be viewed in the framework of dual system models of decision-making, System 1 and System 2; a terminology introduced by Stanovich and West (2000) and popularised by Kahneman (2011) ENREF 21. System 1 refers to heuristic-based decision processes that are fast, automatic, and relatively effortless, although prone to errors, whereas System 2 processes are slower and generally more accurate, but require much more effort and processing power.

A number of empirical results implicate System 1 in matching behaviour, and System 2 in deviation from it. Neimark and Shuford (1959) found that probability matching on a standard task shifted to overmatching when participants could refer back to the results of previous trials during the experiment, or when they were asked to explicitly estimate the reward frequencies of each option, prompting explicit analysis of the outcome probabilities. Similarly, Nies (1962) found significant overmatching when participants were made explicitly aware of stimulus reward probabilities, as did Fantino and Esfandiari (2002). Another manipulation in the latter study found significant overmatching in response to providing a strategy recommendation to another participant midway through the task, requiring the explicit formulation of such a strategy.

Individual differences in heuristic and analytic processing are related to proximal indicators of intelligence. For example, West and Stanovich (2003) found that participants who applied a maximizing strategy had significantly higher SAT scores than those who probability matched. This led them to suggest that the matching response is a non-normative cognitive shortcut that is fast, effortless, and relatively intuitive; accounting for their results in that people higher in cognitive ability are more efficient at deliberative reasoning and thus in identifying the maximizing response. Consistent with this, Koehler and James (2010) found that high scorers on the Cognitive Reflection Test (CRT), a test designed to assess participants’ ability to deliberatively overrule their initial intuitions (Frederick, 2005), were much more likely to maximize (74%) than low scorers (11%). They also found that both many more participants subsequently endorsed maximizing as the best strategy than had actually applied it on the preceding task and that when asked to consider the possible success of both matching and maximizing strategies before the task, participants were significantly more likely to maximize. In light of these results, the authors argued that matching is only the dominant response because it is more readily available than the maximizing strategy. They concluded that matching results from a failure to sufficiently engage in deliberation. Correspondingly, Kogler and Kuehberger (2007) found that cueing System 2, through describing the task as a statistical test rather than a lottery, led to greater maximizing. Furthermore, a number of studies have shown that increasing performance-based financial incentives, which may encourage participants to engage in effortful analytic processing, leads to a reduction in the number of participants who probability match (Brackbill, Starr, & Kappy, 1962; Shanks et al., 2002 ; Siegel & Goldstein, 1959).

However, as with the apophenia account, the evidence is not entirely consistent. Jones and Liverant (1960) found significantly greater maximizing in nursery-(4-6) than elementary-age children (9-11), who tended to probability match. Derks and Paclisanu (1967) also found an overall positive relationship between age and matching across multiple age groups from nursery to college. These findings led to suggestions that maximizing is the more basic associational response, driven by positive recency, while probability matching results from some form of cognitive control. In the latter paper, studies showing that behaviour eventually approaches maximizing over longer tasks, up to 1,000 trials (Derks, 1962; Edwards, 1961), were also interpreted as being due to fatigue engendering more basic association-driven responses as the task progressed. We tested the heuristic account of probability matching by contrasting choice behaviour, along with self-reported use of intuition versus strategy, in participants who were explicitly told the reward contingencies with participants who learned them through experience.

Overview of Experiments

The apophenia and heuristic accounts of probability matching have often been considered in isolation, with the majority of studies designed specifically to test one or the other. However, the two models need not be mutually exclusive. In the experiments that follow, we aimed to assess the relative contribution of each to probability matching behaviour by considering the accounts together. Two factors were designed to respectively manipulate participants’ use of intuition versus strategy in determining their choices on the task, and the extent to which participants believed there to be a predictable pattern of sequential dependencies in the reward sequence.

One previous study (Fantino & Esfandieri, 2002) also included experimental conditions designed to manipulate both strategy usage and pattern belief, through informing participants of the reward probabilities and that 75% was considered a perfect score, respectively. However, the latter manipulation is quite indirect, with a degree of inference required by participants to interpret it as related to the potential presence of a pattern. Moreover, the assumption that either of these manipulations operated through the mechanisms that were predicted remains untested. In our study, we included a post-experimental questionnaire to assess the extent to which participants adopted an intuitive versus strategic approach to the task and their level of belief regarding whether or not there had been a predictable pattern in the sequence. This was designed to provide an additional, more direct measure of the efficacy of each of our experimental manipulations. Crucially, when combined with the behavioural data in a mediation-analysis, this can provide additional evidence of the putative underlying mechanisms of any observed effects. The questionnaire was also designed to ascertain individuals’ subjective estimates of the reward contingencies. Our intention was to determine whether participants were well calibrated to the actual probabilities of reward. This also enabled a comparative analysis of probability matching behaviour in which interindividual variance in probability estimates is controlled for (cf. Koehler & James, 2009).

Experiment 1

In Experiment 1, we assessed the relative contribution of apophenia and heuristic processing to probability matching through manipulating the information provided to the participants at the start of the task. If probability matching results from apophenia, we predicted that explaining to participants that the sequences of outcomes were random, other than any overall difference in reward probabilities, would lead to an increase in maximizing choices compared to participants who received no such explanation. We also predicted that explicitly providing participants with the outcome probabilities prior to the task would facilitate a strategic approach, reducing heuristic-based decision-making and increasing maximizing behaviour.

Method

Participants

One hundred and twenty members of the University of Nottingham community volunteered to take part in this experiment. Of these, 37 were male and 83 were female. Their ages ranged from 18 to 52 (Mage = 23.27, SD = 4.91).

Participants were paid an inconvenience allowance with a value contingent upon the choices that they made during the experiment. A maximizing strategy would accumulate an average total of £5.88, while matching would lead to an average pay-off of £4.87. Performance at chance would lead to an expected payment of £4.20.

Design

This was a 2 × 2 × 42 mixed model design with Probability (stated vs. learned) and Pattern (standard vs. no-pattern) as between-subjects factors and Block (1 to 42) as a within-subject factor. The basic experimental manipulations related to the instructions that participants received (see Appendix A). In the stated probability conditions, the participants were instructed that the outcome probabilities were 70% versus 30%, and in the learned probability conditions, the participants were not informed of the outcome probabilities. Participants in the no-pattern condition were told that there were no patterns in the sequences of outcomes and that these were entirely independent of the choices that they made, and participants in the standard pattern condition did not receive these instructions.

Which of the two colours was the more likely outcome was counterbalanced between participants to account for any bias in favour of a particular colour. All participants were randomly assigned to an experimental condition.

Procedure

The task was an iterated two-alternative forced choice, predicting whether either a blue or a yellow light bulb would flash on each of 420 trials. There was no initial training period. Task instructions that were provided to participants are shown in Appendices A, B, and C. Each trial began with a black central fixation cross on a white background. The duration of the fixation varied randomly between 1 s and 2 s. This was followed by the appearance of the two light bulbs on the left and right of the computer screen. The right-left position of the two coloured bulbs varied randomly between trials and was not predictive of reward. Participants were explicitly told in the instructions to make their choices on the basis of the colour rather than the position of the bulb, which was random (see Appendix A). On presentation of the stimuli, the participants made their predictions by pressing the q or p keys on a standard QWERTY keyboard to indicate the stimulus to the left or right side of the screen respectively. Shortly after each prediction (500 ms), one of the two light bulbs would be illuminated. Participants received a £0.02 reward for each correct prediction. The probability that each light would be illuminated on any one trial, and thus the reward contingencies of the two options, was fixed throughout the task. One colour bulb had a reward probability of .7 and the other of .3. A running total of winnings was shown throughout the task.

At the end of the experiment, the participants completed a questionnaire (see Appendix D) in which they were asked to estimate the probability of reward for each light bulb (0% to 100%), the extent to which they believed there to have been a predictable pattern in the reward sequence (1 to 5), and the extent to which they relied upon intuition or strategy in making their predictions (1 to 5).

Results

The proportions of maximizing responses for each 10-trial block are shown for each condition in Figure 1, Panel A. Distributions of individual proportions of maximizing choices across the entire task are shown in Figure 2, Panels D-E. These data were entered into a 2 × 2 × 42 mixed model analysis of variance (ANOVA) with Probability (stated vs. learned) and Pattern Instruction (standard vs. no-pattern) as between-subjects factors and Block (1 through 42) as a within-subject factor. A main effect of block, F(19.13, 2218.69) = 13.80, MSe = .04, p < .001, η2p = .11, and a significant linear effect of block, F(1, 116) = 87.09, MSe = .10, p < .001, η2p = .43, indicated that participants learned to allocate more responses to the maximizing alternative as the experiment progressed. A main effect of probability, F(1, 116) = 37.82, MSe = .58, p < .001, η2p = .25, indicated that when participants were told the probabilities of reward they tended to allocate more responses to the more likely outcome than when they had to learn it from experience alone. A reliable interaction between block and probability, F(19.13, 2218.69) = 3.09, MSe = .04, p < .001, η2p = .03, and a reliable linear interaction between the block and probability, F(1, 116) = 15.10, MSe = .10, p < .001, η2p = .12, suggested that stating the probabilities affected the rate at which participants learned to allocate responses to the more likely outcome. However, there was no effect of pattern instruction (F < 1.00) and no interaction between probability and pattern instruction, F(1, 116) = 3.30, MSe = .58, p = .07, η2p = .03. There was also neither an interaction nor a linear interaction between block and pattern instruction (F < 1.00). These nonsignificant results indicate that informing the participants that there was no pattern to be found in the sequence of outcomes did not substantially influence their choices. There were no 3-way interactions (F < 1.00).

Figure 1.

Showing the proportions of maximizing choices over the course of Experiment 1 (Panel A) and Experiment 2 (Panel B). Each block consisted of 10 trials.

Figure 2.

Histograms showing the number of participants who made different proportions of maximizing choices. Panels A to C show responding in the first 10-trial block, Panels D to F show the average proportions of maximizing responses across the whole task, and Panels G to I show average proportions of steady state maximizing choices over the final third of the task. Panels A, D, and G show the standard condition; Panels B, E, and H show the no-pattern condition; and Panels C, F, and I-Experiment 2.

The interaction between block and probability, as shown in Figure 1, Panel A, indicates a lower group-level learning effect in the stated probabilities condition. This might represent a lower learning rate across all participants in this condition. Alternatively, it may be explained by a sub-set of participants beginning the task either at or near maximizing behaviour, having reasoned that this is the optimal strategy from the additional probability information given to them before the task began. Figure 2 shows histograms of individual-level proportions of maximizing choices in the initial 10 trials of the task for each the standard condition (Panel A) and the no-pattern condition (Panel B). These clearly show that more participants in the stated probabilities condition maximized from the outset of the experiment.

In order to assess experimental effects exclusively upon steady-state responding, a separate 2 × 2 between-subjects ANOVA was conducted, with factors of Probability Condition and Pattern Instruction, and the dependent variable of Proportion of Maximizing Choices made over the final third of the task (from trial 280 onwards). This revealed a main effect of probability, F(1, 116) = 15.96, MSe = .02, p < .001, η2p = .12, but neither a main effect of pattern instruction (F < 1.00) nor a reliable interaction effect, F(1, 116) = 3.01, MSe = .02, p = .09, η2p = .03. Distributions of individual proportions of maximizing choices over the final third of the task are shown in Figure 2, Panels G-H.

Subjective estimates of reward probabilities

At the end of each experiment, the participants were asked to say what they thought the outcome probabilities were. These data are shown in Table 1. To determine what effect the instructions had on participants’ representation of the outcome probabilities, we entered these subjective estimates of the outcomes into a 2 × 2 ANOVA with Probability (stated vs. learned) and Pattern Information (standard vs. no-pattern) as between-subjects factors. There was neither an effect of probability, F(1, 116) = 1.89, MSe = .01, p = .17, η2p = .02, nor of pattern information, F(1, 116) = 1.76, MSe = .01, p = .19, η2p = .02, nor a significant interaction, F(1, 116) = 2.77, MSe = .01, p = .10, η2p = .02.

Table 1.

Subjective Estimates of Outcome Probability of the High Probability Option by Condition and Experiment

	Pattern instruction
	Standard		No-pattern		Experiment 2
	M	SE	M	SE	M	SE
Stated	.685	.010	.690	.010	.692	.009
Learned	.727	.020	.686	.013	.651	.017

Heuristics

Participants were asked to rate the extent to which they relied on strategy or intuition to make their decisions (see Table 2). These data were entered into a 2 × 2 ANOVA, with Probability and Pattern Information as between-subjects factors. A main effect of probability F(1, 116) = 4.80, MSe = 1.00, p = .03, η2p = .04, showed that participants in the stated probability conditions reported greater strategy use than those in the learned conditions. There was no effect of pattern instruction (F < 1.00) and no interaction between the two F(1, 116) = 1.20, MSe = 1.00, p = .86, η2p = .01.

Table 2.

Subjective Reports of Strategy Use by Condition and Experiment

	Pattern instruction
	Standard		No-pattern		Experiment 2
	M	SE	M	SE	M	SE
Stated	3.800	0.176	3.633	0.212	4.000	0.198
Learned	3.200	0.188	3.433	0.149	3.600	0.189

Note. Scores range from 1 = pure intuition to 5 = pure strategy.

Apophenia

Finally, participants were asked to rate their belief that the outcome sequences contained a pattern (see Table 3). A third 2 × 2 ANOVA revealed neither a significant main effect of probability condition, F(1, 116) = 1.90, MSe = 1.75, p = .17, η2p = .02, nor, surprisingly, of pattern instruction (F < 1.00). There was no significant interaction between the two factors, F(1, 116) = 1.54, MSe = 1.75, p = .22, η2p = .01.

Table 3.

Subjective Reports of Belief that Outcome Sequences Contained a Pattern by Condition and Experiment

	Pattern instruction
	Standard		No-pattern		Experiment 2
	M	SE	M	SE	M	SE
Stated	2.000	0.209	2.267	0.235	2.267	0.203
Learned	2.633	0.256	2.300	0.263	2.700	0.204

Note. Scores range from 1 = strongly disagree to 5 = strongly agree.

Win-Stay-Lose-Shift Analysis

As an additional variable of interest, proportions of choices over the entire task that were consistent with a Win-Stay-Lose-Shift (WSLS) strategy were calculated for each participant. These data are shown in Table 4. Figure 3 shows the distributions of individual-level behaviour consistent with a WSLS strategy for both the standard and no-pattern conditions (Panels A and B respectively). A 2 × 2 ANOVA was conducted to assess differences in WSLS proportions between conditions. This revealed neither a significant main effect of probability, F(1, 116) = 3.12, MSe = .01, p = .08, η2p = .03, nor of instruction, F(1, 116) = 3.80, MSe = .01, p = .05, η2p = .03, nor a significant interaction (F < 1.00).

Table 4.

Proportions of Win-Stay-Lose-Shift Consistent Choices by Condition

	Pattern instruction
	Standard		No-pattern
	M	SE	M	SE
Stated	.694	.009	.660	.011
Learned	.662	.016	.647	.013

Figure 3.

Histograms showing the number of participants and their choices that were consistent with a raw (Panels A and B) and adjusted (Panels C and D) Win-Stay Lose-Shift (WSLS ) strategy in Experiment 1: Panels A and C show the standard condition and Panels B and D-the no-pattern condition.

Following this, a linear regression revealed a significant positive effect of reported use of a strategy over intuition on raw proportions of WSLS-consistent choices (b = .02, t[118] = 3.48, p < .001), with strategy use explaining 9% sample variance (R2 = .09).

However, it is important to consider here that a strict maximizing strategy will lead to a baseline proportion of .7 WSLS consistent choices (comprising 100% “stay” choices of which 70% will be rewarded “wins”). An entirely WSLS-independent probability matching strategy is calculated to lead to an expected baseline of .58 WSLS-consistent choices. As maximizing increases the proportion of WSLS, congruent choices will increase independently of whether or not WSLS is actively pursued as an approach. This is a concern due to the large discrepancies in maximizing behaviour between participants, in particular the systematic divergence between stated and learned probability conditions and the potentially confounding relationship between maximizing and strategy use.

In order to adjust for this issue, expected baseline proportions of WSLS consistent choices were calculated for each participant based upon their proportion of maximizing choices over the entire task. These were then subtracted from actual proportions of WSLS consistent choices as a measure of the prevalence of WSLS-like choice patterns over and above what would be expected by chance. These data are shown in Table 5. Distributions of individual-level proportions of adjusted WSLS-consistent choices are shown for both standard and no-pattern conditions in Figure 3, panels C-D. Another 2 × 2 ANOVA was then conducted to assess differences in WSLS deviations from baseline between conditions. This revealed significant main effects of both probability, F(1, 116) = 11.57, MSe = .002, p < .001, η2p = .09, and instruction, F(1, 116) = 8.29, MSe = .002, p = .005, η2p = .07, but no significant interaction (F < 1.00). This indicates that when participants are told that there is no pattern or told the reward probabilities, they tend to make fewer choices that are consistent with a WSLS strategy. The histograms suggest that only a few participants across all conditions made substantively more WSLS-consistent choices than their expected baseline. Each of these effects seems, therefore, to result predominantly from a modest shift across a wider range of participants towards making slightly fewer WSLS-consistent choices when aware of the reward probabilities or informed that there is no predictable pattern in the sequence.

Table 5.

Deviation in Proportions of Win-Stay-Lose-Shift Consistent Choices From Baseline, by Condition

	Pattern instruction
	Standard		No-pattern
	M	SE	M	SE
Stated	.027	.008	.006	.007
Learned	.062	.011	.031	.009

Following this, a linear regression was conducted to assess the effect of strategy use on this adjusted WSLS variable. This revealed no significant effect of reported strategy use (b = −.007, t[118] = −1.40, p = .165, R2 = .02). Interestingly, although nonsignificant, the trend is for WSLS choices being associated with a more intuitive rather than strategic approach to the task.

Discussion

The results from Experiment 1 showed that stating the probabilities of reward results in both a greater proportion of maximizing responses and an increase in the use of strategy over intuition, compared to when participants are left to acquire knowledge of the reward probabilities through experience alone. However, informing participants that there were no patterns in the reward sequences, and that they were independent events, neither significantly affected proportions of maximizing choices nor reduced the tendency toward apophenia. These effects were equivalent whether considering choices made over the entire task or steady-state responding over only the final third of trials. The persistence of the positive effect of stating outcome probability of the high probability option on maximizing choices post-learning suggests that this reflects a genuine difference in approach to making choices, rather than one resulting from the more prosaic difference in information available at the outset of the task. In Experiment 2, we introduced a potentially stronger manipulation to reduce the effect of apophenia by concealing the outcome of each prediction over each 10-trial block.

Experiment 2

In Experiment 1, we observed that telling participants about the probability of the outcomes affected the predictions that they made. However, telling the participants that there were no patterns to be discovered in the sequence of outcomes, and that the outcomes were independent of the choices that they made, had no significant effect on their predictions. Moreover, although this manipulation was designed to reduce apophenia, we observed no effect on the participants’ perception of randomness in the sequence of outcomes. In Experiment 2, we attempted a stronger manipulation to reduce apophenia-by concealing the outcome of each prediction and presenting participants instead only with a proportion correct at the end of each 10-trial block. In this manner, we were hoping to preclude the possibility of observing a pattern in the trial-by-trial reward sequence, and therefore to reduce apophenia.

Sixty members of the University of Nottingham community volunteered to take part in this experiment. Of these, 23 were male and 37 were female. Their ages ranged from 17 to 35 (Mage = 22.60, SD = 2.97). The methods of recruitment and of calculating individual participants’ inconvenience allowances were identical to Experiment 1.

This was a 2 × 42 mixed model design with Probability (stated vs. learned) as a between-subjects factor and Block (1 to 42) as a within-subject factor. The basic experimental preparation was identical to Experiment 1 with the exception that the participants were only given the outcome of their predictions at the end of each 10-trial block.

The procedure was identical to Experiment 1 with the exception that participants did not see the illumination of a light bulb after each trial. Instead, feedback was given in written format after 10 trials.

The proportions of maximizing responses for each 10-trial block are shown for each condition in Figure 1, Panel B. The distribution of choices for the first 10 trials are shown in Figure 2, Panel C. This suggests, just as in Experiment 1 (Panels A and B), that participants tended to allocate more of their initial responses to the maximizing choice when they were told the reward probabilities. These data were entered into a 2 × 42 mixed model ANOVA with Probability (stated vs. learned) as a between-subjects factor and Block (1 through 42) as a within-subject factor. A main effect of block, F(14.63, 848.25) = 2.57, MSe = .10, p < .001, η2p = .04, and a significant linear effect of block F(1, 58) = 19.88, MSe = .10, p < .001, η2p = .26, indicated that participants learned to allocate more responses to the maximizing alternative as the experiment progressed. A main effect of probability, F(1, 58) = 10.76, MSe = .78, p < .002, η2p = .16, indicated that when participants were told the probability of reward, they tended to allocate more responses to the more likely outcome than when they had to learn it from experience alone. There was no interaction between block and probability (F < 1.00).

To isolate the effect of probability condition on steady-state responses, an independent samples t test was conducted. This found that the positive effect of stated over learned reward probabilities on proportion of maximizing choices remained over the final third of the task, t(58) = 2.46, p = .02. This effect was, therefore, again not the result of a discrepancy between groups during the learning phase but indicated a more substantive difference in decision making between the two conditions.

Subjective estimates of outcome probability of the high probability option are shown in Table 1. An independent samples t test revealed that participants in the learned probability condition estimated the outcome probability of the high probability option to be lower than the stated probability condition, t(58) = 2.19, p = .03.

Heuristics and Apophenia

Self-reports of strategic choices and belief that the outcome sequences contained patterns are shown in Tables 2 and 3. These did not differ between groups, t(58) = 1.15, p = .15; t(58) = 1.51, p = .14.

Comparisons with Experiment 1

To assess the efficacy of the aggregate feedback manipulation introduced in Experiment 2, these results were compared to the standard condition of Experiment 1.

The proportion of maximizing responses for each 10-trial block were entered into a 2 × 2 × 42 mixed model ANOVA with Probability (stated vs. learned) and Pattern Information (standard vs. experiment 2) as between-subjects factors and Block (1 through 42) as a within-subject factor. A main effect of block, F(18.23, 2115.02) = 7.31, MSe = .062, p < .001, η2p = .06, and a significant linear contrast of block F(1, 116) = 58.20, MSe = .12, p < .001, η2p = .33, indicated that participants learned to allocate more responses to the maximizing option as the experiment progressed. A main effect of probability, F(1, 116) = 39.34, MSe = .66, p < .001, η2p = .25, indicated that when participants were told the probability of reward, they tended to allocate more responses to the more likely outcome than when they had to learn it from experience alone. Although there was no reliable interaction between block and probability, F(18.23, 2115.02) = 1.46, MSe = .06, p = .09, η2p = .01, modelling the interaction between block and probability as a linear contrast did reveal a significant effect, F(1, 116) = 6.91, MSe = .12, p = .01, η2p = .06. This suggests that stating the probabilities again affected the rate at which participants learned to allocate responses to the more likely outcome. However, there was no effect of pattern information (F < 1.00). There were also neither significant interactions between pattern information and probability, F(1, 116) = 1.52, MSe = .66, p = .22, η2p = .01, or block, F(18.23, 2115.02) = 1.29, MSe = .06, p = .18, η2p = .01, nor a reliable linear interaction between pattern information and block, F(1, 116) = 1.19, MSe = .12, p = .28, η2p = .01. These indicate that concealing the trial-by-trial outcome sequence did not influence participants’ choices. There were no three-way interactions (F < 1.00).

In order to again assess experimental effects exclusively upon steady-state responding, a separate 2 × 2 between-subjects ANOVA was conducted, with factors of Probability Condition and Pattern Information, and the dependent variable of Proportion of Maximizing Choices made over the final third of the task. This revealed a main effect of probability condition, F(1, 116) = 20.54, MSe = .02, p < .001, η2p = .15, but neither a main effect of pattern information (F < 1.00), nor an interaction effect (F < 1.00).

Subjective estimates of outcome probability of the high probability option are shown in Table 1. A 2 × 2 ANOVA was conducted with Probability (stated vs. learned) and Pattern Information (standard vs. experiment 2) as between-subjects factors. There was no effect of probability (F < 1.00). However, a main effect of pattern information, F(1, 116) = 5.69, MSe = .006, p = .02, η2p = .05, suggested that participants in Experiment 2 estimated the ‘outcome probability of the high probability option to be lower than those in the standard condition of Experiment 1. A reliable interaction between probability and pattern information, F(1, 116) = 8.23, MSe = .006, p = .005, η2p = .07, indicated that stating the reward probabilities moderated the difference in estimates between the standard condition of Experiment 1 and the aggregate feedback condition of Experiment 2.

Self-reported use of strategy versus intuition (see Table 2) was entered into a 2 × 2 ANOVA with Probability and Pattern Information as between-subjects factors. A main effect of probability, F(1, 116) = 7.09, MSe = 1.06, p = .009, η2p = .06, showed that participants in the stated probability conditions reported greater strategy use than those in the learned conditions. There was no significant effect of pattern information, F(1, 116) = 2.55, MSe = 1.06, p = .11, η2p = .02 and no interaction between the two (F < 1.00).

Self-reported beliefs that the outcome sequences contained a pattern (see Table 3) were entered into a third 2 × 2 ANOVA. A significant main effect of probability condition, F(1, 116) = 5.92, MSe = 1.44, p = .02, η2p = .05, indicated that participants in the learned probability conditions reported significantly higher levels of apophenia than those in the stated conditions. Surprisingly, as in Experiment 1, there was neither an effect of pattern information (F < 1.00) nor an interaction between the two factors (F < 1.00).

Experiment 2 attempted to use a potentially stronger manipulation to reduce the effect of apophenia in probability learning. However, the effect of concealing the outcomes over each 10-trial block was as ineffective as simply telling participants that there was no pattern in the sequence. We infer from this that participants were just as capable of internally generating a perception of a sequential pattern as they were of mistakenly deriving one from an observed outcome sequence. We did not conduct a WSLS analysis on these data because the participants could not see the outcomes of individual trials and could not therefore base a subsequent choice on the result of any single preceding trial.

In the next section, we use mediation analysis to determine the extent to which the observed effect of probability condition on participants’ choices was mediated by its effects on strategy usage and apophenia.

Mediation Analyses

Experiments 1 and 2 each found a significant effect of probability condition on participants’ proportion of maximizing choices. Experiment 1 also found a significant effect of probability condition on self-reported use of strategy. Although, when considered alone, the same effect did not reach significance within the sixty participants in Experiment 2, the observed effect was both in the same direction and of an equivalent size to the combined effect observed across Experiment 1’s conditions. When considering the same aggregate feedback group together with the standard condition of Experiment 1, the increase in power found the effect of probability condition on levels of apophenia as once again statistically significant. By contrast, no effects of pattern instruction or pattern information, in Experiments 1 or 2, respectively, were found upon either strategy use or apophenia.

To investigate the extent to which the effect of probability condition on choice behaviour was mediated by its influence on participants taking a strategic versus intuitive approach to the task and their levels of apophenia, we conducted a mediation analysis. We used the PROCESS macro for SPSS (Hayes, 2012). Data was concatenated across all three apophenia conditions, due to the lack of effect of either pattern instruction (Experiment 1) or aggregate feedback (Experiment 2) on either choice behaviour, levels of strategy use, or apophenia. For this analysis, proportions of maximizing choices were calculated for the final third of the task, with the aim of accounting for learning effects. Mediation model Number 6 was used, with a single independent variable and two mediator variables. Effects were calculated for each of 10,000 bootstrapped samples. Model results are presented in Figure 4 (variable coding is as follows: Stated = 0, Learned = 1; Intuition 1-5 Strategy; Belief no Pattern 1-5 Belief was Pattern; Proportion of Maximizing Choices 0-1).

Figure 4.

Mediation model for all participants. Unstandardized regression coefficients shown with asterisk if significant. Solid and dotted lines indicate significant and non-significant effects respectively. α = .05.

The direct effect of probability condition on strategy usage was found to be significant, t(178) = −2.63, p = .009, with this predictor accounting for 4% of the sample variance (R2 = .04). The direct effect of probability condition on apophenia was found to be nonsignificant, t(177) = 1.19, p = .24, while the direct effect of strategy use on apophenia was found to be significant, t(177) = −4.31, p < .001, with these two predictors accounting for 11% of the sample variance (R2 = .11). The direct effects of probability condition, t(176) = −3.78, p < .001, and strategy use, t(176) = 7.35, p < .001, on proportion of maximizing choices were each found to be significant, whereas the direct effect of apophenia was not, t(176) = −0.31, p = .76. These three predictors accounted for 34% of the sample variance (R2 = .34).

The unstandardized indirect effects of probability condition on proportion of maximizing choices were as follows: Through intuition versus strategy use (−.400) (.068) = −.027, with bias corrected 95% CIs ranging from −.051 to −.008, indicating statistical significance of this effect at α = .05. Through apophenia (.216) (−.002) = −.0005, with bias corrected 95% CIs ranging from −.009 to .003, indicating nonsignificance of this effect at α = .05. Through first strategy use then apophenia (−.400) (−.378) (−.002) = −.0004, with bias corrected 95% CIs ranging from −.004 to .002, indicating that this effect was also nonsignificant at α = .05.

Assessment of Reliability of Within-Group Effects

In the mediation analysis, data from all three apophenia conditions were concatenated. In light of this, it remains unclear whether these effects are consistent between individual experimental conditions. To address this issue, a series of multiple regression analyses were conducted, for each experimental group. Predictor variables were Self-Reported Strategy Use and Apophenia, and the dependent variable was Maximizing Choices over the final third of the task. Results are shown in Table 6.

Table 6.

Effects of Strategy Use and Apophenia Split by Experimental Condition

Experimental Condition	Strategy Use				Apophenia
	b		p		b		p
	Learned	Stated	Learned	Stated	Learned	Stated	Learned	Stated
Standard	.077	.032	.019*	.012*	-.009	-.015	.688	.158
Instruction	.086	.064	.017*	<.001*	.034	.013	.088	.265
Aggregate	.072	.078	.011*	<.001*	-.049	.001	.058	.939

Note. b values are unstandardized, p values uncorrected, asterisks represent significance at α = .05.

Controlling for Variance in Perceived Reward Probabilities— Summary of a Parallel Analysis

In principle, probability matching may be accurately defined as the selection of each alternative in proportion to its subjectively perceived likelihood of reward, even where this differs from an option’s objective reward probability. Since the participants in the learned probabilities conditions were never explicitly told the reward contingencies, it follows that their perceived likelihoods may have differed from the actual values. Although group average estimates of these probabilities were generally well calibrated to actual probabilities, greater variance was seen in both the choice behaviour and probability estimates of participants in the learned probability conditions. Separate linear regressions revealed a reliable effect of perceived reward probabilities on proportions of maximizing choices within the learned (b = .78, t[88] = 4.64, p < .001, R2 = .20) but not the stated (b = .20, t[88] = 0.97, p = .33, R2 = .01) reward probability conditions. This substantiated the hypothesis that the disproportionate variance in proportions of maximizing choices when probabilities were learned was largely driven by individual differences in perceptions of the reward probabilities.

At least one previous study has controlled for individual differences in perceived reward contingencies (Koehler & James, 2009). We ran a parallel analysis to investigate the effects of controlling for this variance in a similar manner. For this analysis, we created a new measure of choice behaviour: pMax-pEst. This measure accounts for between-subjects variance in perception of reward probabilities through simply subtracting each participant’s subjective estimate of the reward probability of the more likely option from their proportion of maximizing choices over the final third of the task. Group averages are shown in Table 7.

Table 7.

Deviation of Choices from Probability Estimates Within Each Experimental Group

Experimental Condition	pMax–pEst
	Learned	Stated
Standard	.078 (.027)	.259 (.015)
Instruction	.168 (.025)	.219(.023)
Aggregate	.170 (.027)	.224 (.019)

Note. SEs are shown in parentheses.

The pattern of results with respect to the effect of apophenia manipulations was different from the analysis in which individual variance in perceptions of reward probabilities was not accounted for. In the present case, and treating the three conditions together, while there was no significant main effect of apophenia condition on deviation from matching, F(2, 174) < 1.00, a significant interaction between probability and apophenia conditions was evident, F(2, 174) = 4 .95, p = .008, η2p = .05. This effect was driven by the relatively low apparent deviation from matching in the learned standard group relative to the learned probabilities conditions of each apophenia manipulation. By contrast, no sign of such an interaction effect on choice behaviour involving either pattern instruction or pattern information was apparent in the primary analyses of Experiments 1 or 2, in which variance in estimates was not adjusted for.

In addition, parallel mediation analyses were conducted upon the dependent variable of apparent deviation from matching (pMax-pEst). These are not detailed here in full, for the purpose of concision, but did indicate a significant negative effect of apophenia within the standard condition only, t(56) = −3.22, p = .002. However, it is clear from our primary analysis that this result does not reflect any effect of apophenia upon proportions of maximizing choices. The alternative explanation is that this is an illusory effect driven instead by an unforeseen relationship between apophenia and probability estimates. The presence of this relationship was confirmed, through correlation analysis, within the standard (r = .34, N = 60, p = .007) but neither instruction (r = .15, N = 60, p = .26) nor aggregate feedback (r = −.17, N = 60, p = .21) conditions.

Reasons for the existence of this relationship are unclear. We are aware of no other studies that have found an equivalent effect, and it may represent nothing more than a statistical fluke. However, based upon these results, we suggest that raw proportions of maximizing choices are, on balance, the more reliable measure. A majority of previous studies have used this measure. However, discrepancies highlighted from running this parallel analysis do raise concerns, and if replicated, may prove a substantial consideration for any studies that wish to adjust for variance in perceived reward probabilities when determining matching behaviour.

Discussion

We examined the effects of apophenia and heuristics on choice behaviour in a probability learning task. In Experiment 1, we manipulated the information that participants received in order to reduce the effects of apophenia and heuristic decision making. In Experiment 2, we used a potentially stronger manipulation, namely, concealing trial-by-trial outcomes, to reduce apophenia. We also collected self-report measures of belief that the sequence had contained a pattern, and of intuitive or strategic approaches to the task.

The results of Experiment 1 indicated higher learning rates, unsurprisingly, in the learned probabilities condition. Relatively lower learning levels when probabilities were stated are limited by the ceiling of maximizing behaviour coupled with a higher initial level of maximizing responses, given the possibility of determining the optimal strategy with reward contingency information before the task began. This pattern of results is highly congruent with those of Newell, Koehler, James, Rakow, and van Ravenzwaaij (2013) , in which an interaction is described between higher initial maximizing driven by ‘top-down’ information and more slowly increasing levels resulting from ‘bottom-up’ feedback-based learning.

Overall, the explicit statement of outcome probability of the high probability option led participants to make a significantly greater proportion of maximizing choices than when probabilities required learning through experience. Similar effects have been reported by both Nies (1962) and Fantino and Esfandiari (2002). Steady-state analyses revealed this to remain the case even in the post-learning period of the task. In fact, strikingly, the average proportion of maximizing choices over the final 10 of the 420 trials, for participants who had to learn reward contingencies through experience alone (M = .79, SE = .04), remained lower than the equivalent proportion on the first 10 trials for participants who were explicitly informed of reward probabilities before the task (M = .85, SE = .03). This suggests that there may be some qualitative impediment to reaching the maximizing response through experience alone. It also extends a finding of Newell and Rakow (2007) that there is a negative effect of experience relative to description after 60 trials.

Participants in the stated probabilities conditions were also found to be significantly more likely to take a strategic, rather than intuitive approach to the task. By contrast, explicitly telling participants that the reward sequence would be random, with no predictable pattern, neither significantly affected choice behaviour nor levels of apophenia. This lack of behavioural effect of stressing the randomness of the sequence seems inconsistent with the findings of both Rubinstein (1959) and Peterson and Ulehla (1965). However, the absence of any effect upon even self-reported levels of apophenia suggested that written instructions might simply not be sufficient to convince participants of the randomness of the sequence.

In addition, we assessed the proportion of choices within each condition of Experiment 1 that was predictable by a simple WSLS strategy, while controlling for the confounding effect of raw proportions of maximizing choices. This revealed that participants made significantly fewer WSLS-consistent choices when probabilities were stated before the task, and also when they were instructed that there would be no predictable pattern in the reward sequence. No significant relationship was found between strategy-use and proportions of WSLS-consistent choices, while the trend was for WSLS-choice behaviour to be related to an intuitive rather than strategic approach to the task. This result seems to run contrary to a finding of Otto, Taylor, and Markman (2011) that WSLS behaviour was less prevalent in participants in a dual-task situation that was designed to compromise executive resources.

In Experiment 2, we concealed the outcome sequence from participants, providing reward feedback in aggregate format over 10-trial blocks. We hypothesised that this would be a stronger method of reducing apophenia than simply informing participants that there would be no pattern. However, comparing this dataset with the standard condition of Experiment 1 revealed that aggregating feedback was also ineffective, both at influencing choice behaviour and reducing levels of apophenia. The absence of any effect here is surprising, particularly considering that, in an almost identical manipulation, Gao and Corter (2015) did find an effect of presenting grouped rather than single trials. It is instead more consistent with the outcome of Koehler and James (2009) who also demonstrated no effect of providing outcome information in aggregate format, although the present study masked trial-by-trial outcomes that were concurrent with responses rather than the outcomes of an independent pre-task learning phase.

In a novel development with respect to the findings of previous studies that have investigated the effects of heuristics or apophenia on probability matching, we applied a mediation analysis to more clearly delineate the pathways of action of any observed effects. This revealed a significant indirect effect of stating reward probabilities, via facilitation of strategy use over intuition, on choice behaviour. This finding corroborates the mechanism by which this experimental manipulation was hypothesised to take effect. These findings are consistent with a heuristic-based account of probability matching, by which matching is an intuitive response and maximizing depends upon more rational deliberation, as espoused by Kogler and Kuehberger (2007) and Koehler and James (2010).

This mediation analysis also indicated that not only were attempted manipulations of apophenia ineffective, but that apophenia itself had no substantive effect on choice behaviour. This fails to replicate results of Unturbe and Corominas (2007). It is important to note that our questionnaire can only account for post-experimental levels of apophenia. It is possible that some participants did, at times during the task, believe there to be a pattern in the sequence, subsequently learning this not to be the case. This is an interesting consideration and may merit further study in itself. However, as we compare the effects of apophenia on steady-state responding, it is ultimately steady-state levels of apophenia that we are concerned with, for which post-experimental questioning should be suited to capture an accurate measure.

One further intriguing result of the mediation analysis was that it highlighted a significant negative relationship between strategy use and apophenia, despite pattern search being a seemingly strategic approach. This is likely due to a substantial subset of participants who applied a highly deliberative approach in determining that the maximizing strategy was optimal, and who also realized that this was incompatible with the presence of a predictable pattern in the reward sequence.

To emphasize the reliability of these findings, a series of separate multiple regression analyses revealed that the significant positive effect of strategy use on steady-state proportions of maximizing choices was evident within each of the six experimental conditions, across both experiments. By contrast, there was no significant effect of reported apophenia evident in any of these conditions.

A parallel analysis was also conducted in which individual differences in participants’ estimates of the reward contingencies was adjusted for, in a similar manner to that of Koehler and James (2009). This analysis revealed remarkably different results than our primary analysis with respect to the apparent effects of apophenia on choices. However, further investigation revealed that these effects were driven by a perplexing relationship between levels of apophenia and estimates of reward probabilities, which was present only within the standard task condition.

In summary, our results fail to replicate previous findings that have implicated apophenia as a key determinant of probability matching. They do, however, indicate a robust effect of intuition versus strategy use and support the conclusion that this is the primary factor behind a predisposition toward matching or maximizing behaviour respectively. Furthermore, a mediation analysis provided statistical verification of the hypothesis that the increase in maximizing behaviour when reward probabilities are stated is driven largely by the promotion of a more strategic approach to the task. These findings are consistent with a heuristic-based account, by which the matching response is prepotent when the decision-maker acts intuitively, but may be overruled if the maximizing strategy is recognized. Additional findings included an unforeseen relationship between self-reported levels of apophenia and perceived reward probabilities, higher levels of WSLS-consistent choices when participants began the task naive of either reward contingencies or the statistical independence of outcomes, and the absence of any reliable relationship between WSLS and overall strategy use.