The importance of replication in comparative psychology: the lesson of elephant quantity judgments.

Field and laboratory studies are sometimes restricted to relatively few individuals (primates: Brannon and Terrace, 1998; aquatic mammals: Kilian et al., 2003; birds: Pepperberg, 2012) due to the obvious difficulty of access to some species for prolonged periods. A small sample size, however, may raise problems of validity in generalizing data to the entire species and using them in cross-species comparisons. For this purpose, replication – one of the main principles of science – is particularly welcome in comparative psychology, especially when dealing with exotic species. Replication research, however, does not seem to have received favorable consideration over time by comparative psychologists. This probably reflects a bias in the guidelines of most international journals, which traditionally encourage original research studies with novel conceptual approaches and results instead of replications of previous works. In a questionnaire study involving reviewers of several psychological journals, it was even found that reviewers considered studies demonstrating new findings as more publishable than studies either replicating an effect or failing to replicate an effect (Neuliep and Crandall, 1993). As a consequence, inappropriate conclusions about the cognitive mechanisms of some species may be drawn. As far as we can determine, this is what may have happened recently regarding elephants’ numerical cognition.

To better understand the issue, it is important to take a step back from the non-verbal numerical systems that are commonly advanced in the literature. Studies on non-human animals, human infants, and adults undertaking non-verbal numerical tasks have provided compelling evidence for the existence of an approximate system of numerical representation. Such a system, commonly referred to as the “analog magnitude system,” seems to have no upper limit but is subject to a ratio limit in accordance with Weber’s Law, which states that the capacity to discriminate between two quantities becomes more accurate as the ratio between the smaller and the larger quantity increases (Beran, 2007). The mechanism is supposed to operate as an accumulator, with numerical magnitudes represented by distance along the accumulator.

Some researchers have also hypothesized the existence of a precise mechanism devoted to the processing of small numbers. This mechanism often is called subitization, defined as the ability to immediately perceive the quantity of small sets without serial counting (see, for example, Bonanni et al., 2011; Agrillo et al., 2012). However, while researchers still debate the existence of this small number system, they generally accept the analog magnitude system and use it to explain the full range of numerical competence in non-human animals. Indeed, how easily an animal can discriminate between two quantities has been found to depend upon the ratio in mammals (e.g., Beran, 2007; Ward and Smuts, 2007), birds (e.g., Al Aïn et al., 2009), amphibians (e.g., Krusche et al., 2010), and fish (e.g., Gòmez-Laplaza and Gerlai, 2011).

A recent paper by Irie and Hasegawa (2012) has now opened a wide-ranging debate in the scientific community regarding the performance of three elephants (Elephas maximus) that undertook a summation task. The authors adopted a modification of the method that Beran (2004) used with chimpanzees: different numbers of food items were sequentially inserted into two opaque boxes so that the subjects could never acquire a global view of the entire contents of the sets and could only use represented quantity information. In Beran’s study, the apes showed the typical sign of the analog magnitude system: ratio dependence. In contrast, elephants’ accuracy in discriminating quantities was reported to be unaffected by numerical ratio in the range of 8–13 items, a true exception to the results found in the literature; a similar conclusion was previously suggested in a study that tested five elephants with the range of 1–6 items (Irie-Sugimoto et al., 2009). In addition, two of three subjects were able to discriminate even a 0.86 ratio (6 vs. 7) at statistically significant levels, suggesting that the elephants were capable of enumerating far beyond the traditional limit of precise number discrimination, which is usually three to four items (Feigenson et al., 2004). These results raise the question as to whether elephants might display an enlarged object-file, the mechanism that is supposed to support subitizing (Butterworth, 2010). A somewhat similar mechanism has been previously invoked to explain some remarkable abilities in birds. Using a similar item-by-item procedure, Rugani et al. (2009) observed that domestic chicks were able to add discrete information beyond the limit of three to four items, suggesting that the chicks could track five distinct objects exactly. If elephants and other organisms do display a higher capacity for object-files than other animals do, their accuracy should decrease in the presence of larger sets, regardless of the ratio. This decrease must be explored in future studies.

The lack of ratio dependence is surprising and has theoretical implications – the very universality of Weber’s Law is at risk – but the question is still far from being resolved. Perdue et al. (in press) tested two elephants to replicate these findings using a more stringent protocol that included a larger number of trials and better control of visual cues. The authors failed to replicate the unique findings reported previously: numerical ratio was the best predictor of elephants’ performance. In the authors’ words, their results put “the elephants back in the herd.”

This is not the proper forum to discuss whether or not elephants do display a qualitatively different system to represent numerical information; further investigation is necessary (using larger samples, different stimuli, and varying paradigms), but at least the lesson we can currently draw is clear: comparative psychologists should be encouraged to replicate previous studies, both by using the same methodology, which eliminates the possibility that the initial results were an accident, and by using different methods to determine whether the results are not simply an artifact of the methodology adopted. A wider replication tradition in comparative psychology will enhance the process of theory refinement and helps us to form a broader comprehension of perceptual and cognitive mechanisms in non-human animals.