Helping in humans and other animals: a fruitful interdisciplinary dialogue.

Humans are arguably unique in the extent and scale of cooperation with unrelated individuals. While pairwise interactions among non-relatives occur in some non-human species, there is scant evidence of the large-scale, often unconditional prosociality that characterizes human social behaviour. Consequently, one may ask whether research on cooperation in humans can offer general insights to researchers working on similar questions in non-human species, and whether research on humans should be published in biology journals. We contend that the answer to both of these questions is yes. Most importantly, social behaviour in humans and other species operates under the same evolutionary framework. Moreover, we highlight how an open dialogue between different fields can inspire studies on humans and non-human species, leading to novel approaches and insights. Biology journals should encourage these discussions rather than drawing artificial boundaries between disciplines. Shared current and future challenges are to study helping in ecologically relevant contexts in order to correctly interpret how payoff matrices translate into inclusive fitness, and to integrate mechanisms into the hitherto largely functional theory. We can and should study human cooperation within a comparative framework in order to gain a full understanding of the evolution of helping.

Introduction

Helping behaviours that increase the direct fitness of recipients underpin several major evolutionary transitions [1]. Acts in which helpers provide any resource (e.g. food, time) are interesting because evolutionary theory strongly emphasizes the importance of competition and selfish behaviour. Humans are adept at helping each other. From a quantitative perspective, this trait is not unique in the animal kingdom; arguably, hymenopterans and other eusocial species are even more helpful within their colonies. However, helping by the latter is explained by biological altruism based on kin selection [2,3], while humans also cooperate with unrelated individuals for direct fitness benefits on a scale that is unmatched by any other species. Importantly, the criteria for cooperating are highly flexible: the same individual may cooperate with friends, colleagues, supporters of the same football club, political affiliates, compatriots or even international alliances. Help can be provided in different currencies (e.g. time/money/physical effort) and is also often provided in situations where it is unclear how return benefits may be accrued, from letting a car out at a busy junction to donating to victims of natural disasters in far-away countries.

The frequency and scale of human helping could depend on several factors that appear to be unique to humans: our capacity for spoken and written language, the use of tags to identify groups, societal-level norms and institutions that both prescribe cooperation and punish defection, various media channels that allow for large-scale communication and coordination, and banks to transfer money—a unique non-perishable resource—anywhere. One might therefore wonder to what extent research on human cooperation yields idiosyncratic explanations, rendering comparisons with other species useless. One may also ask whether research on human cooperation is suitable for publication in biological journals like Proceedings of the Royal Society B. Here, we address this question. We first summarize briefly the enormous impact that theoretical concepts and empirical studies of human cooperation have had on research in other species. We then highlight topics of interdisciplinary interest and shared future challenges. It should become clear that we favour an open-minded and inclusive approach, where humans are just another species that can be studied under the general framework of evolutionary theory. While human cooperation might be more peculiar than cooperation in many non-human species, each species would appear unique if every detail was taken into consideration. Therefore, a distinction between disciplines based on study organisms only hinders progress.

Theory on human helping as inspiration for biological research

Theoretical approaches to understand helping in humans predate evolutionary concepts of helping. The principal tools used by biologists were developed by economists in the form of game theory—a framework to understand how humans should make decisions in strategic interactions [4]. A ‘game’ is a formal mathematical model of an interaction, defining the payoffs to all players. A key insight is that players' payoffs are affected by their own decisions and also by those of their partner(s). Thus, the dominant strategy depends on the strategy that is used by the partner(s). Economists assume that payoffs translate into utility and that players maximize utility. Stylized economic games were developed to study optimal decision rules. In their simplest form, these games consist of two players who can each choose between two actions, for example to cooperate or to defect. Games can be one-shot or repeated over a number of rounds. The resulting payoffs of action combinations can be captured by a 2 × 2 matrix. The matrices for well-known games [5], like the prisoner's dilemma game, the prisoner's delight game and the snowdrift game (also called hawk–dove game) are summarized in figure 1. These games were subsequently adopted by evolutionary biologists to explore when helping behaviour could be evolutionarily stable [6] in a population. Under this evolutionary approach, strategies are inherited traits that specify behaviours [7]. Rather than utility, evolutionary biologists assume that payoffs translate into fitness, with the accompanying assumption that strategies which, on average, increase fitness will be under positive selection.

Figure 1.

Three stylized economic games that differ with respect to the payoff matrix. In the prisoner's dilemma, not helping yields a higher payoff in each interaction no matter how the partner behaves, which makes helping an investment that needs to yield future benefits. Thus, iterated interactions are required for conditional helping to evolve. In the prisoner's delight, helping yields a higher payoff no matter how the partner behaves, which makes helping a self-serving action, even in a single-round game. In the snowdrift game, the best choice depends on the partner's action: help if the partner does not help and do not help if the partner helps. Under these circumstances helping is under negative frequency-dependent selection in a single-round game.

A common goal is to understand why individuals should provide help to others. Economists, never considering the genetic structure of human populations, focused on how helping may increase on average the direct fitness of that actor. This form of helping has been termed ‘mutual benefits’ [8] or ‘cooperation’ [9]. We will use the latter term in this paper, and restrict the term ‘mutualism’ to describe mutual helping between species [10]. Economists demonstrated that cooperative solutions are possible, when the number of rounds times the benefits of mutual cooperation outweigh the cost of cooperating (folk theorem [5,11]). Evolutionary theorists subsequently rediscovered this principle verbally [12] and then mathematically, albeit with limited generality [13]. Economists also showed how supply and demand determine exchange rates [14], an insight that was then incorporated into biological market theory [15,16]. Similarly, the idea that reputational effects in a communication network may affect animal behaviour [17] was foreshadowed by concepts explicitly developed by economists to understand human cooperation [18,19].

Evolutionary biology provided a major conceptual insight thanks to Hamilton's kin selection theory [2,3]. Helping may be altruistic in biological terms, by reducing the actor's lifetime reproductive success, and yet still be positively selected if helper and recipient are related (specifically, when rB − C > 0, where r = relatedness between actor and beneficiary, B = fitness benefit conferred on beneficiary and C = personal fitness cost incurred by actor [2,3]). Thus, the one-sided borrowing from biologists eventually became a fruitful dialogue, not least because cooperation and biological altruism may act simultaneously to promote selection on helping, including in humans (e.g. [20-22]). Indeed, game-theoretic approaches have become increasingly prominent in the attempt to understand the evolution of helping behaviour [7,23]. Importantly, the logic underpinning game-theoretical models of behaviour reflects general principles in evolutionary theory and may hence be applied to any species, including humans.

Empirical research on human helping as inspiration for animal research

We focus on supposed examples of cooperation based on investments. We define an investment as a behaviour that reduces the current payoff of the actor and increases the current payoff of the recipient. Cooperation based on investment appears to be vulnerable to cheaters who do not invest but receive investment from others. A vast theoretical literature has shown that higher-level selection processes (kin/group selection, interdependencies between individuals) may select against cheating. These processes have been relatively neglected in empirical studies, partly because of the difficulty of quantifying them. More ecologically motivated future research may hence reveal that some apparent investments are actually self-serving forms of helping [24]. As with theoretical concepts, empirical research on human helping has had a serious head start over similar research on non-human animals. It is impossible to summarize the existing literature on human helping adequately here. Though most of this research focuses on understanding human social behaviour only, the data and conclusions nevertheless provide inspiration to researchers studying non-human animals, who might look for similar behaviours in their own study systems.

In this context, it is important to distinguish ultimate from proximate questions [25]. Ultimate questions address the adaptive value of helping, which is rather simple: helpful strategies can only be under positive selection if they provide lifetime fitness benefits (+/+) to all participants, the exception being biological altruism (−/+) based on kin selection. Thus, from an ultimate perspective, there is no a priori reason to demarcate research aimed at understanding the evolution of costly social behaviour in humans from similar research on other species. In contrast, the proximate mechanisms underlying social decision-making can be highly diverse: genetic predispositions, physiological states and cognitive mechanisms may all interact to produce social behaviour, and humans might often use idiosyncratic proximate mechanisms to achieve cooperation. Examples of these include mentalizing, fairness preferences, cultural norms, shared intentionality, and the ability to communicate intentions using gestures (such as pointing) and language. These abilities may not be unique to humans, but they are unarguably more pronounced in humans than in any other species. Moreover, variation in proximate mechanisms can affect the means by which cooperation is achieved—and sometimes even the possibility to achieve it [26]. We therefore discuss research on ultimate and proximate explanations for costly social strategies separately.

Ultimate explanations

Humans appear to be an excellent model species to test the predictions of evolutionary game theory. Experimenters can construct precise material payoffs for any possible combination of individual decisions, decide how many rounds are played with whom, and how much information subjects obtain. To understand the adaptive significance of costly helping behaviour, many studies on humans have identified partner control mechanisms—responses to being cheated that reduce the cheater's payoff [27]. These include tit-for-tat-like reciprocity, punishment, reputation effects, partner choice and (relatedly) ostracism (e.g. [28-31]).

After an initial focus on tit-for-tat-like reciprocity (reviewed in [32]), biologists also searched for examples of these same partner control mechanisms in non-human species. Marine cleaning mutualism involving the cleaner wrasse Labroides dimidiatus provided experimental support for all these control mechanisms. Cleaners remove ectoparasites from visiting ‘client’ reef fishes [33]. Nevertheless, conflict arises because cleaners prefer to eat client mucus, which constitutes cheating. Therefore, clients have to make cleaners feed against their preference to receive a good service [34]. Partner control mechanisms become visible when clients respond to cleaners taking a bite of mucus (which correlates with clients visibly jolting in response to cleaner mouth contact). As summarized in [34], client species with access to a single cleaning station punish cleaners through aggressive chases, while clients with access to several cleaning stations terminate the interaction and visit another cleaner for their next inspection. In addition, clients arriving at a cleaning station extract information from any ongoing interaction and invite for inspection only if the cleaner behaves cooperatively. Thus, the cleaner's reputation depends on their behaviour, and they behave more cooperatively if they are observed. Finally, the larger cleaner males may also punish their female partner for cheating a jointly inspected client, a simple form of third party punishment [35] that is fine-tuned to the stakes (i.e. the quality of the client as a food source) [36].

Research on cleaning mutualism was partly inspired by classic studies on the effects of punishment and reputation on human cooperation, which highlighted that the possibility of being punished or being chosen for interactions by observers, respectively, could both promote cooperation at higher levels than when these incentives were absent (e.g. [28,30]). Meanwhile, results from the cleaner fish mutualism have, in turn, inspired subsequent studies on partner choice and asymmetric punishment in humans, for example leading us to investigate whether punishment or partner choice is a more effective incentive to cooperate when both incentives are co-present [37], and to explore whether power asymmetries increase the efficacy of punishment as a cooperation-enforcing mechanism in two-player games [38].

Unlike most non-human species, humans regularly cooperate in large groups of unrelated individuals. Economists and social scientists have therefore pioneered the study of cooperation in groups. The payoffs can be captured using public goods games, where benefits are assumed to be either a linear or a sigmoid function of investments (figure 2). Under the former assumption, the interaction is an n-player prisoner's dilemma and investments therefore risk being biologically altruistic. Where benefits are a nonlinear function of investments, then the interaction is an n-player snowdrift game (a volunteer's dilemma) and contributions are negatively frequency dependent (figure 2) [39,40]. Again, claims about human uniqueness with respect to n-player cooperation have inspired biologists interested in a comparative approach to find suitable non-human model systems in which to apply the human literature on public goods games. Importantly, the most suitable species will not necessarily be the species that are phylogenetically most related to humans, but those that routinely interact in n-player social dilemmas (with non-relatives)—such that n-player social dilemmas constitute an ecologically valid scenario. To this end, species that regularly engage in inter-group conflict may provide a promising arena. Humans promote cooperation in larger groups by providing incentives: rewarding contributors to the public good and punishing so-called free-riders [28,41]. Similarly, female vervet monkeys use these same incentives to increase male participation in inter-group conflicts [42].

Figure 2.

Public goods games. (a) Contribution to a public good creates a surplus. In an n-player prisoner's dilemma game the created value is a linear function of the amount contributed, while in a n-player snowdrift game it is nonlinear (a step function in the figure). The created value is then shared equally among players irrespective of initial contributions. (b) Case examples for the payoffs of a focal player depending on whether she contributes or defects and what her three partners are doing. In the n-player prisoner's dilemma, it is assumed that contributing costs 1 unit and generates a value of 2 units. In the n-player snowdrift it is assumed that contributing costs 1 unit and that 2 contributions are needed to produce a public good of 8 units.

One of the key difficulties in identifying n-player public goods games outside of humans is to obtain informed estimates of both the precise payoff matrices and the fitness consequences. In some cases, in contrast to the frameworks described above, individual actions appear to be self-serving and to provide public goods only as a by-product (e.g. punishment of scale-eating sabre-tooth blennies by their victims [43]; group hunting of multiple non-shared prey [44]). In contrast, many terrestrial group hunting examples involve the killing of a single large prey, where individual payoffs depend crucially on how the prey is shared rather than on the increased hunting success [45]. In such cases, payoffs are affected by ownership, contribution to the hunt, sex and/or position in the hierarchy (e.g. [46-49]), variables that are not typically considered in standard public goods games (but see [50]). Many examples of n-player public goods have been described in microbes, where the production of extracellular molecules constitutes an investment that can provide benefits to non-producers (reviewed in [51]). Since increased production typically yields diminishing benefits, many of these examples yield fitness consequences that correspond to the volunteer's dilemma payoff matrix [40]. These various case studies highlight an important issue: despite the continued focus on n-player prisoner's dilemma payoffs in human laboratory studies, many public goods in humans might also better approximate the nonlinear payoffs of snowdrift/volunteer's dilemma games [40,52]. A key priority for future research on humans is therefore to evaluate the payoffs of real-world interactions and design experiments to capture these in the laboratory.

Proximate explanations

Research on the cognitive mechanisms underpinning human helping might initially appear of little value for understanding helping in other species. This is because humans have a cognitive toolbox that is unmatched by any other species (though there is considerable debate regarding the extent to which differences are qualitative or only quantitative [53]). Many of these cognitive tools are tightly linked to/enhanced by human language, which is in itself arguably the most important tool. Spoken and written language does not only allow for basic communication about behaviour; it also facilitates negotiation, coordination, the expression of some emotions and the establishment of shared intentionality. Language is also the basis for some forms of teaching [54] and the establishment of shared cultural norms. Culture in turn provides a variety of cues that can be used to generate cooperation even between strangers. It seems highly likely that there is a tight link between our cognitive abilities and our ability to cooperate, though it remains unclear whether ecological pressures to cooperate selected for our cognitive abilities or whether these abilities created opportunities for extreme cooperation. Comparative research that evaluates what cognitive processes are used by humans and other species during social interactions might help address this question.

Claims about uniquely human cognition inspired research on animal cooperation that challenged these claims. For example, it has been proposed that humans achieve high levels of cooperation because they have a unique sense of fairness (‘inequity aversion’) and thus split payoffs according to individual contributions [55]. A large body of research has shown that rudimentary forms of disadvantageous inequity aversion—aversion against receiving less than the interaction partner(s)—may be present in some non-human species (reviewed in [56]; but see [57]). In contrast, evidence for advantageous inequity aversion—aversion against receiving more than the interaction partner(s)—is currently lacking in non-human species and is apparently not even ubiquitous in humans [58].

In contrast to cognition, endocrinological research offers straightforward opportunities for a comparative approach, as humans are just standard mammals when it comes to hormones, neurohormones or neurotransmitters. Nevertheless, social scientists have often taken the lead in exploring the effect of these substances on helping behaviour. Research on the effects of oxytocin provides a case example. Oxytocin facilitates bonding between mammalian mothers and their offspring [59]. Research on humans revealed that this function may have been co-opted for creating bonds between unrelated individuals: increased oxytocin increases trust, without increasing risky behaviour overall, and increases within-group cooperation and between-group competition [60]. Oxytocin also mediates helping between unrelated individuals in several non-human mammals, including chimpanzees [61], dogs [62], vampire bats [63] and voles [64]. We note however that the robustness of various findings—in particular those based on exogenous application of oxytocin—is strongly debated (see [65]), and the jury is still out on how central-nervous and peripheral oxytocin mediates social behaviour in humans and other species. These concerns notwithstanding, research on endocrinological mechanisms underpinning social behaviour has been and will remain an interdisciplinary project.

Towards a more ecologically valid approach to helping in humans and other species

We have repeatedly emphasized how influential research on human helping was for biological research on other species. We believe that there is great potential for ever closer exchange of ideas and methods. Most importantly, biology has a long history of thinking about the problem of laboratory artefacts. While this does not mean that all biological research on helping is ecologically relevant [66], we will now highlight various important concerns about research on human helping from an ecological perspective. We postulate that further progress will depend on empirical data informing models rather than on experiments fitted to the assumptions of models.

First, we note that most experimental research taking an evolutionary approach to understand human cooperation precludes key features of the human cognitive toolbox, such as language, shared intentionality and shared group identity. This is because empiricists typically develop experiments in accordance with evolutionary game theory, which focuses on strategies rather than underlying mechanisms [67]. For instance, as theoretical models do not incorporate communication, subjects are typically prevented from talking to each other in experiments. Also, high levels of cooperation in humans typically occur between friends, colleagues or culturally created in-groups, while experiments often follow model assumptions and hence involve anonymous interactions between strangers. Thus, many experimental studies on humans are designed to test the predictions of general evolutionary game theory models rather than designed to explain how humans achieve extremely high levels of cooperation. As a consequence, we propose that typical economic experiments only yield baseline levels of human cooperation and that such levels may also be observed in various other species [68]. Of course, cooperation could similarly decrease under different conditions. A recent study [69] found that human cooperation increased under conditions in which subjects could talk to each other, in particular when in-group identity was triggered. We expect that the larger the group size and/or the incentive to cheat and/or the challenge to coordinate, the more important the human cognitive toolbox becomes to achieve high levels of cooperation. Studies that have allowed for communication during experiments have shown that communication can increase cooperation, either via gossip to deter cheating [70] or by allowing subjects to more efficiently coordinate actions [71]. Varying relationship quality between subjects will also be likely to yield additional insights, both in humans and other species.

An important goal for a biological approach to cooperation is to determine how the findings of abstract laboratory experiments apply in the real world [24,51,65,66]. Economic games that are typically used to study human behaviour are theory-driven but highly artificial. These abstract games can allow us to identify with a high degree of control how the various pillars that structure social interactions (e.g. anonymity, punishment, partner choice) directionally affect behaviour under the assumption that ceteris paribus these general effects apply in all settings [72]. Simple abstract games also allow a method for studying and quantifying variation in helping behaviour within and across populations (e.g. [73-75]). It is also likely that excluding more human-specific features like language has encouraged the interdisciplinary dialogue as both biologists and social scientists could use similar paradigms. Nevertheless, more effort should now be aimed at identifying if and how the findings from laboratory studies translate into real-world behaviours. Failing to do so runs the risk that empirical findings serve only to test the predictions of game-theoretical models and have little real-world relevance. We illustrate this point by summarizing discussion first on the meaning of payoff matrices in general, and then on indirect reciprocity as one specific example.

Payoff matrices

It is unclear to what extent the payoffs used in standard laboratory games approximate the payoffs of interactions that occur in the real world. The assumption that payoffs correlate positively with individual fitness holds in populations that are well mixed both with respect to genetic structure and with respect to potential interaction partners. The situation changes when limited migration and overlapping generations lead to kin structure and the potential for biological altruism, and when populations are structured into demes (groups) that compete with each other through contest or scramble competition. In those cases, material payoffs often provide a poor correlate of fitness. Instead, interacting individuals might become interdependent [2,3,76,77]. Interdependence has been proposed to be key to the evolution of extreme cooperation in humans [78]. Importantly, letting two highly interdependent individuals play a one-shot game with a prisoner's dilemma payoff matrix leads to confusion because the players' best option with respect to fitness is to either fully cooperate or to cooperate at least with some probability [24]. This is because interdependency can alter the fitness consequences of a prisoner's dilemma payoff matrix in such a way that fitness can be described by a prisoner's delight game (where cooperating yields higher payoffs/fitness via by-product benefits to the partner) or by a snowdrift game (where cooperating is under negative frequency dependent selection; figure 1). For example, zebra finches, a species with obligate bi-parental care, fail to cooperate with strangers in an experiment that uses an iterated prisoner's dilemma payoff matrix, but they show rather unconditional cooperation when paired with their social partner [79], perhaps due to interdependence between social partners [31]. A major question arising from the interdependence hypothesis to explain human uniqueness in levels of cooperation [78] is hence whether human interdependence is (or was) much more pronounced than in any other species, or whether some unique cognitive tools allowed humans to create extreme mutually beneficial interdependencies between unrelated individuals.

Indirect reciprocity

Indirect reciprocity also offers a cautionary tale on the importance of ecological validity. Indirect reciprocity occurs when an investment to help a recipient yields return benefits by an investment of a third party rather than by the initial recipient. Typically, indirect reciprocity therefore involves the existence of a reputation or an image score, and assessment rules determine how different actions affect reputation. A first detailed analysis of stable decision rules was provided by Kandori [19] and extended by Ohtsuki & Iwasa [80]. However, there is mixed evidence regarding whether people actually use these stable rules to judge the actions of others. Early evidence indicated that these second-order judgement rules were too cognitively complex to be used [81], while more recent evidence has indicated that reputation assessments can be predicated on second-order information regarding the context of helpful [82] or punitive [83] behaviour. Perhaps a more fundamental concern with the importance of indirect reciprocity as a general mechanism for supporting cooperation is the lack of real-world evidence that people behave in this way (but see [84]). One key paper that claims to have demonstrated indirect reciprocity in the real world [85] instead simply demonstrates that individuals show concern for reputation, which is not the same thing, as the crucial component—individuals with good reputation receive voluntary rewards from others—is missing.

One other major mechanism by which concern for reputation could yield downstream benefits is via partner choice. There is ample real-world evidence—including from non-human species—that partner choice is an important force underpinning cooperation, and the pressure to be chosen as a partner can lead to strategic [86] (and even competitive [29,87]) investments in reputation. Laboratory studies demonstrating indirect reciprocity may therefore be tapping into psychological mechanisms aimed at striking up mutually cooperative relationships with partners that have a good reputation, even though this is not possible in most laboratory studies of indirect reciprocity. Under the logic of error management [88], one could further predict that the high payoffs of striking up just one mutually productive relationship by ‘rewarding’ a helpful individual could sustain several small investments in rewards that do not ultimately lead to a relationship (cf. [89]). Error-management strategies could therefore result in behaviours that had the appearance of ‘rewarding’ helpful individuals in one-shot encounters, but would actually function to establish productive relationships. Experiments investigating the adaptive significance of acquiring a good reputation under real-world settings are now crucial to determine the relative importance of indirect reciprocity and reputation-based partner choice as mechanisms supporting cooperation.

This discussion highlights a larger issue of experimenter demand [90] in laboratory studies of human behaviour—changes in behaviour that occur because of what the subject believes to be appropriate in that context, rather than due to intrinsic motives or preferences. Most laboratory studies of indirect reciprocity have limited the behavioural options available to players. Thus, although indirect reciprocity is observed in laboratory experiments, we cannot rule out that these behaviours result from the expression of emotions whose only possible outlet in the context of the experiment is to reward helpful others. These emotions might well produce alternative behaviours in real-world scenarios that are nevertheless prevented by the rather impoverished selection available in the laboratory. Attempts to approximate reality by giving players more options in empirical games can affect the expression of behaviour (e.g. [36,83]). We suggest that the next wave of human evolutionary behavioural sciences ought to fully embrace these complexities in order to understand how behaviour in artificial laboratory settings relates to that in the real world.

General conclusion and outlook

We certainly support the idea that studies on human helping behaviour are relevant for biological research. In the tradition of Darwin [91], the highest relevance is achieved by studies that take an explicit evolutionary approach and refer at least to some extent to empirical and/or theoretical studies on other species. This view is also reflected in papers published recently in Proceedings of the Royal Society B (see electronic supplementary material).

Potentially, a unifying framework could be developed by studying how individuals decide whether to help, to cheat, to punish or to switch partners. This issue of decision-making links function and mechanisms. There is a clear need to study these processes [92] because humans and other animals do not use the simple strategies investigated in game-theoretic models (e.g. [93-95]). To determine why not, we should study social cognition—the mechanisms by which animals acquire, process, store and act on information from other individuals [96]—in its broadest biological sense. Perception of relevant stimuli can fundamentally affect decision-making. For example, it has been proposed that humans and other animals use heuristics or rules of thumb [97] to reach decisions quickly by ignoring a portion of the available information [98]. These processes are probably routed in well-established universal learning mechanisms, such as learning based on positive or negative reinforcement [99]. Excitingly, even in humans, reinforcement learning may explain various deviations as well as conformity with payoff-maximising behaviour [100]: for example, if behavioural option A yields a small gain in most trials, positive reinforcement may cause subjects to prefer this option over a more profitable option B that yields a high reward in few trials.

Recent theoretical studies have started to explicitly model reinforcement learning over the lifetime of individuals and selection on specific reinforcement learning parameters (i.e. the change in the probability of repeating a behaviour after receiving a reward) to study the consequences on social behaviour [101,102]. The models show that selection acting on reinforcement learning can yield cooperative solutions in an iterated prisoner's dilemma as well as consistent co-operators and defectors within pairs playing a repeated snowdrift game. What is still missing in the models is an integration of perceptual aspects. Early ethologists pointed out that learning needs to be studied within evolutionary history (i.e. within the ecology of a species). This is because evolution may shape the perception of species such that certain stimuli are more likely than others to elicit learning through positive or negative reinforcement. To give a concrete example, cleaner wrasse need to give priority to visitor clients over resident clients, as the former would otherwise swim off and visit another cleaner [34]. Species can be identified by their colour patterns and body shape, while the food (various species of ectoparasites) is highly overlapping between residents and visitors. As a consequence, cleaners can readily learn to preferentially approach an ephemeral food plate that differs from a permanent food plate only with respect to colour and patterns, a task that is extremely difficult for primates as well as rats and pigeons [103-105]. However, if the food items are coloured differently, or if food is hidden under cups of different colours, capuchin monkeys readily learn to prefer the ephemeral food source [106]. Taken together, the studies show that performance in the same biological market task varies according to a species's ability to perceive the relevant stimulus. Perception, strength of perceived reinforcement on actions and memory capacities (declarative, episodic or simply emotional) will all contribute to variation in cooperation within and between species.

In conclusion, we affirm that humans are just another species to test evolutionary theory. Research on human cooperation that takes a clear ecological or evolutionary perspective is as biologically relevant as research on any other species. Although helping has long been considered as an evolutionary puzzle that needs to reconciled with evolutionary theory and its emphasis on egoism, we believe that this puzzle has already been solved in the sense that there are many concepts that provide conditions under which biological altruism and cooperation can be favoured by selection. What is currently lacking is a general framework that can explain variation in helping tendencies within and between species, with human cooperation being the single most idiosyncratic data point. The current puzzle is thus why human cooperation is so unique on a quantitative level—and, moreover, why we also observe such striking variation in cooperation among different human individuals, groups and societies. We have argued that to solve the puzzle, we need to be more explicit about the links between cooperation and ecology and between cooperation and cognition (see also [107]). Both issues warrant a comparative approach, making research on human cooperation an interdisciplinary project of high biological relevance.