The evolution of conditional moral assessment in indirect reciprocity.

Indirect reciprocity is a major mechanism in the maintenance of cooperation among unrelated individuals. Indirect reciprocity leads to conditional cooperation according to social norms that discriminate the good (those who deserve to be rewarded with help) and the bad (those who should be punished by refusal of help). Despite intensive research, however, there is no definitive consensus on what social norms best promote cooperation through indirect reciprocity, and it remains unclear even how those who refuse to help the bad should be assessed. Here, we propose a new simple norm called "Staying" that prescribes abstaining from assessment. Under the Staying norm, the image of the person who makes the decision to give help stays the same as in the last assessment if the person on the receiving end has a bad image. In this case, the choice about whether or not to give help to the potential receiver does not affect the image of the potential giver. We analyze the Staying norm in terms of evolutionary game theory and demonstrate that Staying is most effective in establishing cooperation compared to the prevailing social norms, which rely on constant monitoring and unconditional assessment. The application of Staying suggests that the strict application of moral judgment is limited.

How to get unrelated people to mutually cooperate is a fundamental issue in today’s highly mobile society. The last decades have seen researchers exploring indirect reciprocity, which is a major mechanism in the maintenance of cooperation between non-relatives12345678910. The emergence of cooperation by indirect reciprocity can be summarized as, “I will help you if you have helped someone”10. Because helping is costly, however, self-interested recipients of help tend to freeload off others without further reciprocation, and unconditional cooperation is unlikely to evolve unless a specific supportive mechanism is provided11. Conditional cooperation, a main paradigm for exploring cooperation12, suggests that cooperation should be channeled to those who deserve help by using social network assessment systems, such as reputation or gossip media131415161718.

How should one assess others’ past behaviors? The simplest social norm, called Scoring, assesses those who give and refuse to give help as good and bad, respectively1920. This norm depends only on individuals’ previous actions. Since the seminal study of Nowak and Sigmund10, Scoring has been investigated in terms of evolutionary game theory, primarily using a donor–recipient giving game.

Unconditionally applying the Scoring norm raises a key question: Is it morally or socially acceptable to refuse to help someone with a bad image? This point is Scoring’s Achilles’ heel in the typical good-or-bad binary system10. By definition, when a discriminator refuses to help a potential opponent with a bad image, the discriminator’s image decisions become clouded. Thus, in the Scoring norm, a bad image is contagious. Although helping can redress the discriminator’s image, a bad image may cause the discriminator to undergo rejection by other discriminators. Even slight involuntary errors can damage a discriminator’s image and thus payoff. The Scoring norm, therefore, results in the mutual defection of all players2122; this is referred as the “Scoring dilemma”.

To address this dilemma, social norms have been developed that distinguish between justified and unjustified defection by accounting for the recipient’s image1. In the case of a bad recipient being refused help, such refusal should not damage the donor’s image (i.e., justified defection)12122. Indeed, the top eight social norms, identified from 4,096 candidate strategies by systematic research (called the “leading eight”)2324 share a common relevant feature—if a good donor refuses to help a bad recipient, the donor is assigned a good image (see Table S1).

Although the leading social norms are highly sophisticated, they are thus cognitively costly. Indeed, all of the leading eight rely on (i) the donor’s last action and (ii) the recipient’s last image (i.e., second-order social norms), and six of them also rely on (iii) the donor’s last image (i.e., third-order social norms)25. Accounting for the image of both players would be rational in theory yet may overtax individuals in practice. Empirical studies on indirect reciprocity games have reported experimental results that many participants’ assessments have appeared to rely only on the player’s actions, not also on the images a player views26. We posit that there is a simpler method than that provided by the leading eight.

To tackle the Scoring dilemma, we formulate an opposite approach from prior paradigms by not applying higher-order social norms. We consider the effects of ignoring anxiety-producing stimuli from controversial interactions (“selective inattention”27). Assessing a donor who interacts with a recipient who has a bad image is a difficult task and is likely to lead an observer to feel regret about his or her assessment, regardless of whether the assessment is determined to be good or bad. This stressful situation may lead an observer to intuitively prefer inattention to interaction, that is, to abstain from making assessments. There is much supportive evidence of inattentive behavior by experiments2829 and field research303132. Recent experimental data also suggest that a substantial fraction of subjects selectively consider the information of donor and recipient (that is, first- and second-order information) when making moralistic decisions33. Thus, we assume conditional observation on the opponent’s image3435 (Fig. 1). The assessment system abstains from observation and thus also from assessment of whether or not refusal of help to the bad recipient is justified. Conditional assessment can prevent damage to the donor’s image as it substantially generalizes the standard framework of indirect reciprocity to a meta-choice of {Assess, Preserve}. In the Assess case, the assessment is made according to the specific social norm, whereas in the Preserve case, the pre-existing image of the focal player is kept as is. The Preserve option is applicable for a broad range of assessment systems. We apply a conditional assessment for Scoring, leading to a new social norm—to perform an assessment as Scoring when a potential recipient has a good image; otherwise, to abstain from assessment. We call this new norm Staying.

Figure 1

Conditional assessment in giving games.

In the Staying rule, (a) the observer assesses the donor’s image score, if the recipient has a good image; (b) otherwise, the observer does not assess the donor’s image score, which stays the same.

Results

We model our paradigm on the giving game in which the donor player has an opportunity to help the recipient player at a personal cost c > 0; if the donor helps, then the recipient earns benefit b > 0, with b > c. For simplicity, we assume that all discriminators share the same information about all personal images provided by this unique assessment system. We then consider both (i) implementation error2136, by which intentional help involuntarily fails with probability e1, and (ii) assessment error25, by which the assessment system mistakenly assesses a good donor as bad or a bad donor as good, with probability e2.

We compare Staying with the four most prevailing social norms103738: Scoring1920394041, Simple-standing12122, Stern-judging337, and Shunning542 (Table 1). Simple-standing and Stern-judging are the only two second-order social norms among the leading eight. When a donor refuses to help a bad recipient, Simple-standing and Stern-judging assess the donor as good, whereas Scoring and Shunning evaluate the donor as bad. Simple-standing, the most tolerant norm, assigns a good image to a person helping an individual evaluated as bad, and under Shunning, the most strict norm, a bad image is assigned to a donor who does not help a potential recipient evaluated as bad. In contrast with the four most prevailing social norms, no assessment is performed under Staying; the player’s pre-existing image is simply preserved.

Table 1

How social norms make moral assessments in giving games.

Conditions	Recipient’s image	G	G	B	B
	Donor’s action	C	D	C	D
Assessment: What does the donor image look like?	Staying	G	B	P	P
	Scoring	G	B	G	B
	Simple-standing	G	B	G	G
	Stern-judging	G	B	B	G
	Shunning	G	B	B	B

“G” and “B” describe a good and bad image, respectively. “C” and “D” denote an action to help and to refuse to help, respectively. “P” means the image of a donor remains unchanged (“Preserve”).

To study the evolutionary effects of these different social norms, we assume distinct strategies in very large populations: cooperators, defectors, and discriminators. Cooperators unconditionally give help; defectors unconditionally refuse to help; and discriminators, irrespective of the social norm, give and refuse help to good and bad recipients, respectively. In this study, we investigate the replicator dynamics describing the tendency whereby strategies that result in above-average earnings grow in frequency3943. We note that the three homogeneous population states of cooperators, defectors, and discriminators are trivial equilibria of the replicator dynamics. We further assume that image updating is much faster than the time scale of game interactions so that we can study the replicator dynamics at a stationary state of the image system43. More detail is provided in Methods.

We first describe the Staying paradigm (Fig. 2a). Staying results in a defector–discriminator mixed equilibrium R with the fraction of discriminators

Figure 2

Evolution of indirect reciprocity with different social norms.

The triangles describe a simplex of the state space {(x, y, z): x + y + z = 1}, where x, y, z ≥ 0 denote the frequencies of cooperators, defectors, and discriminators, respectively. Each node (X, Y, or Z: x, y, or z = 1) of the triangle corresponds to the homogeneous state of each specific strategy. (a) Under Staying, discriminators always are better off than cooperators. Thus, cooperators will vanish and the population will eventually converge to either node Z or Y. (b) Under Scoring, a continuum of equilibria connects boundary attractor Q and repeller R. The population drifts along the continuum and moves close to R, eventually attaining node Y. (c–e) Under Simple-standing, Stern-judging, or Shunning, the dynamics are qualitatively similar to those in (a). The basin of attraction for node Z is wider in (a) than in (b–e). Parameters: c = 1, b = 1.5, e1 = e2 = 0.01. R corresponds approximately to z = 0.02 in (a) or z = 0.66 in (b–e).

where λ = (1 − e1)(1 − 2e2) describes social visibility, the probability that a donor’s intention to cooperate is clearly recognized by an observer. There is no interior equilibrium that consists of all three strategies. Equilibrium R consists exclusively of defectors and discriminators and is a unique non-trivial equilibrium. Equilibrium R is unstable, and its z value indicates the minimal frequency of discriminators required to invade then take over a population of defectors. For a sufficiently small assessment error e2, equilibrium R appears in the state space, and depending on the initial conditions, the ending population consists exclusively of either defectors or discriminators. As the assessment error e2 gets close to 0, equilibrium R approaches the state of 100% defectors, and the range of initial conditions leading the population to evolve to the state of 100% discriminators expands (see Methods for detailed analysis).

The other four rules examined are Scoring, Simple-standing, Stern-judging, and Shunning (Fig. 2b–e). In these cases, for a sufficiently small cost–benefit ratio c/b, the evolutionary dynamics result in an unstable mixed equilibrium R of defectors and discriminators, with the fraction of discriminators

This value is common across Scoring, Simple-standing, Stern-judging, and Shunning, yet generically different from that of Staying in eq. (1). The evolutionary dynamics with Scoring, Simple-standing, Stern-judging, and Shunning are all bi-stable, similarly to that with Staying (see Supplementary Information, Text S1 for detailed analysis). In Simple-standing, Stern-judging, and Shunning, similarly, there is no interior equilibrium, and R is a unique non-trivial equilibrium. In contrast to this, in Scoring there can be a continuum of equilibria in the interior state space, and R is an end of the continuum (Fig. 2b).

It follows from eqs (1) and (2) that for e2 < 1 − c/b, the range of initial conditions leading to selecting for discriminators between the two strategies is wider for Staying than for the other four norms. Note that in any case of Scoring, Simple-standing, Stern-judging, and Shunning, a fraction of discriminators less than c/b is incapable of invading successfully. In striking contrast to this, for a sufficiently small assessment error e2, Staying can thrive even if the initial fraction of discriminators is very low taking over the population of defectors, irrespective of the cost–benefit ratio.

Discussion

Most theory on the evolution of cooperation by indirect reciprocity is based on unconditional assessment. Evolutionary study on conditional assessment has started mainly by individual-based simulations, in which the corresponding assessment rule was named “us-TFT”3435. In this paper, we fully analyze Staying, which is characterized by conditional assessment, and reveal that discriminators with Staying are more likely to invade the population of defectors than those with the four most prevailing social norms of indirect reciprocity103738: Scoring1920394041, Simple-standing12122, Stern-judging337, and Shunning542 (Table 1). In mutual defection, within the population of defectors, either Simple-standing or Stern-judging leads defectors evaluated as bad to look good and then to exploit help from other discriminators. In contrast, under the Scoring or Shunning norm, discriminators are evaluated as bad as a result of interacting with defectors, leading to rejection by other discriminators; this is the main reason why the four social norms Scoring, Simple-standing, Stern-judging, and Shunning are unlikely to emerge. In contrast, Staying can leave the images of good discriminators and bad defectors intact; this enables discriminators to channel their cooperation and subvert the stalemate of mutual defection even with a small perturbation of the population state (which is on the order of assessment errors; see eq. (2)).

The advantages of Staying are not limited to the emergence of effectiveness. A third-order social norm in the leading eight, called Strict-standing23 (or L7 ref. 43), can provide the same image dynamics as Staying (see Table S1). This fact indicates that Strict-standing is as evolutionarily stable as Staying; however, Strict-standing and Staying are conceptually different. Staying preserves the donor’s image as a result of abstention from observation; in contrast, Strict-standing reassigns the previous image during execution of the observation. The leading eight, which share the obligatory nature of unconditional observation, are less advantageous than Staying in terms of the load and error of cognitive process and information transfer. As such, conditional assessment, a property that characterizes Staying, can facilitate savings on the cost associated with running moral assessment systems. This would help mitigate the problem of second-order free riders who fail to contribute to costly assessments and only rely on assessment information provided or financed by others4445.

Our model can serve as a point of departure for investigating the effects of conditional assessment in various situations relevant to the evolution and development of human cooperation. Our model resembles a simplified top-down situation in which a unique media source delivers a story to infinitely large followers, who share the same assessment. The top-down situation considered has been extended in investigating situations in which assessment criteria and perception process vary by different individuals4647, populations are finite and structured48495051, and individuals often interact by both direct and indirect reciprocity5253, as in small villages with gossip systems. Also, advances in information technology (IT) would promote the broad use of assessment systems with multivalued scores1954 or continuously varying scores55, which otherwise usually require more cognitive load than do the binary assessment models considered. Thus, the results of future work that examines conditional assessment in indirect reciprocity by local gossip networks or sophisticated IT-aided rating, as such, will be fascinating.

Refusing to help the bad is difficult to morally assess. The situation causes a variety of controversial opinions and, viewed as a kind of punishment, is related to the argument about the need to consider the reputation of the punisher5657585960616263. Previous study of indirect reciprocity answered strictly to this situation with a clear standard of justice; in contrast, Staying suspends the application of a scoring rule to this situation. Staying can be seen as a social norm applied loosely to some extent. As is known, the application of the law is difficult, and judicial discretion sometimes has to work. The controversy has been continuing between two principles. One recognizes that a law is sometimes forced to be applied loosely and tolerates judgment based on the judge’s belief64. The other inhibits the judicial discretion and requires a judge to apply a law strictly65. We unveiled the excellence of Staying in forming social order. The findings suggest a limitation of the strict application of rules. In this sense, our study implies that the evolutionary study of indirect reciprocity can contribute to a further understanding of social norms and the law.

Methods

Indirect reciprocity in the giving game

The main model is based on the standard framework for the evolution of indirect reciprocity232425. Using this framework, discriminators are given a strategy by an assessment rule (called social norm) combined with an action rule. We base indirect reciprocity on the giving game, which is a two-player donation game in which one player acts as a donor and the other a recipient. The donor is given the opportunity to choose to help the recipient at a personal cost. The recipient can only receive help from the donor, if any is forthcoming. In other words, there is no option to reject help.

In this giving game, action rules prescribe to discriminators who are acting as potential donors how to respond to a potential recipient in a specific situation depending on the last image scores of both donor and recipient. We consider a simple model in which each individual is endowed with a binary image score of “good” or “bad.” The action rule we apply to discriminators is to give help to a good recipient or to refuse to help a bad one, unless otherwise specified. After observing every interaction in the giving game, discriminators assign the donor’s image by following the specific social norm, which is a function of (i) the donor’s last action, (ii) the recipient’s last image, and (iii) the donor’s last image. When depending only on (i), the rule is called first order, when depending on both (i) and (ii), it is called second order, and when depending on (i), (ii), and (iii), it is called third order23.

Conditional assessment

In the present study, we examine conditional assessment3435, which specifies a meta-choice of “Assess” or “Preserve.” When choosing Assess, discriminators assign either a good or a bad image to a potential donor; when choosing Preserve, discriminators abstain from assessment. Based on this concept of conditional assessment, we introduce “Staying,” which is a new social norm specifying that when a potential recipient is good, the donor’s image should be assessed, as is done under Scoring, and when a potential recipient is bad, the donor’s image should be preserved (that is, left unchanged).

To understand Staying, we also comparatively explore all first- and second-order social norms that take into account the recipient’s last image and the donor’s last action103738. In particular, we focus on the four most prevailing social norms: Scoring1920394041, Simple-standing12122, Stern-judging337, and Shunning542. Scoring is the best-known first-order social norm, and the other three are second-order social norms. Tables 1 and S1 provide full details of these social norms. When assessing the use of Staying and the four norms, the consensus is that if a recipient has a good image, a good image should be assigned to those who helped and a bad image to those who did not help, the same as in the simplest case, Scoring.

Observation, information, and errors

In the model, we consider both public information and indirect observation. We assume indirect observation as players with the same social norm adopting and equally sharing the same image of a focal player, which has been provided by a representative observer. For simplicity, we also assume perfect information in which the probability that players know the image of a potential recipient is 100%. In addition, we consider both implementation error213638 and assessment error25. We denote by e1 the probability that a player who intends to give help but fails to do so. On the other hand, for a player who intends to refuse to help, no implementation error occurs in the intentional refusal, and the player inevitably does so. The unilaterality of the implementation error, as such, can occur, in particular, when there is a lack of resources for helping, in which case it matters for an individual who intends to help (leading to the “phenotypic defector”36), yet does not for an individual who intends to refuse to help. We denote by e2 the probability that an observer mistakenly assigns a bad image to a donor who should get a good one or assigns a good image to a donor who should get a bad one. The assessment error e2 can be considered small enough (<1/2). The combination of the implementation and assessment errors is often considered in the theoretical study of the leading social norms for the evolution of cooperation by indirect reciprocity232425.

Evolutionary dynamics

To study the evolutionary dynamics of discriminators, we employ a continuous-entry model. An individual’s social learning (or birth and death) sometimes happens, and when it does, this changes the strategy distribution in the population3958. For analytic simplicity, we consider that in an individual’s lifetime, that individual infinitely plays the one-round giving game with different opponents. We also consider an infinitely large population to examine replicator dynamics43, which, in general, are described as dx/dt = x(P − P), where x denotes the relative frequency of strategy S; P is the expected payoff for strategy S, given by the limit in the mean of the payoff per round for the strategy; and P is the average payoff over the population, given by ∑xP. We note that each homogeneous state with x = 1 is a trivial equilibrium of the replicator dynamics. First, we examine three strategies: discriminators [S = Z], cooperators [X], and defectors [Y]. Cooperators unconditionally intend to help a potential recipient, and in contrast, defectors unconditionally intend not to help a potential recipient.

Image dynamics

To describe the dynamics of image scores, both good and bad, we use g to denote the frequency of individuals with a good image among individuals adopting the same strategy S, and we use g to denote the average fraction of individuals with good images over the population; thus, g = xg + yg + zg. In addition, we use g to denote the probability that a good image is assigned to a potential donor who adopts strategy S and also faces a potential recipient with an image score I = good [G] or bad [B]. The population size is very large, so we assume that the composition of the population does not change between consecutive one-round giving games3943. Thus, the frequencies of good players satisfy

Staying norm

We begin by analyzing the Staying norm, in which case eq. (3) is described as

where ε = (1 − e1)(1 − e2) + e1e2. In eq. (4), the first term εg in the sum for g or g describes the probability that a cooperator or discriminator who faces a good recipient (probability g) is assigned a good image by giving help with no errors (probability (1 − e1)(1 − e2)) or by failing to give help with both errors (probability e1e2). For g, the term e2g in the sum expresses the probability that a defector who faces a good recipient (probability g) is assigned a good image through assessment errors (probability e2). The second term in the sum, g(1 − g), describes the probability that a donor who faces a bad recipient (probability 1 − g) is assigned a good image. In this case, according to the definition of Staying, the probability of finding a good discriminator should remain unchanged as g. Solving these equations leads to g = e2, g = g = ε, and thus g = e2y + ε(1 − y).

Then, the expected payoffs are given by

This yields

For x = 0, that is, on edge YZ in Fig. 2a, we have

This results in z = z such that it satisfies P − P = 0, leading to

Considering that 0 < e2 < 1/2 and b > c, a boundary equilibrium R with (x, y, z) = (0, 1 − z, z enters the edge YZ for the typical parameter settings (b > c, and e1 and e2 are sufficiently small). Along the edge, the replicator dynamics are described as dz/dt = z(1 − z)(P − P), and this yields dz/dt < 0 for z < z) or dz/dt > 0 for z > z. Equilibrium R is repelling along the edge and divides the edge into the basins of attraction for the homogeneous states of defectors (y = 1) and discriminators (z = 1).

Next, we turn to the payoff difference between cooperators and discriminators

Considering g = g = ε, it follows that

where the average frequency of good g is less than 1 when both errors are non-zero. This means that for the typical parameter settings, cooperators are dominated by discriminators and defectors, thus leading the population to converge to edge YZ (x = 0). It follows that the homogeneous state of discriminators (z = 1) is evolutionarily stable for sufficiently small errors (see Fig. 2a) and becomes globally stable when there is no assessment error, e2 = 0.

Additional Information

How to cite this article: Sasaki, T. et al. The evolution of conditional moral assessment in indirect reciprocity. Sci. Rep. 7, 41870; doi: 10.1038/srep41870 (2017).

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.