Multi-strategy evolutionary games: A Markov chain approach.

Interacting strategies in evolutionary games is studied analytically in a well-mixed population using a Markov chain method. By establishing a correspondence between an evolutionary game and Markov chain dynamics, we show that results obtained from the fundamental matrix method in Markov chain dynamics are equivalent to corresponding ones in the evolutionary game. In the conventional fundamental matrix method, quantities like fixation probability and fixation time are calculable. Using a theorem in the fundamental matrix method, conditional fixation time in the absorbing Markov chain is calculable. Also, in the ergodic Markov chain, the stationary probability distribution that describes the Markov chain's stationary state is calculable analytically. Finally, the Rock, scissor, paper evolutionary game are evaluated as an example, and the results of the analytical method and simulations are compared. Using this analytical method saves time and computational facility compared to prevalent simulation methods.

Introduction

Today, Evolutionary Game theory (EGT) is a progressive topic in many branches of science from economy to biology [1-10]. EGT provides powerful tools for many problems in which the system’s dynamics depend on the interaction between agents. The interactions between strategies are often described by evolutionary games. The performance of strategies in evolutionary games is determined by the game’s payoff matrix, which determines each strategy’s spread rate. Greater payoff in the game leads to more tendency to spread in the population for any strategy. In an infinite well-mixed population, dynamics of the system is governed by a deterministic equation called replicator equation [11, 12], but in a finite population the dynamics is stochastic [13-21].

In a stochastic evolutionary game, the population is divided into several strategies and individuals interact with each other based on their strategies. The process is advanced by discrete time steps. In each time step, the frequency of strategies changes by one or remains unchanged. The game’s payoff matrix and frequency of each strategy identify the probability of events at each time step. Another factor that influences the dynamics of the population is the update rule. Update rule identifies how payoff matrix and frequencies distribute the probabilities of events in each time step. Depending on the update rule, the evolutionary game can be stopped when one of the strategies overcomes all other strategies (fixation), or continues forever. The structure of the population can also affect the dynamics of population. Unfolding an evolutionary game in graph-structured populations is the subject of many investigation [13, 22–29]. Cooperative behaviors in games like public good game or prisoner’s dilemma is a charming topic in evolutionary games investitations [30-36].

In stochastic evolutionary games, the fixation of a strategy is the favorite subject. Numerical simulation is the subject of many studies in finite populations [37-40]; also there are many investigations that evaluated the dynamics of evolutionary games analytically [18, 41–46]. In analytical evaluation, the evolutionary process is often considered as a generalization of the Moran process [47], and it has been done for games with two strategies. The most famous analytical method for analyzing evolutionary games is the recursive equation method [48, 49]. In this method, two interesting quantities, fixation probability and fixation time obtain in terms of finite series. Evolutionary games with more than two strategies are not studied analytically so far.

Considering the individual’s mutation, the population’s dynamics are governed by an evolutionary game with no fixation strategy. So, after many time steps, the configuration of the population reaches a stable state. This steady state is described by a stationary probability distribution which determines after a long run, each configuration of population how much is possible. In both cases (games with fixed strategies and games with no fixed strategies), as the number of strategies increases, more time and computational facilities are needed for simulation of the evolutionary game, so proposing an analytical method for evaluating evolutionary games with more than two strategies is helpful. This study aims to provide an analytical method for obtaining concepts in evolutionary games that getting them by simulation takes long time and needs extensive computational facilities.

Markov chain method has been used for analyzing evolutionary games sincessfully [50-52] but it has never been used in an organized and intensive way. In this paper we stabilize the Markov chain method as a reliable method for evaluating evolutionary games. In this method corresponding to each evolutionary game, a Markov chain is introduced. Essential concepts in evolutionary games such as fixation probability, conditional fixation time, and stationary probability distribution are related to concepts in the Markov chain. Using the fundamental matrix method in the equivalence Markov chain, we can calculate essential concepts in the Markov chain, which leads to calculating essential quantities in the evolutionary game. Although this method is designed for a discrete-time system, it could be used for a time-continuous system by considering some approximation.

The organization of the paper is as follows. In general method section we review the Markov chain method and explain a practical theorem for obtaining conditional fixation times which is proven in tha Appendix. In evolutionary game section we establish correspondence between evolutionary games and Markov chains and will clarify how essential concepts in evolutionary games can be obtained from the fundamental matrix method. In result and discussion we apply our approach to an evolutionary game with three strategies. Here the famous rock, scissor, paper evolutionary game is used and results of analytical method and simulations are compared to each other. Conclusion is devoted to a summary and concluding remarks.

General method

Markov chain and fundamental matrix method

In this section, We briefly review the fundamental matrix method in Markov chains and obtain a formula for calculating conditional absorption time. In the next section, by establishing a correspondence between states of the Markov chain and states of the evolutionary game, this theorem provides handy information about the dynamic of the evolutionary process in the fixation path.

A Markov chain is described as S set of states S = {s1, s2, s3, …} and a process which starts in one of these states and move to another state. If the chain is currently in state s, then it moves to state s with probability denote by p. The point is that the probability that the chain moves from state s to state s depends on the initial state s and final state s not upon which states the chain was in before the state s. The probabilities p constructed the transition matrix P. If v be a vector that determines the probability distribution in step i, then probability distribution in step i + 1 is v = v P. If there are states in the Markov chain that leaving these states is impossible, these states are called absorbing states and Markov chain called absorbing Markov chain. If i be an absorbing state, then p = 1 and when the chain is in this state, the Markov chain ends. Other states which are not absorbing are called transient. There are three valuable concepts related to absorbing Markov chain. The first is the probability that the chain starts from transient state i, will be absorbed in absorbing state j (b). The second is absorption time (t), the expected number of steps before the chain is absorbed in one of absorbing states, given that the chain starts from state i and the last concept is conditional absorption time (τ), the expected number of steps before the chain is absorbed in the absorbing state s given that the chain starts in transient state i. It is necessary to emphasize that absorption time differs from conditional absorption time. In fact, absorption time is a weighted average of conditional absorption time among different absorbing states. There is a helpful method for calculating absorption probabilities and absorption time called the fundamental matrix method. In this method, at first the transition matrix is written in the canonical form as follows: In other words, in canonical form, we labeled states so that the absorbing states consider as final states. The so-called fundamental matrix is defined as N = (I − Q)−1 and is useful to obtain absorption probabilities and absorption time. Let us define t to be the (average) absorption time of the Markov chain starting from state i and , ,… the absorption probabilities correspond to absorption states a1, a2,… starting from state i, respectively. According to the approach in Ref [53] the matrix notation can be used to denote these quantities: where in the above T is the number of transient states. Using the fundamental matrix method one can obtain absorption probabilities and times as follow: where c = (1, 1, ⋯, 1). If there is no absorbing state in Markov chain then the Markov chain is called ergodic. In the ergodic Markov chain, it is possible to go from every state to every other state after finite steps. If P is the transition matrix of the ergodic Markov chain then for n → ∞ the P approach a limiting matrix W with all rows the same vector w, called fixed row vector for P. It means after a long run, the Markov chain reaches an equilibrium which probability that chain be in state j determine by w. Obviously, wP = w means w is the left null vector of matrix P − I.

In other words, the fixed row vector of P is left eigenvector of P with eigenvalue one. The fundamental matrix method does not represent a recipe for calculating the conditional fixation time. Now we describe a theorem to calculate the conditional fixation time for any absorbing state by adding some details to the fundamental matrix method.

Theorem: Let τ be the conditional fixation time for absorption in absorbing state a given that Markov chain starts from transient state i. Using matrix notation, we have where in above equation and T is the number of transition states.

The proof of this theorem present in the Appendix. Also there is a proof with different notation for the theorem in Ref [54].

Evolutionary games corresponding Marokov chains

This section develops a method based on correspondence between Markov chain dynamics and evolutionary game dynamics. This correspondence provides a sound mathematical device for analyzing evolutionary games.

Consider a population with size N which n strategies interact with each other according to a payoff matrix

In each time step, the expected payoff of each strategy is obtained in terms of frequency of strategies and payoff matrix as where π(i) is excepted payoff of strategy i and f is frequency of strategy j. Generally, the expected payoff interpreted as the fitness of strategy in evolutionary game theory, in other words, strategies spread with rates that are proportional to their expected payoff. There are many ways to obtain the fitness of a strategy from its expected payoff, like an exponential payoff to fitness mapping. Depending on the update rule of dynamic, there is a possibility that the evolutionary process leads to the fixation of a strategy which means one strategy overcomes other strategies and occupies the whole population forever. In evolutionary games with fixation strategies, three concepts are noteworthy. Fixation probability, the probability that a strategy fix in population, the fixation time, the average steps of time that an evolutionary process fixed to one of its fixation strategies and conditional fixation time, the average steps of time that evolutionary game fixed in a specific strategy. Update rule could be in such a way that there is no possibility for any strategy that overcomes other strategies forever. In this situation, after a long run with many steps of time, the population reaches to stable condition, which means the probability that the evolutionary process is in each state approach a stationary value.

In the evolutionary process the state of population describe by frequency of each strategy like {f, f, f, …} which f + f + …f = N. Direct calculation shows that the number of states is In each time step, one strategy is chosen for reproduction and replaces its offspring inplace another strategy. In other words, in each step, the frequency of a strategy increases by one, and frequency of another strategy decreases by one, and the state of the evolutionary game changes. Update rule of the evolutionary game determines which strategy has a higher probability of reproduction and which strategy has a higher probability of being replaced. It is possible that the strategy that is chosen for reproduction and the strategy that vanishes be the same, in this situation, the state of the evolutionary game remain unchanged.

Corresponding to each evolutionary game with l fixation strategies, there is a Markov chain with l absorption states, also, corresponding to each evolutionary game with no fixation strategy, there is an ergodic Markov chain. States in evolutionary game dynamic can be considered as Markov chain states. Transition matrix of corresponding Markov chain obtains by update rule of the evolutionary game.

Fixation probability, Fixation time, and conditional fixation time in the evolutionary game correspond to absorption probability, absorption time, and conditional absorption time in the Markov chain. Since we have the fundamental method in Markov chain theory, this duality between Markov chain dynamics and evolutionary game dynamics is so helpful to analyze the evolutionary games. In games with fixation strategies using the theorem of section, one can obtain conditional fixation time for each strategy, and in evolutionary games with no fixation strategy, the stationary probability distribution of strategies is obtained by calculating the left null vector of matrix P − I. In the next section, we use this correspondence for analyzing rock, scissor paper game.

Results and discussion

As an example of what we said, in this section, we analyze the most famous game with three strategies, the rock, scissors, paper game (RSP game) [4, 55–67]. In the RSP game, each strategy overcomes the next one cyclically.

In the real world, coexistence of many species occurs over three competing species interacting with each other like the rock-paper-scissors game. According to the anticipation of some models, the coexistence of all three competitors is possible if the interaction between them becomes local. In reference [68], the coexistence of three populations of Escherichia coli was empirically studied. According to this, coexistence is preserved when the interaction between species is localized. When dispersal and interaction are nonlocal, the diversity is lost, and one species occupies the whole population. Another example of the rock-paper-scissors evolutionary game in biology is changing in the frequency of adult side-blotched lizards. In reference [69], the authors studied the frequencies of three side-blotched lizard morph from 1990-95. According to their observations, the fitness of each morph is dependent on other morphs. They suggest an evolutionary stable strategy model which predicts each morph frequency. Estimating parameters of payoff matrix of RSP game by field data, the model predicted the morphs oscillation frequencies.

Without loss of generality, the payoff matrix of RSP game can be depicted as follow where a, b > 0. At first, we set the update rule so that the evolutionary process ends when the whole population occupies by one strategy. Therefore, the Markov chain is an absorbing Markov chain. By changing the update rule of the evolutionary game, we establish the possibility of mutation, which means when a strategy extincts, there is a probability that other strategies mutate to extincted strategy and it appears in the population again. In this situation, the evolutionary process never ends but after a long run, it reaches a stable position, and the corresponding Markov chain is an ergodic Markov chain.

RSP game with absorbing states

Consider a population with size N that each member of the population can be one of three types rock, scissor, and paper. We denote the three strategies rock, scissors, and paper as 1, 2 and 3, respectively. The evolutionary process runs upon a birth and death update rule. According to this update rule, one member of the population is chosen for reproduction at each step of time. The chosen member selects randomly another member of the population to be replaced with its offspring. The probabilities of selection for reproduction and being replaced for each strategy are proportional to their frequency. The expected payoff of each strategy is involved in the update rule via the Fermi distribution function [70]. The probability that in each step of time, strategy k replaced with strategy l is which f and f are frequency of strategies k and l respectively and F is fermi function define as follow where β > 0 is constant. The expected payoff for strategies (π) can be calculate for k = 1, 2, 3 as

According to Eq (7), when a strategy extincted, there is no possibility that appears again in the population, and sooner or later, the whole population occupies by one of the strategies. It means the corresponding Markov chain is an absorption Markov chain. According to Eq (6) the number of states in this Markov chain is . States of the Markov chain can be arranged in an equilateral triangle. Fig 1 shows the corresponding Markov chain of the evolutionary game with this specific update rule for N = 10. Arrows show the allowed transitions between states. Fig 1 can also be considered as a simplex that determined states of the evolutionary game. The vertices of the triangle are absorption states that are correspond to fixation strategies in the evolutionary game. When Markov chain is on the triangle’s sides, it is impossible to return inside the triangle because by this specific update rule, when a strategy extinct, it never comes back. When Markov chain is on a triangle’s side, it absorbs in one of two vertices side. We are interested in obtaining fixation probability and conditional fixation time for any state in the simplex. After constructing the transition matrix using Eq (7) and calculating the fundamental matrix, one can obtain the fixation probability of every state of simplex for three absorption states.

Fig 1

The absorbing Markov chain corresponds to an evolutionary RSP game with a population N = 10.

The total number of states is 55, and the arrows determine the allowed transitions between states. Some arrows are two-way, and some of them are one-way arrows. Inside the simplex, all the states are transient, and transitions between them are two ways. Transitions between the inside of simplex and sides are one way. It means when the Markov chain is in the states of sides, it never goes back inside the simplex. In other words, when a strategy extinct, it never appears in population anymore. Transitions between states of sides are two-way too, except transitions between absorption states and their neighbors, which is one way.

After finding the fixation probability of states, by using the theorem of section, we can obtain conditional fixation time for any state of the simplex.

To observe the footprint of the RSP game, we set the elements of the payoff matrix in the neutral case and strong selection both. In the neutral case, the elements of payoff matrix are a1 = a2 = a3 = 1, b1 = b2 = b3 = 2. In the strong selection case, we set the elements of the payoff matrix extremely in favor of the paper strategy and in detriment of rock strategy. In this case, we have a1 = a3 = 1, a2 = 300, b1 = b3 = 0, b2 = 300. Figs 2–4, show the fixation probability of paper, scissors and rock strategies respectively, when the process begins in each state in the simplex. In the neutral selection case, when the distance between the beginning state and absorption state decreases, the probability of absorption increases. After changing the payoff matrix in favor of the strategy paper, the probability of absorption to the strategy paper increased for all states inside the simplex. In this case, states with long distance to fixation state R = 0, S = 0, P = N also have a high probability of absorbing to this fixation state.

Fig 2

The simulation (a) and analytical (b) results for fixation probability of strategy paper in an RSP game with neutral selection. The size of population is 50. The states close to absorption state P = N, S = 0, R = 0 have a higher chance of absorbing in this absorption state. In (c) and (d) same results were shown with a strong selection in favor of paper strategy and detriment of rock strategy. Compared to the neutral case, many states have a higher chance to absorb to paper strategy.

Fig 4

The simulation (a) and analytical (b) results for fixation probability of strategy rock in an RSP game with neutral selection. The size of the population is 50. In (c) and (d) same results were shown with a strong selection in favor of paper strategy and detriment of rock strategy. Since the payoff matrix is in detriment of rock strategy, many states, even those who are close to absorption state P = 0, S = 0, R = N have fewer chances to absorb to P = 0, S = 0, R = N.

The simulation (a) and analytical (b) results for conditional fixation time of strategy paper for an RSP game with a neutral selection. The size of the population is 50. No wonder that states are close to absorption state P = N, S = 0, R = 0 reaches this absorption state by the fewer steps. In (c) and (d) same results were shown for a strong selection in favor of paper strategy and detriment of rock strategy. Compared to the neutral case, the number of steps for reaching P = N, S = 0, R = 0 is reduced due to strong selection in favor of strategy P.

Also there are states that have high probability of absorbing to scissors strategy in neutral case, but in strong selection case, they have high probability of absorbing to paper strategy. That is because the payoff matrix changed in favor of paper strategy. Also, some states have a high probability of absorbing to rock strategy in neutral case, but in the strong selection case, they have a high probability of absorbing to scissors strategy. That is because we changed the payoff matrix to the detriment of the rock strategy. In the strong selection case, there are fewer states with a high probability of absorbing to rock strategy. Changing the payoff matrix has effects on conditional fixation time too. Figs 5–7, show the conditional fixation time of paper, scissors and rock strategy respectively, when the process begins in each state in the simplex.

Fig 5

The simulation (a) and analytical (b) results for fixation probability of strategy paper in an RSP game with neutral selection. The size of population is 50. The states close to absorption state P = N, S = 0, R = 0 have a higher chance of absorbing in this absorption state. In (c) and (d) same results were shown with a strong selection in favor of paper strategy and detriment of rock strategy. Compared to the neutral case, many states have a higher chance to absorb to paper strategy.

Fig 7

The simulation (a) and analytical (b) results for conditional fixation time of strategy rock for an RSP game with a neutral selection. The size of the population is 50. In (c) and (d) same results were shown for a strong selection in favor of paper strategy and detriment of rock strategy. Due to strong selection against strategy rock, conditional fixation time increase for all states of simplex.

Comparing conditional fixation time in the neutral and strong selection cases shows that absorption to the paper strategy happens in a shorter time in the strong selection case. As shown in Fig 6, the states which are close to fixation state R = N, S = 0, P = 0, in the strong selection case, absorb in strategy scissors in a shorter time. Also, the conditional fixation time for absorbing in the rock strategy increases in the strong selection case for all simplex states. The reason again is changing the payoff matrix to the detriment of rock strategy. In all figures, the results from the analytical approach and simulations are compared to each other. In most of them, simulation results coincide with analytical results. Still, in Figs 6 and 7 in the part of strong selection, the similarity is not so obvious. Since in some states, the probability of absorption to rock strategy is very low in the strong selection case, we need a lot of realization of the evolutionary game to reach a limited number realization ended in rock strategy. It means simulation should repeat more times for obtaining an accurate result. The same is true for conditional fixation time of scissors strategy. The hardness of getting simulation results in some conditions shows the necessity of invent of an analytical method.

Fig 6

The simulation (a) and analytical (b) results for fixation probability of strategy scissors in an RSP game with neutral selection. The size of the population is 50. The states close to absorption state P = 0, S = N, R = 0 have a higher chance of absorbing in this absorption state. In (c) and (d) same results were shown with a strong selection in favor of paper strategy and detriment of rock strategy. Compared to the neutral case, some states are close to P = 0, S = 0, R = N but have a high chance to absorb in P = 0, S = N, R = 0 strategy. The reason is imposing strong selection to the detriment of rock strategy.

RSP game without absorbtion states

One may set the update rule in such a way that none of the strategies fix forever. In this situation, the corresponding Markov chain is an ergodic Markov chain. To compare our final result to the numerical result obtained in previous works, we use the update rule of Ref. [39]. According to this update rule, the probability that in each time step, one member of the population switches from strategy l to strategy k is proportional to T = ε + W(π − π) where ε is a positive value which guarantees mutation in the process and W is zero when the argument is negative. W works like the identical function when the argument is positive or zero. Elements of the transition matrix can be calculated as follow where i → j are all allowed transitions in each state of the Markov chain. Fig 8 shows the Markov chain corresponding to this update rule. Unlike the previous update rule, when the Markov chain is in any state, there is a nonzero probability that exits from that state and therefore, there is no absorption state. The limiting probability distribution of the evolutionary game can be obtained by calculating the left null vector of matrix P − I. Fig 9 shows the analytical and simulation results for limiting probability distribution after a long run (100 million steps). The simulation and analytical results agree with each other. As a double-check, one can compare the results with simulation results obtained with the same update rule in Re. [39].

Fig 8

The Markov chain corresponds to an ergodic RSP game for N = 10.

The total number of states is 55 and the arrows determine the allowed transition between states. All arrows are two-way which means when the Markov chain is in a state there is a non-zero probability to escape from it.

Fig 9

Stationary probability distribution of RSP game with population 50.

The update rule is according to Eq 10. The payoff matrix in (a) and (b) is a = 1, b = .5 in (c) and (d) a = 1, b = 1 and in (e) and (f) a = 1, b = 2. To evaluate non-neutral selection in (g) and (h) the payoff matrix set as a = 1, b1 = .5, b2 = 2/3, b3 = 3.

Conclusion

This paper introduced the Markov chain method as an accurate analytical method for analyzing evolutionary game dynamics. Before this, the Makov chain method was used for studying two strategies evolutionary game or Moran process, but using the theorem explained in section, the Markov chain method can be used for any evolutionary game with any number of strategies. This method is flexible with changing the update rule of the evolutionary game. In the case of update rules which determine some fixation strategies, the fixation probability of each strategy and fixation time were calculable by the typical Markov chain method. By the theorem of section one can obtain conditional fixation time for each strategy. As an example, RSP games are evaluated with two update rules. In the first update rule, each of the three strategies can be fixed. Using the fundamental method, fixation probability and conditional fixation time of one of the strategies obtained were consistent with simulation results. In the second update rule, mutation is possible in the evolutionary game, and there is no fixed strategy. Getting the left null vector of matrix P − I leads to the limited probability distribution in agreement with simulation results. This method could also be applied to evolutionary games with more than three strategies.

There is wide possibility of application of Markov chain method not only RPS game. In refrences [50, 51], we used Markov chain method for evaluating the Moran process. In many situations the issue of social dilemma represented by either Prisoner’s Dilemma, Chicken, or Stag Hunt games [71, 72], therefore, applying this method on archetype 2 × 2 symmetric games will lead to significant results.

Codes related to this article is hosted on Github at https://github.com/mehdiphy/rock-scissors-paper-evolutionary-game.

Appendix

In this appendix, we will prove the theorem of section

The theorem is about calculating conditional absorption time in absorbing the Markov chain. It has already been proven [53] that in absorbing the Markov chain the fundamental matrix N = (I − Q)−1 is exists and can be written in an infinite series

Let s and s be two transient states. We assume that the chain starts in state s. Let X( be a random variable which equals 1 if the chain is in state j after k steps and equals 0 otherwise. Let denote the outcome that corresponds to the absorbing of the chain to the absorbing state s. We need to calculate to obtain conditional absorbing time, τ. To this end, we use the following relation for conditional probability

Clearly and .

Now, using we arrive at

The expected number of times the chain is in state s in the first m steps given that it absorb in state s and starts in state si is when m goes to infinity we have Using these conditional probabilities we can calculate the conditional absorbing time, τ, as where .

This way, one can obtain the average conditional absorption time for the processes which are eventually absorbed to each arbitrary absorbing state.

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

17 Jan 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Jun Tanimoto

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We suggest you thoroughly copyedit your manuscript for language usage, spelling, and grammar. If you do not know anyone who can help you do this, you may wish to consider employing a professional scientific editing service.

Whilst you may use any professional scientific editing service of your choice, PLOS has partnered with both American Journal Experts (AJE) and Editage to provide discounted services to PLOS authors. Both organizations have experience helping authors meet PLOS guidelines and can provide language editing, translation, manuscript formatting, and figure formatting to ensure your manuscript meets our submission guidelines. To take advantage of our partnership with AJE, visit the AJE website (http://learn.aje.com/plos/) for a 15% discount off AJE services. To take advantage of our partnership with Editage, visit the Editage website (www.editage.com) and enter referral code PLOSEDIT for a 15% discount off Editage services.  If the PLOS editorial team finds any language issues in text that either AJE or Editage has edited, the service provider will re-edit the text for free.

Upon resubmission, please provide the following:

The name of the colleague or the details of the professional service that edited your manuscript

A copy of your manuscript showing your changes by either highlighting them or using track changes (uploaded as a *supporting information* file)

A clean copy of the edited manuscript (uploaded as the new *manuscript* file)”

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This MS, supported by healthy science and mathematics, delivers a nice and interesting finding. Markov chain approach, although the authors said a ‘theoretical’ approach, provides the dynamics of a time-discrete system. In this sense, the methodology can be said an ‘approximation’ of an inherently time0continuous system. I would like to suggest the authors to note this remark somewhere to be appropriate in the revised MS.

After building up the formulation, the latter part shows numerical results applied to Rock-Paper-Scissors (RPS) game that is one of the architype symmetric games of 2-player & 3-strategy game class. The authors compared the theoretical result based on their approach with MAS result. Both are well consistent.

RPS game is a quite good template which may attract quite a few people dealing with EGT. In this sense, I suggest the authors to more deeply review some up-coming fruits on RPS game for recent years by citing some of them, e.g., The role of pairwise nonlinear evolutionary dynamics in the rock–paper–scissors game with noise, Applied Mathematics and Computation 394, 125767, 2021.

Besides RPS game, a question coming to mind is why the authors not trying to 2 by 2 game including Prisoner’s Dilemma (PD), Chicken, Stag Hunt (SH) and Trivial game classes that have been regarded as the most typical template in the filed of EGT. More importantly, the issue of social dilemma represented by either PD, Chicken, or SH might be more pressing in terms of application than RPS game. Although I wouldn’t go as far as to say that the authors should do with further series of Multi Agent Simulations (MASs), they should mention something somewhere (perhaps, at the end of conclusive remark) that there is wide possibility of application not only RPS game but also archetype 2 by 2 symmetric games by citing recent some review papers or books; e.g., (i) Sociophysics Approach to Epidemics, Springer, 2021, (ii) Evolutionary Games with Sociophysics: Analysis of Traffic Flow and Epidemics, Springer, 2019.

Reviewer #2: The authors studied the multi strategies (rock-paper-scissors) evolutionary game both in analytically and numerically using Markov chain model. Despite the interest and originality of the work, I could not fully evaluate its implementation and soundness. I have doubts about the novelty and the strength of advance required for the reputed journal. Therefore, at this stage, the manuscript cannot be recommended for publication.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

29 Jan 2022

We made the corrections suggested by respecred refrees.

Submitted filename: Response to Reviewers.pdf

Click here for additional data file.

2 Feb 2022

Multi-strategy evolutionary games: a Markov chain approach

PONE-D-21-40483R1

Dear Dr. Hajihashemi,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Jun Tanimoto

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The revised manuscript if this version by the authors seems adequately persuasive me. Thus, at this moment, I can suggest it as acceptable.

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

8 Feb 2022

PONE-D-21-40483R1

Multi-strategy evolutionary games: a Markov chain approach

Dear Dr. Hajihashemi:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Prof. Jun Tanimoto

Academic Editor

PLOS ONE