The canonical equation of adaptive dynamics for life histories: from fitness-returns to selection gradients and Pontryagin's maximum principle.

This paper should be read as addendum to Dieckmann et al. (J Theor Biol 241:370-389, 2006) and Parvinen et al. (J Math Biol 67: 509-533, 2013). Our goal is, using little more than high-school calculus, to (1) exhibit the form of the canonical equation of adaptive dynamics for classical life history problems, where the examples in Dieckmann et al. (J Theor Biol 241:370-389, 2006) and Parvinen et al. (J Math Biol 67: 509-533, 2013) are chosen such that they avoid a number of the problems that one gets in this most relevant of applications, (2) derive the fitness gradient occurring in the CE from simple fitness return arguments, (3) show explicitly that setting said fitness gradient equal to zero results in the classical marginal value principle from evolutionary ecology, (4) show that the latter in turn is equivalent to Pontryagin's maximum principle, a well known equivalence that however in the literature is given either ex cathedra or is proven with more advanced tools, (5) connect the classical optimisation arguments of life history theory a little better to real biology (Mendelian populations with separate sexes subject to an environmental feedback loop), (6) make a minor improvement to the form of the CE for the examples in Dieckmann et al. and Parvinen et al.

Introduction

In their recent paper “Function-valued adaptive dynamics and optimal control theory”, Parvinen et al. (2013) give (i) an abstract recipe for calculating the selection gradient for function valued traits affecting the i(ndividual)-dynamics of physiologically structured populations for use in the canonical equation of adaptive dynamics (in the terminology of Metz and Diekmann (1986); Parvinen et al. refer to these models as process-mediated) and (ii) a recipe for calculating the corresponding evolutionarily steady strategies (ESS-es) by using Pontryagin’s maximum principle (c.q. evolutionarily singular strategies (ess-es) if we confine ourselves to the first order condition derived from this principle). They subsequently apply these recipes to derive concrete expressions for three sample models. However, they do not explicitly consider the relationship between (i) and (ii) but for numerically demonstrating that for their special models the adaptive trajectories approach the ess. In this note we (i) demonstrate how the selection gradient can be calculated from a concrete starting point by using the idea of fitness returns, which gives an interpretation to the components of the resulting formulas, and (ii) show that setting the selection gradient equal to zero leads to a classical marginal value argument which turns out to be equivalent to the local version of Pontryagin’s maximum principle.

Terminology We employ the term fitness return here for a concept that is widely used in evolutionary ecology, often also under this name, but for which we failed to find an explicit definition. If some fitness proxy can be decomposed as the sum of a number of terms that supposedly stand for the contributions of different pathways by which fitness can accrue, we call the effect of a strategy change on the contribution of a pathway the fitness return through that pathway. For a global ESS the sum of all fitness returns is non-positive whatever the strategy change. For local ESS-es we consider only the fitness returns of infinitesimal strategy changes. To accord with common usage these should be called marginal returns. However, as these are the only returns that we consider we shall drop the epithet. When the attention is confined to an infinitesimal neighbourhood of a reference strategy far more fitness proxies allow a conceptually useful additive decomposition thanks to the rules of differential calculus. All that is needed is a biologically interpretable way in which the proxy can be decomposed as a differentiable function of a number of differentiable functions of the strategy. The (marginal) fitness return through one of these functions is then defined as the sensitivity of the proxy to the strategy change in a thought experiment in which we keep the argument of all other functions unchanged. The fitness returns from state dependent decisions are usually determined from first principles conditional on the state under consideration. The epithet conditional is customarily dropped in this case. The (marginal) fitness return from a compound decision involving more than one state is calculated by summing the fitness returns for the separate states weighted with their lifetime occurrence frequencies or duration.

To keep the arguments accessible for evolutionary ecologists, we restrict our calculations from the start to the most commonly encountered class of life history models and use the simplest possible mathematical arguments rather than a more advanced functional analytical approach. In the appendices we will sketch how the same arguments can be obtained more rigorously. Basically we assume our readers to be knowledgeable only about demography and the attendant elementary results from probability theory, but not about systems theory or dynamic optimisation.

On selection gradients, canonical equations, and evolutionarily singular strategies, a summary

Below we consider a life history model in which individuals are characterised by two dynamical variables, a physiological state, assumed to move deterministically, and a probability of still being alive, in addition to an inherited strategy u influencing their dynamics. The strategy u (or if we talk specifically about the resident’s strategy, or if we talk about a mutant strategy) is supposed to be a function of the state of the individual taking values in [0, 1]. To make our life simple we assume that on the population dynamical time scale the community dynamics converges to an equilibrium, which generates the non-fluctuating environment , with the strategy currently in residence. This assumption of a non-fluctuating environment allows us to make use of the fitness proxy , the average lifetime offspring production of a mutant in the environment , calculated e.g. by integrating the average rate of producing offspring over the age of an individual. Consistency requires that . If , a mutant has a positive probability to invade, else it cannot invade. The invasion fitness of a mutant is by definition equal to the asymptotic exponential growth rate of a mutant population in the environment (Metz et al. 1992; Metz 2008). For close to 1 this invasion fitness is well approximated bywhere is the average age at which the residents give birth in the environment (Metz and Diekmann 1986; Durinx et al. 2008).

Remark 1

Dieckmann et al. (2006) and Parvinen et al. (2013) consider seasonal differential-equation-based models where it is possible to calculate the invasion fitness directly by subtracting the time-averaged death rate from the time-averaged birth rate. For such models fitness takes the explicit form of an integral over the year cycle, and there is no need to fall back on an approximation. However, in the usual continuous time life history models only can be expressed explicitly as an integral. The availability of such an integral-based expression formed the basis for the developments in Parvinen et al. (2013), and will also be the cornerstone for our calculations.

The so-called selection gradient G tells how the invasion fitness of a close to depends on the difference . Mathematically, the selection gradient is the derivative of the invasion fitness for evaluated at . From the previous approximation formula for the invasion fitness it follows that we can calculate G as(Durinx et al. 2008; Metz 2008). In this formula is an abstract differential quotient, i.e. a linear map transforming functions of the physiological state into a number that linearly approximates the nonlinear dependence of on .

In view of our stress on life history models, let us moreover assume that u is an allocation, so that u takes values in [0, 1]. The assumptions of a non-fluctuating resident environment and a deterministically moving physiological state moreover allow us to represent the strategy u as a function of the age a of an individual, i.e., . In that case we can write for a function :(c.f., Parvinen et al. 2013). Hence the problem of calculating G reduces to that of calculating the function g. On the assumption that mutations are rare and mutational steps small the dynamics of u can on the evolutionary time scale be described by the so-called canonical equation (CE) of adaptive dynamics (Dieckmann and Law 1996; Champagnat 2003; Dieckmann et al. 2006; Parvinen et al. 2006, 2013; Durinx et al. 2008; Méléard and Tran 2009; Champagnat and Méléard 2011; Gupta et al. 2014)with the average age at which the residents die, the between individual variance of their offspring numbers (i.e., if is a lifetime offspring number of the i-th individual, ), their equilibrium population size, the (small) probability at a birth event of a mutation affecting u, and c the (small) covariance kernel of the mutational steps, i.e., if denotes a mutational step in u, thenThe form of the CE given above is the one for clonally reproducing organisms (the customary assumption in most of life history theory which, however, usually is left implicit). In Appendix 1 we briefly consider its extension to Mendelian diploids.

Our formula for the CE is slightly more complicated than the one in Dieckmann et al. (2006) and Parvinen et al. (2013). The reason is that these authors did not consider local constraints on the strategy, whereas in our case , for each possible age . See Appendix 2 for further information. Another difference is that Dieckmann et al. (2006) and Parvinen et al. (2013) have set the factor equal to 2, in keeping with the idea that for the i-models underlying the standard ordinary differential equation (ODE) models the distribution of the lifetime offspring number is geometric. Moreover, for the standard ODE models and since the g in Dieckmann et al. (2006) and Parvinen et al. (2013) corresponds to our , the in (4) cancels. Appendix 3 treats the corresponding considerations for the periodic ODE case considered by Dieckmann et al. (2006) and Parvinen et al. (2013), with as outcome that in this case their should be interpreted as a harmonic death-rate weighted mean of the population sizes over a cycle.

The equilibria of the CE are the so-called ess-es. If these strategies are moreover (local) fitness maxima for the corresponding then they are also evolutionary equilibria, to which we refer as (local) ESS-es. (An alternative is that at an attracting ess the population starts to accumulate variation, so that it no longer stays quasi-monomorphic as is supposed in the derivation of the CE. (The latter on good grounds: see Geritz et al. 2002; Geritz 2005; Dercole and Rinaldi 2008, Appendix 2).) Another way to calculate ESS-es is to maximise the invasion fitness, or alternatively , over , leading to a function-valued map1, followed by solving the equation . It is here that Pontryagin’s maximum principle is encountered (e.g., Pontryagin et al. 1964; Intrilligator 1971). This principle is derived by considering the differential equations for the i-states as constraints on their time development, and to extend the idea of Lagrange multipliers as encountered in finite dimensional optimisation problems to this case. The Lagrange multipliers then become functions of time, which can be shown to satisfy a set of differential equations, and for this reason are referred to as co-states (or adjoints). In Sect. 6 we give explicit expressions for the life history models described in the next section. Appendix 5 shows how Pontryagin’s maximum principle can be derived directly from a weak variant of Bellman’s principle of optimality, which is rather better known among ecologists.

Model ingredients

Before we get to the specifics we first introduce some notational conventions in order to keep our formulas from becoming too unwieldy.

ConventionsThe two dynamical variables characterising an individual are (i) one i-state variable, to wit its body mass m, increasing from a fixed birth mass , and (ii) its probability P to be still alive, starting from . The energy intake by an individual with body mass m will be denoted by e(m). The strategy of an individual will be denoted by , where u(a) determines which fraction of the energy intake at age a is used for reproduction while the remains are used for growth. The body mass just increases with , while the birth rate is assumed to depend monotonically on the available energy as . Finally, the energy allocation is assumed also to affect the death rate . All three functions e, b, and d also implicitly depend on . In this model the average lifetime offspring number of a mutant strategy equalsNote that if , due to the value of necessarily . Moreover, we assume that the tail of P is bounded by a negative exponential and that b and e are bounded. These assumptions derive from the biology behind the example and imply that the improper integral in (6) exists.

The argument will be usually hidden.

Similarly we shall often hide the argument u in expressions like P(a; u) for the probability that an individual survives till age a, or m(a; u) for its body size at that age.

When we use the argument u, then u stands either for , , or , with the context making clear which is the case.

For a function of a single scalar variable we use a prime to indicate its derivative. A superscript dot indicates a derivative for age, also when a function has other arguments as well.

Calculating the selection gradient from a fitness-returns argument

We shall express g from Eq. (3) in terms of the fitness returns r, that is, the proportional effects of small local changes in u on the total future reproduction. To calculate r(a; u) we proceed by means of a thought experiment. For a living individual aged a we increase u between a and by an amount , i.e., we construct a function , , for and 0 elsewhere (see Fig. 1),

Fig. 1

Illustration of the fitness-returns argument where u is any control path. The idea is to do a thought experiment in which we increase u between a and with a block B of height , and observe its effect on fitness

Calculate the resulting expected change in the expected life-time offspring number, multiply this number with and let both and go to zero. Since the fitness return by definition is calculated conditional on an individual surviving to a, only the fraction P(a; u) surviving till age a contributes in this manner to . HenceTo calculate those expected additional offspring numbers we proceed in the spirit of the marginal value theorem, that is, we calculate and then add the components of r contributed by different routes. These components include the immediate additional number of offspring coming from the temporary increase in energy allocation to reproduction and the decreases in future offspring numbers caused by the future smaller size and lesser survival caused by the temporary decrease in allocation to growth and to staving off death. We start with the calculation of the second and third components. Let and denote the differences and , respectively. Then, by a first-order Taylor expansion of and with respect to , we obtain for which impliesThe immediate offspring gain from this strategy change over the time interval (for an individual that survived till a) isFrom onwards Eq. (8) apply with set to zero and with initial condition (9).

The future loss of offspring from this change in strategy for an individual that already has survived till a is The linearity of Eq. (8) with implies that and are linearly dependent on the initial conditions given by (9), and therefore the outcome (11) is proportional to . To make the coming calculation more transparent we introduce new functions , , , defined bywhere can be interpreted as the conditional survival of an individual that has already survived to age a, i.e., , given the strategy u, and as the relative amount by which a small perturbation in m present at age will propagate into the future given u. Similarly allows an alternative interpretation as relative amount by which a small perturbation in m present at age will propagate into the future given u. For and we can then express the fitness return r(a; u) as follows:At an ess the return r(a) should be 0 when , non-positive when and non-negative when .

The other ingredients of the canonical equation

To complete the canonical equation we need to find expressions for , , and . Since , the expression is a probability density. Furthermore,thus also is a probability density. Therefore, and can be expressed directly:where the last equation comes from integrating by parts and using .

To calculate we have to be more specific about the microstructure of the reproduction process. The assumption that naturally leads to (5) is that for an individual that is still alive the births come in a Poisson process with rate b(ue(m)), or, slightly more generally, in clutches of average size C(u, e(m)) produced according to a Poisson process with rate . We confine ourselves here to the former option. In such a case, for a given age at death a the total offspring number is Poisson distributed with meanIn general, a is a realisation of a random variable . Hence, the lifetime offspring number is a mixture of Poisson random variables. The mean of is nothing but the average lifetime offspring number2Therefore, , with(integration by parts).

Finally, from the general rules for mixtures of distributions3

Locating fitness maxima by means of Pontryagin’s maximum principle

The equilibria of the canonical equation are called ess-es. The reason for this from a differential equations viewpoint unusual terminology is that among the ess-es only the ESSes, characterised by the fact that they are also maxima of the current fitness landscapes, are immune to evolutionary change. One way of calculating ESSes for life history problems is to make use of Pontryagin’s maximum principle to locate the fitness maxima in that go with a given and then to set .

In contrast to the canonical equation, Pontryagin’s maximum principle, at least in its original form and with a number of standard assumptions satisfied, is textbook material. In this section we will just in the wake of Intrilligator (1971) state the conditions that an optimal u has to satisfy. For a discussion of different variants of Pontryagin’s maximum principle and its connection to the Bellman’s principle of optimality (Bellman 1957), see Appendix 5. In the notation of Intrilligator (1971), Eq. (6) can be rewritten in the following form:with the state vector, J the quantity to be optimised, calculated as the lifetime integral of I, and the right hand side of the differential equation for . Pontryagin’s maximum principle then says that if maximises J, then at each age it also maximises the so-called Hamiltonian, defined aswith being the so-called co-state (or adjoint) vector, where its components satisfy the differential equationswith final conditions . If maximises in (6), then it also maximisesIn other words, if J is maximised by , thenat each . This implies thatObviously, to assure that is a local maximum of J, resp. H, the derivative of with respect to u(a) has to be negative whenever .

The co-states and in (26) can be expressed from (24) as follows:where and have to be chosen such that . In Appendix 4 we show that:The detailed match between the results from the two approaches more generally follows from the correspondence between Bellman’s principle of optimality and Pontryagin’s maximum principle that we work out in some detail in Appendix 5.

which can be interpreted as the marginal loss or gain per unit weight change (sensitivity) of lifetime offspring due to lower subsequent weights, and

which can be interpreted as the sensitivity of lifetime offspring due to lower subsequent survival. Moreover, in the same appendix we show that Formulas (12)–(13) for calculating the fitness returns (c.q. the selection gradient) and Formulas (26)–(27) for the derivative of the Hamiltonian with respect to u, are equivalent.

On the practical side we point at the fact that even when one is only interested in calculating an ESS with the help of Pontryagin’s maximum principle, and has no particular interest in the evolutionary trajectories by which this ESS may be reached, running some discretised variant of the canonical equation can still provide an effective computational implementation of that principle as used in ESS calculations.

Discussion

The main contribution of this note is that we carefully set up the CE for life history decisions. As it turned out, a few details had to be added to the exposition in Parvinen et al. (2013). In particular, it was necessary to extend the canonical equation so as to be able to handle inequality constraints. In addition, there was the small detail of the appearance of an additional multiplicative factor accounting for the difference in the initial branching process that mutants have to get through before getting established compared to the linear birth and death process that appears in this role for ODE population models (c.f. Durinx et al. 2008).

Given the venerable history of Pontryagin’s maximum principle and its applications to life history theory it should raise no wonder that interpreting the co-states is not new. In particular, Jesus Alberto Leon already did so in the nineteen-seventies (Leon 1976); see also Perrin and Sibly (1993). However, in those days there was no canonical equation around and hence no need to make a connection. Moreover, these early authors put forward the interpretation seemingly ex cathedra, and only post hoc and summarily related it to a marginal value argument, without exhibiting the explicit connection made in our Sects. 4 and 6 and Appendices 4 and 5. In particular, they did not consider “co-state variables” for other u than the optimal one. Precisely these “generalised co-state variables” occur as ingredients of the selection gradient. Although such variables are already used in numerical approaches to Pontryagin’s maximum principle (e.g. Näslund et al. 1974), we believe that our explicit calculations add to the biological understanding of the mathematical structure of eco-evolutionary models.

As a final point we note that the argument that we provide in Sect. 5, although this was not spelled out there, is exemplary of a more general principle. When we delve a little more deeply into the stochastic models for individual behavior, as was necessary in order to calculate , it generally becomes clear how embarrassingly oversimplified such models tend to be. In our case it turned out that it was implicitly assumed that microscopically the production of young is coupled far more loosely to the energy flow to reproduction than seemingly is assumed at the macroscopic level. Real organisms first have to accumulate the necessary energy that then is transformed into the birth of a young, instead of randomly producing young on the basis of the instantaneous availability of resources. Therefore in reality the production of young usually is far more regular than Poisson (so that is close to ), and at a given time depends also on past energy availabilities. Hence the idea that the average rate of offspring production at age a is just a function b of is at best only a rough approximation. One possible justification is that most of the time varies only slowly compared to the rate at which young are produced, and that if reproduction does occur spread out in time, no two individuals will be in the same phase of their reproduction cycle, so that at any one time the effective offspring production of the individuals that have a size close to the scalar may well be on average close to . However, the modelling community is still a long way from proving any rigorous approximation theorems of this ilk. (See Heijmans and Metz (1989) for another possible justification, which, however, is less often applicable in a general life history context.) Of course we also made other simplifying assumptions, like neglecting basal metabolism. However, these simplifications were only put in to ease the exposition, raise no deep mathematical issue, and hence can presumably be relaxed without great difficulty.