Cannings models, population size changes and multiple-merger coalescents.

Multiple-merger coalescents, e.g. [Formula: see text]-n-coalescents, have been proposed as models of the genealogy of n sampled individuals for a range of populations whose genealogical structures are not captured well by Kingman's n-coalescent. [Formula: see text]-n-coalescents can be seen as the limit process of the discrete genealogies of Cannings models with fixed population size, when time is rescaled and population size [Formula: see text]. As established for Kingman's n-coalescent, moderate population size fluctuations in the discrete population model should be reflected by a time-change of the limit coalescent. For [Formula: see text]-n-coalescents, this has been explicitly shown for only a limited subclass of [Formula: see text]-n-coalescents and exponentially growing populations. This article gives a more general construction of time-changed [Formula: see text]-n-coalescents as limits of specific Cannings models with rather arbitrary time changes.

Introduction

The genealogies of samples from populations with highly variant offspring numbers, for instance due to sweepstake reproduction or rapid selection, are not well modelled by Kingman’s n-coalescent. As a more realistic alternative, multiple-merger coalescents, especially -coalescents have been proposed, as reviewed in Tellier and Lemaire (2014), Irwin et al. (2016) and Eldon et al. (2016). -n-coalescents, introduced by Pitman (1999), Sagitov (1999), Donnelly and Kurtz (1999), are Markovian processes , which describe the genealogy of a set of individuals . This is done by representing the ancestral lineages present at time t of these individuals by the sets of offspring of each ancestral lineage in the sample. Thus, can be defined as a random process with states in the set of partitions of and transitions via merging of blocks (i.e. merging of ancestral lineages to a common ancestor). For a -n-coalescent, the infinitesimal rates of any merger of k of b present lineages is given by , where is a finite measure on [0, 1]. This includes Kingman’s n-coalescent if is the Dirac measure in 0.

As in the case of Kingman’s n-coalescent being the limit genealogy from samples taken from a discrete Wright–Fisher or Moran model, -n-coalescents can be constructed as the (weak) limit of genealogies from samples of size n taken from Cannings models. The limit is reached as population size N goes to infinity and time is rescaled, see Möhle and Sagitov (2001). Time is rescaled by using generations in the discrete model as one unit of evolutionary (coalescent) time in the limit, where is the probability that two individuals picked in a generation have the same parent one generation before. In the discrete models, the population size N is fixed across all generations.

Only populations in an equilibrium state are described well by models with fixed population sizes. This idealized condition often does not apply to natural populations. In particular, due to fluctuating environmental conditions population sizes are expected to fluctuate likewise. Two standard models of population size changes are timespans of exponential growth or decline, as well as population bottlenecks, where population size drops to a fixed size smaller than N for a timespan on the evolutionary (coalescent) timescale. Such changes are featured in coalescent simulators as ms (Hudson 2002) or msprime (Kelleher et al. 2016). The latter changes are also the model of population size changes in PSMC (Li and Durbin 2011) or similar approaches as SMC++ (Terhorst et al. 2017). For the Wright–Fisher model, which converges to Kingman’s n-coalescent if population size N is fixed for all generations, the same scaling from discrete genealogy to limit is valid for population size changes which maintain a population size of order N at all times, see Griffiths and Tavare (1994) or Kaj and Krone (2003). The resulting limit process is Kingman’s n-coalescent, whose timescale is (non-linearly) transformed. However, size changes too extreme can yield a non-bifurcating (multiple merger) genealogy, see Birkner et al. (2009, Sect. 6.1).

For -n-coalescents, the link between fluctuating population sizes in the discrete models and the time-change in the coalescent limit is somewhat less established. While conditions for convergence of the discrete genealogies to a limit process are given in Möhle (2002), no explicit construction of haploid Cannings models leading to an analogous limit, a -n-coalescent with changed time scale, is given. For a specific case, the Dirac n-coalescent for an exponentially growing population, such a construction has been given in Matuszewski et al. (2017), based on the fixed-N Cannings model (modified Moran model) from Eldon and Wakeley (2006). However, also other -n-coalescents (or Cannings models which should converge to these) with changed time scale have been recently discussed and applied as models of genealogies, see Spence et al. (2016), Kato et al. (2017), Alter and Louzoun (2016) and Hoscheit and Pybus (2019). This leads to the goal of this article, which is to extend the approach in Matuszewski et al. (2017) to explicitly give a construction of time-changed -coalescents as limits of Cannings models with fluctuating population sizes. The Cannings models used are modified Moran models, see e.g. Huillet and Möhle (2013), and the Cannings models introduced in Schweinsberg (2003). The main tool to establish the convergence to the time-changed -n-coalescent is, as in Matuszewski et al. (2017), applying Möhle (2002, Thm. 2.2).

For diploid Cannings models, the umbrella model from Koskela and Wilke Berenguer (2019) gives a general framework to add population size changes, selection, recombination and population structure to the fixed-N-model. There, if one only considers population size changes, the limit is a time-changed -n-coalescent, a coalescent process with simultaneous multiple mergers. The focus in the present paper is slightly different though, the aim is to explicitly construct Cannings models that converge, after linear time scaling, to a time-changed -n-coalescent, while Koskela and Wilke Berenguer (2019) concentrates on the convergence itself.

Models and main results

Cannings models (Cannings 1974, 1975) describe the probabilistic structure of the pedigree (offspring-parent relations) of a finite population in generations with integer-valued population sizes . The individuals in generation v produce offspring, where and offspring sizes are exchangeable, i.e. for any permutation . The offspring generation then consists of these individuals in arbitrary order (independent of the parents). The case for all v is denoted as the fixed-N case.

From now on, look at the genealogy of the population in generation 0. For convenience, denote the generations in reverse order by , i.e. if one looks i generations back, this is denoted by . The population sizes are defined relative to a reference size N, in a way that if , also . From now on, use . The goal is to establish a limit process of the discrete genealogies as . The discrete genealogy of a sample of size n in generation 0 is a random process with values in the partitions of , where i, j are in the same block of iff they share the same ancestor in generation r.

The terminology from Möhle (2002) is used with slight adaptations. Let be the probability that two arbitrary individuals in generation have the same ancestor in generation r in the model with reference population size N. To clarify, is the coalescence probability for individuals in generation if population sizes are variable, while denotes the coalescence probability in the fixed-N case. Define and letbe its shifted pseudo-inverse. For l and , setas the probability that in generation , from individuals sampled from the Cannings model, specific sets of individuals each find a common ancestor one generation before (generation r), where ancestors of different sets are different. For , . See Möhle (1998) for details.

Consider a sequence of fixed-N Cannings models for each with as and transition probabilities for a merger of individuals , converging to a -n-coalescent with infinitesimal transition rates when scaled by , i.e.in the Skorohod sense as . Eq. (3) is satisfied iffor as , see (Möhle and Sagitov 2001, Thm. 2.1). We will establish a variant of Möhle (2002, Corollary 2.4) to show convergence of a variety of Cannings models with variable population sizes to (time-changed) -n-coalescents. For this, we need some assumptions. Most importantly, an asymptotically infinite sum needs to be controlled. For this, we introduce a concept of o-terms: A sequence of null sequences is for a null sequence if summing the first members of still vanishes as . This will be denoted by , i.e.For instance, this condition is satisfied if have the same null sequence majorant as for all .

Fix . We assume for all :The first class of Cannings models used to construct time-changed -n-coalescents are modified Moran models. In a modified Moran model, only a single individual has more than one offspring (and may have many offspring). Following Huillet and Möhle (2013), define the modified (haploid) Moran model with fixed population size N. Let be i.i.d. random variables with values in , let be a r.v. with their common distribution. In each generation ,Specific modified Moran models leading to Dirac n-coalescents as genealogy limits have been introduced as population models with skewed offspring distributions, see Eldon and Wakeley (2006) and Matuszewski et al. (2017), for fixed and variable population sizes.

Population size changes of order N leading to a well-defined population size profile in coalescent time, i.e. for positive and finite functions .

One randomly chosen individual has offspring,

randomly chosen individuals have no offspring,

The other have one offspring each,

For any -n-coalescent with (denoted by ), Möhle (2013, Prop. 3.4) show that there always exist fixed-N modified Moran models such that their rescaled genealogies converge to the -n-coalescent. These can be constructed via a random variable , that is distributed like the merger size of the first merger in a -N-coalescent. As shown in (Huillet and Möhle 2013, Eq. 9), this meanswhere is the total transition rate of the -N-coalescent and X has distribution .

To add population size changes from generation to generation to the modified Moran model, the relationship between offspring and parent generation needs to be defined. This will be done by adjusting the fixed-N model: If in generation r, there are individuals, first run a fixed- modified Moran model, producing (potential) offspring. Let denote the number of offspring in generation of the multiplying parent in generation r.

If population size declines from generation r to , sample individuals randomly (without replacement) from the potential offspring consisting of offspring of the multiplying parent and single offspring. If individuals are added to the population (population growth), i.e.to still end up with a modified Moran model one has two options. Additional individuals can be added as further offspring of the already multiplying parent. A second option is to add individuals as offspring of the non-reproducing individuals from generation r in the fixed- model. Each originally non-reproducing parent can have one offspring, so this allows one to add individuals. The number of additional individuals can be divided between these two options, let denote the individuals added as offspring to the multiplying parent (which means that individuals are added as offspring of non-reproducing parents from the fixed- model). Expressed differently, there are offspring from which share the same parent, while all other offspring are single offspring of other parents (who all differ). See Fig. 1 for an example.

Fig. 1

Example of allocation of individuals when population size is increasing. Left: start with a fixed size Moran model with . Right: population increases by , from which individual is allocated to the multiplying parent from generation r in the fixed size model (and 2 to non-reproducing individuals from the fixed size model in generation r). This results in

While some care has to be taken to not change coalescence probabilities (see Remark 5 for an example), there will be different possibilities to choose . For Dirac-n-coalescents with exponential growth (on the coalescent time scale), Matuszewski et al. (2017) used . A reasonable approach may also be to set (close to) proportional to the fraction of offspring coming from the multiplying parent: Each of the added individuals are added to the multiplying parent with probability (with the obvious constraint that after individuals are added as offspring of non-reproducing parents, all further individuals need to be added to the multiplying parent). As for the fixed-size models, we consider the genealogy of a sample of n individuals, which is denoted by

The main results of the present paper show that the two allocation schemes allow one to construct -n-coalescent limits of the genealogies of these modified Moran models if population sizes vary in the discrete models in ways described by Eq. (4).

Theorem 1

Let so that defined by Eq. (6) satisfiesDefine a modified Moran model for fixed N byfor sets s.t. , are independent and for . Let be a positive real function. Then, there exist population sizes satisfying Eq. (4) for so that the genealogies of the modified Moran model with variable population sizes convergein the Skorohod-sense, where and is a -n-coalescent. In the discrete model, additional individuals can be added in any way so that the resulting model is still a modified Moran model.

For not covered by Theorem 1, one can choose slightly different modified Moran models that converge to a -n-coalescent limit for an arbitrary population size profile on the coalescent time scale.

Theorem 2

Fix so that defined by Eq. (6) satisfiesFor fixed population size N, define modified Moran models via . Let be a positive function describing the population size profile. Then, there exist population sizes satisfying Eq. (4) for so that the genealogies of the modified Moran model with variable population sizes fulfill in the Skorohod-sense, where is a -n-coalescent. In the discrete model, additional individuals are added solely as offspring of non-reproducing parents from the fixed- model, unless as . In that case, they can be added any way that preserves that the model is still a modified Moran model.

Remark 1

The condition of in both theorems is not very important: If one scales by instead of for any , the rescaled discrete genealogies converge to the -n-coalescent.

To get convergence to a time-changed -n-coalescent in Theorem 2, i.e.in the Skorohod-sense as , one needs thatexists for all . The following corollary shows that, at least for certain measures , this condition is met.

Corollary 1

Let be a Beta(a,b)-distribution with and . Let . Then, there exist population sizes satisfying Eq. (4) for so that the genealogies of the modified Moran model with variable population sizes fulfill in the Skorohod-sense, where and is a Beta(a, b)-n-coalescent. In the discrete model, additional individuals can be added in any way so that the resulting model is still a modified Moran model.

The specific models used in each of the two theorems are not the only possibilities of modified Moran models with variable population sizes to converge to -n-coalescents. For instance, if one only allows certain population size changes, one can also use the modified Moran model used in Theorem 2 for some covered by Theorem 1.

Corollary 2

Let be a Beta(a,b)-distribution with and . Consider an exponentially growing modified Moran model population on the coalescent time scale, i.e. Then, there exist population sizes satisfying Eq. (4) for so that the genealogies of the modified Moran model with variable population sizes fulfills in the Skorohod-sense, where and is a -n-coalescent. In the discrete model, additional individuals can be added in any way so that the resulting model is still a modified Moran model.

Finally, for the classic Moran model, i.e. the modified Moran model with and defined via Eq. (17) for , we can establish

Proposition 1

For the standard Moran model and a population size profile , there exist population size changes allowed by Eq. (4) so that in the Skorohod-sense, where . Individuals are added only as offspring of non-reproducing parents (in the fixed- model) if the population size increases.

For Beta-n-coalescents, , genealogies sampled from the fixed-N Cannings models introduced in Schweinsberg (2003) also converge weakly to these Beta coalescent processes (after rescaling of time) as .

This model (for fixed population size N) lets each individual in any generation r produce a number of (potential) offspring , i.i.d. across individuals and generations, distributed as a tail-heavy random variable X with , i.e.where is a constant and . Then, N offspring are chosen to form the next generation. If less than N offspring are produced, the missing next generation individuals are arbitrarily associated with parents. Here, this is done by randomly choosing a parent, which preserves exchangeability and makes the model a Cannings model. The genealogies of a sample of size n converge as and time rescaled by to the Beta(,)-n-coalescent, see Schweinsberg (2003, Thm. 4).

This model can very easily extended to variable population sizes by just sampling from the potential offspring. The tail-heavy distributions used produce, asymptotically as , enough potential offspring to cover growing population sizes of order N as allowed by Eq. (4).

Lemma 1

Let . Assume that for any fixed t, for all there exists a null sequence with as . Then, with and .

This gives us an alternative Cannings model with variable population sizes to define time-changed Beta coalescents as the limit of its discrete genealogies.

Theorem 3

Consider the Cannings model coming from sampling from potential i.i.d. offspring following Eq. (9) with parameter . For any , there exist variable population sizes fulfilling (4) for so that the discrete n-coalescents converge in the Skorohod-sense, where and where is the Beta()-n-coalescent.

The time-change function , which appears in Theorem 1, Corollary 2, Propositions 1 and 3 simplifies considerably for exponential growth on the coalescent time scale, i.e. for in Eq. (4) (corresponding to population sizes given by for ).

Corollary 3

For a population size profiles of exponential growth (on the coalescent scale) with growth rate and for for , the time-change function has the formThis implies that the waiting time between coalescent events are Gompertz distributed with parameters and , i.e. the waiting time T for the next coalescence event, given the last coalescence at into b lineages, fulfills

Remark 2

It is well-known that for Kingman’s n-coalescent with exponential growth, waiting times for coalescence events follow a Gompertz distribution, e.g. see Slatkin and Hudson (1991, Eq. 5), Polanski et al. (2003). For time-changed Dirac coalescents appearing as limits of modified Moran models with , Eq. (10) appeared in Matuszewski et al. (2017).

Discussion

As for the Wright–Fisher model, genealogies of samples taken from (haploid) modified Moran and other Cannings models can be approximated by a time-change of their limit coalescent process, when the population sizes of the discrete models are fluctuating, but are always of the same order of size. As for models with fixed population size, time intervals of generations in the discrete model correspond to a time interval of length t in the continuous time limit. The approach of this study was to build on existing Cannings models that converge for fixed population size to the -n-coalescent and just change the population sizes gradually from generation to generation, which includes adjusting parent-offspring allocation between generations. This raises the question whether the used Cannings models and the adjustment of ancestral relationships have biological interpretations and are a reasonable model for at least some real populations.

Interpretation of the Cannings models and allocation schemes used

The modified Moran models used to construct a time-changed -n-coalescent with [defined via Eq. (6), introduced in Huillet and Möhle (2013)] can be described as follows (for fixed N): On top of a standard Moran model choice of one parent M with two offspring and one individual in the parent generation with no offpring, there is a random probability X for each other individual in the parent generation to not have offspring in the next generation. X is drawn from , potentially only activated in a given generation with a low probability , . From the individuals that have offspring, all but M reproduce once, and M replaces itself and all non-reproducing individuals by its offspring. These models capture the concept of sweepstake reproduction (Hedgecock and Pudovkin 2011), though the assumption of a single individual with more than one offspring is rather artificial. For a non-random X and large families appearing occasionally at rate of order , this model is very similar to the discrete modified Moran model from Eldon and Wakeley (2006) used to describe sweepstake reproduction (and that was used in Matuszewski et al. (2017) as a basis to construct a time-changed Dirac n-coalescent). Both models lead to the same Dirac coalescent limit and have the same time rescaling order . In Eldon and Wakeley (2006), instead of randomly choosing individuals to not reproduce with probability X, a fixed number of individuals are chosen at random to not reproduce on top of the Moran choice (again with a small probability in each generation for this to happen). For random X, similar models also appear in Hartmann and Huillet (2018) and Eldon (2012).

The other class of Cannings models used to capture skewed offspring distributions, defined via Eq. (9), lead to the specific class of Beta(, )-n-coalescents. They have been proposed as a model of type-III survivorship, where all individuals produce many offspring with a high juvenile mortality, see e.g. Steinrücken et al. (2013, Sect. 2.3), also leading to sweepstake-like phenomena. While both classes of Cannings models allow the Bolthausen-Sznitman n-coalescent () as a possible limit model, the discrete models used to explicitly construct it are not based on modelling a directed selection process due to selective advantages of certain ancestral lineages. Thus, the results do not answer whether adding population size changes to a model of rapid selection or genetic draft as in Desai et al. (2013), Neher and Hallatschek (2013), Schweinsberg (2017) also leads to its rescaled genealogies being described by a time-changed Bolthausen-Sznitman n-coalescent.

To construct time-changed -n-coalescents as limits of genealogies in modified Moran models, the approach here is to adjust fixed-N modified Moran models for growing or decreasing population sizes. Sampling the next generation from the fixed-N offspring when there is population decline maintains on average the ratio between the large family and the rest off the individuals. This means that the population decrease, e.g. due to less resources available, has the same chance to affect each offspring of the fixed-size model. Additional individuals can be added to the family of the multiplying parent or by allowing parents with no offspring from the fixed-N allocation scheme to have exactly one offspring. For some sequences of modified Moran models, any partition of additional individuals to these two allocation forms is possible, e.g. allocate them randomly to the multiplying parent (with offspring) from the fixed-size model with probability (with the constraint that we cannot add more than individuals to non-reproducing parents). The merit of this random allocation is that it is trying to maintain the ratio from the fixed-size model. As for sampling a smaller number of individuals, this describes that population size increase, e.g. due to more resources available, follows (approximately and on average) the sweepstake pattern of the fixed-N model. From a biological viewpoint, other allocation schemes can also be interpreted: Adding the additional offspring completely to the largest family, as done in Matuszewski et al. (2017), could describe a scenario where new resources become available and only the multiple-offspring parent can claim them for its offspring. In contrast, adding individuals as single offspring of non-reproducing parents from the fixed-size model relaxes the (viability) “selection” pressure of the modified Moran model by allowing more non-multiplying parents (resp. their offspring) to survive, e.g. due to the additional resources. For the models covered in Theorem 3 from Schweinsberg (2003), population size changes in either direction are modelled by sampling from a pool of more individuals than the current population size, thus additional or decreasing resources affect the offspring of different parents in the same way.

Influence of the choice of Cannings model on the limit

Many results in the present paper allow us to scale the time in the discrete models with as in the fixed N case so that the scaled genealogies converge to a time-changed -n-coalescent . This time-change depends both on the population size profile on the coalescent time scale from Eq. (4) and the (asymptotic properties of) the coalescence probabilities , i.e. how many discrete generation correspond to one unit of coalescent time. For instance, consider an exponentially growing population (on the coalescent time scale, for ) and two different models leading to a time-changed -n-coalescent (): the ones from Corollary 1 and Theorem 3. From Eq. (10), we see that depends on the product . For the model from Corollary 2, and for the one from Proposition 3, it is . Thus, the exact same time-changed -n-coalescent can appear as limit model for genealogies with different population size profiles on the coalescent time scale. As already discussed in (Matuszewski et al. 2017) in the case of time-changed Dirac-n-coalescents, this poses a problem for inference: If one wants to infer directly (instead of the compound parameter ), has to be known. This means that specifying/identifying the Cannings model leading to the limit process would be necessary to directly estimate . This is very similar to the effect that e.g. Watterson’s estimator only estimates the mutation rate on the coalescent time scale, and not the mutation rate in one generation, see e.g. Eldon and Wakeley (2006, p. 2627). Another example for different leading to the same time-scaled coalescent limit for different Cannings models is given by the genealogy limit from the Wright–Fisher model and the (usual) Moran models. It is well known, see e.g. Griffiths and Tavare (1994), that the rescaled genealogy of a sample from a Wright–Fisher model with population size profile converges to Kingman’s n-coalescent with time change as in Eq. (22) with . However, for the classic Moran model, Proposition 1 shows that Eq. (22) holds with .

For families of Cannings models, if the coalescence probability is of order , a curious phenomenon appears: Population size changes of order N do not even alter the limit genealogy. An example is the model from Proposition 3 for the Bolthausen-Sznitman n-coalescent (). One can interpret this for a population described by the model as follows: Even instantaneous bottlenecks or expansions do not influence the effect that a very large family appearing in a generation has on the genealogy. How the population reproduces, i.e. how the offspring distributions compare between different parents, is thus fully controlling the genealogy, regardless of changes that alter the population sizes overall, e.g. changes in range and/or resources.

Proofs

This section contains the proof of the presented statements as well as some further remarks.

Converging to a time-changed coalescent: sufficient conditions

First, recall this special case of Möhle (2002, Thm. 2.2)

Corollary 4

If we satisfy, for any fixed t,the discrete-time coalescent , so rescaled in time, converges in distribution (Skorohod-sense) to a continuous-time Markov chain with transition function , where Q is a transition rate matrix with entries , (so diagonal entries are the negative row sums of the other entries).

Remark 3

When compared to the original formulation of Möhle (2002, Thm 2.2), the limit here can be described as a homogeneous Markov chain with rate matrix Q instead of the more complicated original description of the transition probabilities as a product integral of matrix-valued measures. This directly follows from the stronger condition (12), where for Möhle (2002, Thm 2.2) to hold only convergence and not linear dependence on t is needed. Indeed, if (12) holds, the value of the product measure in Möhle (2002, Thm. 2.2) has the form Qt. This is stated on Möhle (2002, p. 209), see also Eq. (24) therein. Then, the form of the transition function is described on Möhle (2002, p. 203).

Now, recall the conditions (4), (5). Additionally, consider the following control condition for the fluctuations of as :

For , there exist withfor all . For instance, when (4) holds, this condition is satisfied if , where f is regularly varying (at ). If Eqs. (5) and (13) hold, choosing in Eq. (5) yieldsas .

Now, we establish an easy-to-verify variant of Möhle (2002, Corollary 2.4).

Lemma 2

Consider a sequence of Cannings models with reference size and variable population size which fulfill conditions (4), (5), (13), and whose genealogies of a sample of size n, if one would fix the population sizes , would be in the domain of attraction of a -n-coalescent (rescaled by ). Then, Corollary 4 can be applied, so in the Skorohod-sense.

If furthermore exists, we have, with ,as

Proof

Size changes of order N satisfy the first part of Condition (11). Its second part is then satisfied by (14), which in turn is satisfied due to (5) and (13). Also due to (14), is bounded byas and and thus its pseudo-inverse bywith an appropriate . This implies that the time change function for the discrete models in Corollary 4 is of order . Knowing this, we computeas The second equation is valid due to the uniform convergence of

in r for ( is bounded from below on the timescale used). This allows us to pull out . This shows that condition 12 is satisfied and thus establishes the convergence of to the same -n-coalescent as the fixed-size model. Eq. (15) follows as described in Möhle (1998, Sec. 4).

Remark 4

The condition for Eq. (15) to hold is a weak condition, since is of order . Additionally, the linear scaling in (15) makes it easy to introduce a mutation structure. Let mutation be introduced in the discrete model by allowing mutations from parent to offspring with a rate . If as , the mutations on the time-scaled -n-coalescent are given by a Poisson point process with homogeneous intensity .

The next step is to establish a special case of Lemma 2 which only considers modified Moran models with changing population sizes.

Remark 5

Depending on the magnitude of a population size increase, adding individuals as further offspring of the multiplying parent from the fixed-size modified Moran model can strongly increase coalescence probabilities. For instance, for a population expansion of size Nm, if one just expands by adding to the offspring number of the individual with multiple offspring in a single generation, the coalescence probability for this generation is dominated by the population size change. Then , leading to as . Thus, from generation to r, coalescence is still happening with positive probability as , which shows that a potential limit coalescent cannot just be a (non-degenerately) time-changed -n-coalescent, a continuous-time (inhomogeneous) Markovian process. This has an implication for modelling of real populations: The genealogy of a sudden population expansion, happening at a specific generation, where a single genotype/individual is responsible for the population growth, is not given by a time-changed continuous-time -n-coalescent.

We recall some properties of fixed-N modified Moran models.

Lemma 3

As : is equivalent to

If as , the genealogies in the modified Moran models converge, with a rescaling of time by , to a -n coalescent if

If is distributed as in Eq. (6) for all .

(i) from Huillet and Möhle (2013, Lemma 3.2), (ii) from Huillet and Möhle (2013, Theorem 3.3), (iii) from Huillet and Möhle (2013, Eq. 10)

The following proposition provides criteria for genealogies in modified Moran models with fluctuating population sizes to converge to a -n-coalescent after a suitable time change.

Proposition 2

Consider a fixed-N modified Moran model so that as and that (17) holds for a finite measure on [0, 1]. From this, construct a modified Moran model with varying population sizes which satisfy the following conditions. Assume that Eqs. (4) and (13) are satisfied. Assume further as . Let be the number of individuals in generation allocated as offspring of the multiplying parent of the fixed- model from generation r. If , further assume and for a constant and . Additionally, assume .

Based on the fixed-size modified Moran model and define a modified Moran model with population sizes and offspring variable for all .

Then, in the Skorohod-sense, where is the -n-coalescent limit for the fixed-N modified Moran model.

This is shown by applying Lemma 2. All conditions but Eq. (5) of it are clearly fulfilled under the assumptions of the proposition currently proven, see also Lemma 3.

To show (5), first assume . Then,where s(n, k) are Stirling numbers of the first kind. From Eq. (17), we see that uniformly in r as (convergence of the first summand is at least as fast as for ), while we will now show that the sum following this term in Eq. (19) vanishes asymptotically. We will give an upper bound for , independent from r. For this, we need to recall several technical points: For an upper bound, we can always omit terms of the form for , we assume for and as in this proposition and we havewhich follows from the fact that decreases for . With all this, we can observe that, for and ,as . We can thus establish convergence, uniform in r, in Eq. (19), since we have just shown the uniform convergence of all its summands:as .

Regardless of the allocation of the new individuals, the population model is a modified Moran model with a single multiplying parent. Thus, to show Eq. (5) one only needs to show for . Compute furtherEquation follows from Eq. (21) and, for the first factor, from being a null sequence.

Now, consider . Then, we get the offspring population by sampling individuals out of , from which share one common parent. Thus, this is again a modified Moran model, where is conditionally hypergeometrically distributed with . Using the factorial moment of the hypergeometric distribution leads to

Remark 6

In Proposition 2, if additionally exists for all , Eq. (15) is satisfied, too (since Lemma 2 holds).

Finally, the following lemma provides sufficient conditions for shifting the time-change from pre-limit to limit.

Lemma 4

Assume that for a Cannings model with variable population sizes , the discrete n-coalescents satisfy in the Skorohod-sense as , where is a -n-coalescent and is defined via Eq. (1). Further assume that Eq. (4) is satisfied for a positive real function and that for a function for or for a constant . Then, the convergence can be equivalently expressed as in the Skorohod-sense, where , where is used if .

is the pseudo-inverse of , sosince for a sequence of functions, the inverses converge iff the original functions converge and since cf has the inverse . The shift by -1 does not alter the limit here, since its effect vanishes as due to the multiplication with . It is important to note here that any terms of order can be omitted when computing . Thus, we can replace by and even by for a constant . Since , analogous to Griffiths and Tavare (1994), we can show, for ,where for convergence, observe that there is pointwise convergencefor inside the integral and that bounded convergence is applicable since Eq. (4) ensures that the integrand is in . If f(x) is a logarithm, we have, using defined by for with ,

Remark 7

The integral representation of the time change is a deterministic version of the coalescent intensity from Kaj and Krone (2003, Sect. 1.3), just applied to Cannings models leading to non-Kingman -n-coalescents.

As described in Möhle (1998, Section 4), the time-changed -n-coalescent limit can also be expressed by its infinitesimal rates for a merger of k of n present lineages. This is also the form in which the limit process of the diploid umbrella model from Koskela and Wilke Berenguer (2019) is given.

Conditioned that the limit coalescent has at time coalesced into a state with b blocks, what is the distribution of the waiting time T for the next coalescence event? If , this means that in the non-rescaled -n-coalescent , we wait for the next coalescence. This waiting time in is exponentially distributed with parameter (total rate of coalescence). Thus,

Proof of Corollary 3

The form of is a direct consequence of Lemma 4, since . Then, plugging into Eq. (23) yields the distribution for the next coalescence event, the Gompertz distribution parameters as e.g. described in Lenart (2014, Eq. 3) can be read off.

Proofs of convergence to a time-changed coalescent: modified Moran models

The modified Moran models used in Theorems 1 and 2 were introduced in Huillet and Möhle (2013, Prop. 4), the latter model with a small modification to ensure that there is always a parent with at least two offspring, see also Huillet and Möhle (2013, Example 4.1).

Proof of Theorem 1

Assume that as also holds for any subsequence. If not, restrict to a subsequence for which this is true and define the limit only along this subsequence.

First, we verify that , thus converges to 0 and that the fixed-N model converges to the -n-coalescent. Let be the coalescence probability in a fixed-N modified Moran model with as the number of offspring of the multiplying parent. This ensures . Moreover, the assumptions made ensure that has a lower bound , so we can define s.t. for any . Then, the following is satisfied as and where Eq. uses , which follows from Eq. (18) with . This establishes the convergence to the -n-coalescent in the fixed-N case. Now we assume variable population sizes . First, observe that, since , it is enough to add occasionally a single individual from generation r to to generate any population size changes allowed in Eq. (4) including bottlenecks which are instantaneous on the coalescent time scale. This single individual can then be added as offspring of a non-multiplying parent from the fixed- model or as an offspring of the already multiplying parent. To see the latter, observe that . Then, as in Huillet and Möhle (2013, third remark p. 8), one hasIf both , the equation above shows that . If but does, we still have as . Thus, Proposition 1 allows one to add the one individual also to the already multiplying parent.

Thus, we have verified all conditions but Eq. (13) to apply Proposition 2. However, this follows from regularly varying. Since , we can also shift the non-linear time-change to the limit due to Lemma 4.

For the proof of the next theorem, we use the following

Lemma 5

Let and let be distributed as in Eq. (6) for any . Let satisfy Eq. (4). Then, Eq. (13) is satisfied.

Equation (18) shows , where is the total transition rate for the first jump of a -N-coalescent. Without restriction, assume (for , an analogous proof provides bounds for ). Further assume that the N-coalescent is just the restriction of the -coalescent on individuals . Any merger in the N-coalescent is then also a merger in the -coalescent, which shows . In contrast, the first merger in the -coalescent is only a merger in the N-coalescent if it features at least two individuals from . The probability of this is bounded from below by the probability that the first two of the blocks merged in the N are from . This implies

Proof of Theorem 2

In the fixed-N case, Eq. (6) implies that necessarily needs that as . This is equivalent to , see (Pitman 1999, Eq. 7). Thus, convergence to the -n-coalescent is shown in (Huillet and Möhle 2013, Prop. 3.4). Now, switch to variable population sizes . Since as , it is enough to add one individual per generation to cover any population growth profile covered by Eq. (4). This can always be done by letting a parent not reproducing in the fixed-size model reproduce (once). To add as further offspring of the multiplying parent, assume as . Then, as shown in the proof of Theorem 1. Thus, Proposition 1 provides that at most for arbitrary individuals can be added per generation to the already multiplying parent. This allows for adding up to any fixed number individuals per generation. Additionally, Eq. (13) is satisfied due to Lemma 5. We can thus apply Proposition 2, with an arbitrary allocation of additional individuals that yields a modified Moran model.

Proof of: if Theorem2holds, Eq. (8) implies Eq. (15) We just need to show that the condition in Remark 6 is equivalent to Eq. (8). From the proof of Lemma 4 [the arguments surrounding Eq. (22)] combined with Eq. (5), we see that existence of is equivalent to the existence of .

To prove the Corollaries 1 and 2, we collect some properties of the modified Moran models with with given by Eq. (6) leading to Beta-(a, b)-coalescents for , . From Huillet and Möhle (2013, Eq. (10) +  Corollary A.1),

Proof of Corollary 1

From Eq. (25), it follows that for , satisfies as . Thus, such -n-coalescents are covered by Theorem 2. Additionally from Eq. (25), has a form that is covered by Lemma 4, which allows us to shift the time-change in Theorem 2 to the limit coalescent and also shows the form of in Corollary 1.

Proof of Corollary 2

We reiterate the proof of Theorem 2. Let for , which satisfies . Thus, in the fixed-N case, again (Huillet and Möhle 2013, Prop. 3.4) ensures the convergence of the discrete genealogies to the -n-coalescent when properly rescaled as . Furthermore, Lemma 5 shows that Eq. (13) is satisfied Since , we can use to satisfy (4). Thus, we only need to show that the population size increase per generation does not violate the conditions of Proposition 2. Indeed,individuals at most have to be added. These can be added as additional offspring of the multiplying parent from the fixed- model, if the condition to apply it from Lemma 2 are met. For considered here, one has , see Eq. (25). From Lemma 2 we see that then we are allowed to add individuals. Huillet and Möhle (2013, Remark p. 9) shows for a constant , so such growth is indeed covered (and we can then still use and add the other individuals to non-reproducing parents from the fixed- model). Thus, we can establish convergence using Proposition 2 and shift the time-change to the limit using Lemma 4, since is essentially a negative power of N.

Proof of Proposition 1

For , Eq. (6) shows , so the modified Moran model is the normal Model model in this case. Since as , Theorem 2 applies. Since , we can apply Lemma 4 to shift the time-change to the coalescent limit.

Proofs of converging to a time-changed coalescent: model from Schweinsberg (2003)

Proof of Lemma 1

It suffices to reiterate the proof of Schweinsberg (2003, Lemma 5) briefly. For , consider the generating function . Let . Then, fulfillsSince and , there exists and so that . Moreover, for this we find so that for . For such N, as computed above, one getswhere . Setting completes the proof.

Proof of Theorem 3

Recall that, for , the coalescence probability in this (fixed-N) model satisfies , where B is the Beta function, see Schweinsberg (2003, Lemma 13). For , instead , see Schweinsberg (2003, Lemma 16). Check the conditions necessary to apply Lemma 2: the model and the assumptions above satisfy as , (4) and, since is regularly varying, also (13). The changes of population sizes from generation to generation are enough to cover instantanous population size changes on the coalescent time scale (and these are the most extreme changes allowed in Eq. (4)): for a (coalescent time) instantaneous change of size mN, one can set as for generations. Thus, only (5) needs to be verified. Since offspring are sampled from potential offspring, the transition probabilities of the discrete coalescent can be formulated analogously to Eq. (2) asThis means one just needs to show thatwhich follows fromuniformly in r. To show the latter, uniform convergence, proceed as follows. First, observe that, since (13) holds,is uniformly bounded (again, since is bounded from below by , there is uniform convergence in r of the first factor as ). Thus, we only need to show . For this, observe that the function for is strictly increasing (decreasing) if (if ). This shows that there are so thatThis implies that it is sufficient to show as for any , which follows from uniformly in r. Further computation showsSince , this vanishes uniformly. Thus, Lemma 2 can be applied, establishing convergence of . Lemma 4 then ensures that the time-change can be shifted to the limit, since is either essentially a negative power or a logarithm of N.