Entrainment in the master equation.

The master equation plays an important role in many scientific fields including physics, chemistry, systems biology, physical finance and sociodynamics. We consider the master equation with periodic transition rates. This may represent an external periodic excitation like the 24 h solar day in biological systems or periodic traffic lights in a model of vehicular traffic. Using tools from systems and control theory, we prove that under mild technical conditions every solution of the master equation converges to a periodic solution with the same period as the rates. In other words, the master equation entrains (or phase locks) to periodic excitations. We describe two applications of our theoretical results to important models from statistical mechanics and epidemiology.

Introduction

Consider a physical system that can be in one of exactly N possible configurations and let x(t) denote the probability that the system is in configuration i at time t. We record the probabilities of all configurations at time t by the (column) state-vector . Every entry of this vector takes values in the closed interval [0,1].

The master equation describes the time evolution of these probabilities. It can be explained intuitively as describing the balance of probability currents going in and out of each possible state. To formulate the master equation for a specific model, one needs to know the rates of transition p from configuration i to configuration j. A rigorous derivation of the master equation for a chemically reacting gas-phase system that is kept well stirred and in thermal equilibrium is given in [1]. The master equation plays a fundamental role in physics (where it is sometimes referred to as the Pauli master equation), chemistry, systems biology, sociodynamics and more. (For example, see the monographs [2,3] for more details.)

In this paper, we treat the general case where the transition rates p may depend on both time t and on the probability distribution x(t) at time t. The resulting system of differential equations (see (1.2) below) constitutes a time-varying nonlinear dynamical system.

Note that even the special case, where the transition rates do not depend on the state x (the master equation (1.2) is then linear), is of general interest because it is intimately connected with the theory of Markov processes on finite configuration spaces. The relation is the following: such a Markov process is uniquely determined by an initial probability distribution on the configurations 1,…,N and by transition probabilities (Q(t)) that denote the probabilities to be in configuration i at time t given that the system is in configuration j at time τ4,5].

In many physical systems, the number of possible configurations N can be very large. For example, the well-known totally asymmetric simple exclusion principle TASEP model (e.g. [6,7] and the references therein) includes a lattice of n consecutive sites, and each site can be either free or occupied by a particle, so the number of possible configurations is N=2. In such cases, simulating the master equation and numerically calculating its steady state may be difficult even for small values of n and special methods must be applied (e.g. [7,8]).

Here, we are interested in deriving theoretical results that hold for any N. Specifically, we consider the case where the transition rates p(t,x) are periodic in time t with a common period T>0. In this situation, we arrive at a T-periodic master equation. For such systems, we consider the problem of entrainment (or phase-locking):

Problem 1.1

Given a system described by a T-periodic master equation, determine if for every initial condition the probabilities x(t), i=1,…,N, converge to a periodic solution with period T. If this is so, determine if the periodic solution is unique or not.

In other words, if we view the transition rates as a T-periodic excitation, then the problem is to determine if the state of the system entrains, that is, converges to a periodic trajectory with the same period T. If this is so, an important question is whether there exists a unique periodic trajectory γ and then every solution converges to γ.

Entrainment is important in many natural and artificial systems. For example, organisms are often exposed to periodic excitations like the 24 h solar day and the periodic cell-cycle division process. Proper functioning often requires accurate entrainment of various biological processes to this excitation [9]. For example, cardiac arrhythmia is a heart disorder occurring when every other pulse generated by the sinoatrial node pacemaker is ineffective in driving the ventricular rhythm [10].

Epidemics of infectious diseases often correlate with seasonal changes and the required interventions, such as pulse vaccination, may also need to be periodic [11]. In mathematical population models, this means that the so-called transmission parameter is periodic, with a period of 1 year, and entrainment means that the spread of epidemics converges to a periodic pattern with the same period. As another example, traffic flow is often controlled by periodically varying traffic lights. In this context, entrainment means that the traffic flow converges to a periodic pattern with the same period as the traffic lights. This observation could be useful for the study of the green wave phenomenon [12]. Another example, from the field of power electronics, involves connecting a synchronous generator to the electric grid. The periodically varying voltage in the grid may be interpreted as a periodic excitation to the generator, and proper functioning requires the generator to entrain to this excitation (e.g. [13] and the references therein).

Our main results provide affirmative answers for problem 1.1 under quite general assumptions. Basic regularity assumptions on the transition probabilities that are required throughout the paper are summarized in assumption 2.1. In theorem 2.2, we then formulate with (2.5), see also (2.3), a condition that guarantees entrainment. It is observed in corollary 2.3 that condition (2.5) is always satisfied in the linear case. Uniqueness of the periodic attractor is shown in theorem 2.5 under the additional assumption of irreducibility, a condition that is well known in the theory of Markov processes (a definition of irreducibility is provided just before the statement of theorem 2.5).

In the special case of time-invariant rates, problem 1.1 reduces to determining if every solution converges to a steady state, and whether there exists a unique steady state. Indeed, time-invariant rates are T-periodic for any T>0 and thus entrainment means convergence to a solution that is T-periodic for any T>0, i.e. a steady state. This basic observation is the content of corollary 2.4.

As the xs represent probabilities, for all t≥t0. The structure of the master equation guarantees that if x(t) satisfies (1.1) at time t=t0, then (1.1) holds for all t≥t0 even when the xs are not necessarily linked to probabilities (see (1.9) below). Our results also hold of course in this case. The next example demonstrates such a case.

Example 1.2

An important topic in sociodynamics is the formation of large cities due to population migration. Haag [2], ch. 8 considers a master equation describing the flow of individuals between N settlements. The transition rates p in this model represent the probability per time unit that an individual living in settlement i will migrate to settlement j. A mean-field approximation of this master equation yields a model in the form (1.2), where x represents the average density at settlement i, and , with k>0. This models the fact that the rate of transition from settlement i to settlement j increases when the population in settlement j is larger than in i, i.e. the tendency of individuals to migrate to larger cities. Note that the rates here are state-dependent, but not time-dependent. However, it is natural to assume that migration decisions depend on the season. For example, the tendency to migrate to colder cities may decrease (increase) in the winter (summer). This can be modelled by adding time dependence, say, changing the scaling parameters k to functions k(t) that are periodic with a period of 1 year. Then the transition rates depend on both state and time, and are periodic.

It is important to note that, in general, nonlinear dynamical systems do not entrain to periodic excitations. Indeed, Nikolaev et al. [14] discusses two ‘simple looking’ nonlinear dynamical systems whose response to periodic forcing is chaotic (rather than periodic). Moreover, these systems commonly appear as components of larger sensing and signal transduction pathways in systems biology. This highlights the importance of proving that entrainment does hold in specific classes of dynamical systems.

Although entrainment has attracted enormous research attention, it seems that it has not been addressed before for the general case of systems modelled using a T-periodic master equation. Here we apply the theory of cooperative dynamical systems admitting a first integral to derive conditions guaranteeing that the answer to problem 1.1 is affirmative. In §3, we describe two applications of our approach to important systems from statistical physics. The first is the totally asymmetric simple exclusion process (TASEP). This model has been introduced in the context of biocellular processes [15] and has become the standard model for the flow of ribosomes along the mRNA molecule during translation [16,17]. More generally, TASEP has become a paradigmatic model for the statistical mechanics of non-equilibrium systems [6,7,18]. It is in particular used to study the stochastic dynamics of interacting particle systems such as vehicular traffic [19].

The second application is to an important model from epidemiology called the stochastic susceptible–infected–susceptible (SIS) model.

The remainder of this paper is organized as follows. In the following subsection, we briefly explain the central mathematical concepts used in the proofs of our main results theorems 2.2 and 2.5. The exact mathematical formulation of these results is provided in §2. The subsequent section then describes the two applications to statistical physics and to epidemiology mentioned above. This is followed by a brief discussion of the significance of the results and an outlook on possible future directions of research in §4. The appendix includes all the proofs. These are based on known tools, yet we are able to use the special structure of the master equation to derive stronger results than those available in the literature on monotone dynamical systems.

Formulation of master equation and concepts of proof

We begin by formulating the master equation, determined by given transition rates p(t,x)≥0, that governs the evolution of the probability distribution on N configurations:

In lemma A.4 below, it is shown in a slightly more general setting that system (1.2) defines a flow in the set of probability distributions . For T>0, we refer to (1.2) as the T-periodic master equation if for all i,j, all t and all x. Note that this includes the case where one (or several) of the rates is (are) T-periodic with T>0, and the other rates are time-independent, as a time-independent function satisfies (1.3) for all T. Clearly, from (1.3), it also follows that p(t+kT,x)=p(t,x) for any integer k. To make the period a well-defined notion one, therefore, often requires that the period is the minimal real number T>0 for which (1.3) is satisfied. Then constant functions do not have a period. As we want to include here the case of time-independent transition rates, e.g. corollary 2.3, we do not require the minimality of the common period T in (1.3).

For small values of N, it is sometimes possible to solve the master equation and then analyse entrainment directly. The next example demonstrates this.

Example 1.3

Consider the master equation (1.2) with N=2 and continuous time- (but not state-) dependent rates, i.e. p=p(t)≥0. Then (1.2) can be written as Assume also that all the rates are periodic with period T>0. Using the fact that x1(t)+x2(t)≡1 yields Recall that x1(0),x2(0)∈[0,1] with x1(0)+x2(0)=1. Equation (1.5) implies that x1(t)∈[0,1], for all t≥0, and thus x2(t)∈[0,1], for all t≥0. Solving (1.5) yields where . To analyse if every solution converges to a periodic solution, we consider two cases.

Case 1: If p12(t)+p21(t)≡0, then (1.6) yields x(t)≡x(0), i.e. every point in the state-space is an equilibrium point. This means in particular that every solution is a periodic solution with period T.

Case 2: Assume that there exists a time t*∈[0,T) such that p12(t*)+p21(t*)>0. (Note that by continuity this in fact holds on a time interval that includes t*.) The solution (1.6) is periodic with period T if and only if x1(T)=x1(0), i.e. if and only if It is straightforward to show that the right-hand side in this equation is in [0,1], so in this case, there exists a unique periodic trajectory γ(t), with γ1(0) equal to the expression in (1.7) and γ2(0)=1−γ1(0). To determine if every trajectory converges to γ, let z(t):=x(t)−γ(t), that is, the difference between the solution emanating from x0 and the unique periodic solution. Then As p12(t)+p21(t) is non-negative for all t, positive on a time interval and T-periodic, z1(t) converges to zero and we conclude that any trajectory of the system converges to the unique periodic solution γ.

Of course, when N>2 and the rates depend on both t and x, this type of explicit analysis is impossible, and the proof of entrainment requires a different approach.

In general, proving that a time-varying nonlinear dynamical system entrains to periodic excitations is non-trivial. Rigorous proofs are known for two classes of dynamical systems: contractive systems and monotone systems with additional structure like a tridiagonal Jacobian [20] or admitting a first integral.

A system is called contractive if any two trajectories approach one another at an exponential rate [21,22]. Such systems entrain to periodic excitations [9,23]. An important special case is asymptotically stable linear systems with an additive periodic input u, that is, systems in the form with , a Hurwitz matrix,[1] and . In this case, x(t) converges to a periodic solution γ(t) and it is also possible to obtain a closed-form description of γ using the transfer function of the linear system [24]. We note that even in the case that the ps in (1.2) do not depend on x, i.e. when (1.2) is linear in x, the master equation is not of the form (1.8) because the periodic influence in (1.2) enters through the transition rates p and not through an additive input channel.

Next, we turn to the notion of a first integral. Define by H(y):=y1+⋯+y. Equation (1.2) implies that so that the value of H(x(t)) remains constant under the flow, that is, H is a first integral of (1.2).

A system is called monotone if its flow preserves a partial order, induced by an appropriate cone K, between its initial conditions [25]. An important special case of monotone systems is cooperative systems for which the cone K is the positive orthant. To explain this, define a partial ordering between vectors by a≤b if every entry of a is smaller or equal to the corresponding entry of b. For example, for vectors in but A system is called cooperative if for any two initial conditions a,b with a≤b the solutions satisfy x(t,a)≤x(t,b) for any time t≥0. In other words, the dynamics preserves the ordering between the initial conditions.

Cooperative systems that admit a first integral entrain to periodic excitations. It is interesting to note that proofs of this property often follow from contraction arguments [26].

The master equation (1.2) is, in general, not contractive, although as we will show in theorem A.8 below it is on the ‘verge of contraction’ with respect to the ℓ1 vector norm (see [27] for some related considerations). However, (1.2) admits a first integral and is often a cooperative system (see theorem A.9 below). In particular, when the rates do not depend on the state, i.e. p=p(t), then (1.2) is always cooperative.

Main results

We begin by specifying the exact conditions on (1.2) that are assumed throughout. For a set S, let int(S) denote the interior of S. For any time t, x(t) is an N-dimensional column vector that includes the probabilities of all N possible configurations. The relevant state-space is thus For an initial time t0≥0 and an initial condition x(t0), let x(t;t0,x(t0)) denote the solution of (1.2) at time t≥t0. For our purposes, it will be convenient to assume that the vector field associated with system (1.2) is not only defined on the set Ω, but on all the closed positive cones

Throughout this paper, we assume that the following condition holds.

Assumption 2.1

There exists T>0 such that the transition rates p(t,x) are: continuous and non-negative on ; continuously differentiable with respect to x on and the derivative admits a continuous extension onto ; and are jointly periodic with period T, that is, for all i,j, all and all .

Let relint(Ω) denote the relative interior of Ω, that is, Note that if the rates are only defined on x∈Ω, with partial derivatives with respect to x on relint(Ω) with continuous extensions to Ω, then they can be extended to so that the conditions in assumption 2.1 hold. For example, by defining them to be constant on rays through the origin and multiplied by a cut-off function χ(|x|1), where χ is a smooth function with a compact support in , satisfying χ(s)=1 for s=1, and where |x|1 denotes the ℓ1-norm of x.

We now determine the conditions guaranteeing that (1.2) is a cooperative dynamical system. Note that (1.2) can be written as where is a matrix with entries The Jacobian of the vector field f is the N×N matrix where B is the matrix with entries . Recall that a matrix is called Metzler if every off-diagonal entry of M is non-negative. It follows that if , for all i≠j, all t≥t0 and all x∈Ω then J(t,x) is Metzler for all t≥t0 and all x∈Ω.

We can now state our first result.

Theorem 2.2

Suppose that Then, for any t0≥0 and any x(t0)∈Ω the solution x(t;t0,x(t0)) of (1.2) converges to a periodic solution with period T.

If the rates depend on time, but not on the state, i.e. p=p(t) for all i,j, then the condition in theorem 2.2 always holds, and this yields the following result.

Corollary 2.3

If p=p(t) for all i,j, then for any t0≥0 and any x(t0)∈Ω the solution x(t;t0,x(t0)) of (1.2) converges to a periodic solution with period T.

Thus, theorem 2.2 describes a technical condition guaranteeing entrainment, and this condition automatically holds in the case where all the rates are functions of time only.

If the rates depend on the state, but not on time then we may apply theorem 2.2 for all T>0. Thus, the trajectories converge to a periodic solution with an arbitrary period, i.e. a steady state. This yields the following result.

Corollary 2.4

If p=p(x) for all i,j and in addition condition (2.5) holds, then for any t0≥0 and any x(t0)∈Ω the solution x(t;t0,x(t0)) of (1.2) converges to a steady state.

In some applications, it is useful to establish that all trajectories of (1.2) converge to a unique periodic trajectory. Recall that a matrix , with n≥2, is said to be reducible if there exists a permutation matrix P∈{0,1}, and an integer 1≤r≤n−1 such that , where , , and is a zero matrix. A matrix is called irreducible if it is not reducible. It is well known that a Metzler matrix M is irreducible if and only if the graph associated with the adjacency matrix of M is strongly connected [28], theorems 6.2.14 and 6.2.24.

Theorem 2.5

Suppose that the conditions in theorem 2.2 hold and, furthermore, that there exists a time t*≥t0 such that A(t*,x)+B(t*,x) is an irreducible matrix for all x∈Ω. Then (1.2) admits a unique periodic solution γ in Ω, with period T, and every solution x(t;t0,x(t0)) with x(t0)∈Ω converges to γ at an exponential rate.

Example 2.6

Consider the system in example 1.3. This is of the form (2.2) with If there exists a time t* such that p12(t*),p21(t*)>0 then A(t*) is irreducible. We conclude that, in this case, all the conditions in theorem 2.5 hold, so the system admits a unique T-periodic solution γ and every trajectory converges to γ. This agrees of course with the results of the analysis in example 1.3 above where we arrived at the same conclusion under the slightly weaker assumption that p12(t*)+p21(t*)>0.

The next section describes an application of our results to two important models.

Applications

Entrainment in totally asymmetric simple exclusion process

The totally asymmetric simple exclusion process (TASEP) is a stochastic model of particles hopping along a one-dimensional chain. A particle at site k hops to site k+1 (the next site on the right) with an exponentially distributed probability[2] with rate h, provided the site k+1 is not occupied by another particle. This simple exclusion property generates an indirect link between the particles and allows to model the formation of traffic jams. Indeed, if a particle ‘gets stuck’ for a long time in the same site, then other particles accumulate behind it. At the left end of the chain particles enter with a certain entry rate α>0 and at the right end particles leave with a rate β>0 (figure 1).

Figure 1.

The TASEP model includes particles randomly hopping along a chain of n sites. Note that the particle in site 1 cannot hop forward because site 2 contains a particle.

As pointed out in the introduction, TASEP has become a standard tool for modelling ribosome flow during translation, and is a paradigmatic model for the statistical mechanics of non-equilibrium systems. We note that in the classical TASEP model the rates α, β and h are constants, but several papers considered TASEP with periodic rates [29-31] that can be used, for example, as models for vehicular traffic controlled by periodically varying traffic signals.

It was shown in [32] that the dynamic mean-field approximation of TASEP, called the ribosome flow model (RFM), entrains. However, the RFM is not a master equation and the proof of entrainment in [32] is based on different ideas. For more on the analysis of the RFM, see e.g. [33-36].

For a chain of length n, denoting an occupied site by 1 and a free site by 0, the set of possible configurations is {0,1}, and thus the number of possible configurations is N=2. The dynamics of TASEP can be expressed as a master equation with transition rates p that depend on the values α, β and h, i=1,…,n. For the sake of simplicity, we will show this in the specific case n=2, but all our results below hold for any value of n.

When n=2, the possible configurations of particles along the chain are C1:=(0,0), C2:=(0,1), C3:=(1,0) and C4:=(1,1). Let x(t) denote the probability that the system is in configuration C at time t, for example, x1(t) is the probability that both sites are empty at time t. Then x1 may decrease (increase) due to the transition C1→C3 [C2→C1], i.e. when a particle enters the first site (a particle in the second site hops out of the chain). This gives

Similar considerations for all configurations lead to the master equation , with If the entry, exit and hopping rates are time dependent and periodic, all with the same period T, one easily sees that the resulting master equation satisfies assumption 2.1 as well as all assumptions of theorem 2.2. Hence, we conclude that every solution of the master equation starting in Ω converges to a periodic solution with period T. Moreover, if there exists a time t* such that α(t*),β(t*),h1(t*)>0, then A(t*) is irreducible. Hence, the conditions of theorem 2.5 are also satisfied, so we conclude that the periodic solution is unique and convergence takes place at an exponential rate. It is not difficult to show that the same holds for TASEP with any length n.

Example 3.1

When n=3, the possible particle configurations are C1:= (0,0,0), C2:=(0,0,1), C3:=(0,1,0), C4:=(0,1,1),…,C8:=(1,1,1). Let x(t) denote the probability that the system is in configuration C at time t. The TASEP master equation in this case is , with We simulated this system with the rates and initial condition . Note that all the rates here are jointly periodic with period 2π. Figure 2 depicts x1(t) (black square), x4(t) (red asterisk) and x8(t) (blue circle) as a function of t (we depict only three xs to avoid cluttering the figure). Note that as the entry rate α(t) is maximal and the exit rate β(t) is minimal at t=0, the probability x8(t) [x1(t)] to be in state (1,1,1) [(0,0,0)] quickly increases (decreases) near t=0. As time progresses, the probabilities converge to a periodic pattern with period 2π.

Figure 2.

Probabilities x1(t) (black square), x4(t) (red asterisk) and x8(t) (blue circle) as a function of t in example 3.1.

Entrainment of the probabilities x has consequences for other quantities of interest in statistical mechanics. For instance, an important quantity is the occupation density, i.e. the probability that site k is occupied, often denoted by 〈τ〉, cf. [37,38]. Denoting the kth component of the configuration C∈{0,1} by C, a straightforward computation reveals that It is thus immediate that the occupation densities also converge to a unique periodic solution.

This phenomenon has already been observed empirically in [29] that studied a semi-infinite and finite TASEP coupled at the end to a reservoir with a periodic time-varying particle density. This models, for example, a traffic lane ending with a periodically varying traffic light. The simulations in [29] suggest that this leads to the development of a sawteeth density profile along the chain, and that ‘The sawteeth profile is changing with time, but it regains its shape after each complete period…’ [29], p. 011122-2 (see also [30,31] for some related considerations).

Our results can also be interpreted in terms of the particles along the chain in TASEP. As the expectation of the occupation densities 〈τ〉 converges to a periodic solution, this means that, in the long term, the TASEP dynamics ‘fluctuates’ around a periodic ‘mean’ solution (see e.g. the simulation results depicted in fig. 5 in [32]). Moreover, in [30,31] it was found for closely related models that the limiting periodic density profiles (whose existence is also guaranteed by our results) have an interesting structure that depends in a non-trivial way on the frequency of the transition rates.

Entrainment in a stochastic susceptible–infected–susceptible model

The stochastic SIS model plays an important role in mathematical epidemiology [39]. But, as noted in [40], it is usually studied under the assumption of fixed contact and recovery rates. Here, we apply our results to prove entrainment in an SIS model with periodic rates.

Consider a population of size N divided into susceptible and infected individuals. Let S(t) [I(t)] denote the size of the susceptible (infected) part of the population at time t, so that S(t)+I(t)≡N. We assume two mechanisms for infection. The first is by contact with an infected individual and depends on the contact rate a(t). The second is by some external agent (modelling, say, insect bite) with rate c(t). The recovery rate is b(t) (figure 3). We assume that a(t), b(t) and c(t) are continuous and take non-negative values for all time t.

Figure 3.

Three mechanisms for transitions between the classes of susceptible and infected and the associated rates.

If I(t)=n (so S(t)=N−n), then the probability that one individual recovers in the time interval [t,t+dt] is b(t)n dt+o(dt), and the probability for one new infection to occur in this time interval is a(t)n((N−n)/N) dt+c(t)((N−n)/N) dt+o(dt). For n∈{0,…,N}, let P(t) denote the probability that I(t)=n. This yields the master equation: for n∈{0,1,…,N}, where we define P−1=P:=0, and for simplicity omit the dependence on t. This set of N+1 equations may be written in matrix form as where x:=[P0 P1 … P]′∈[0,1], M:=P−D, with D:=diag(0,b,2b,…,Nb), and P is the (N+1)×(N+1) matrix: where q:=1−i/N. Note that M(t) is Metzler, as a(t),b(t) and c(t) are non-negative for all t. Thus, theorems 2.2 and 2.5 yield the following result.

Corollary 3.2

If a(t), b(t) and c(t) are all T-periodic then any solution of (3.1) with converges to a T-periodic solution. Furthermore, if there exists a time t*≥0 such that then there exists a unique T-periodic solution γ in Ω and every solution converges to γ.

Example 3.3

Consider the stochastic SIS model with N=3, a(t)=1, and . These rates are non-negative and jointly T-periodic for T=2π and clearly there exists t*≥0 such that b(t*)c(t*)>0. Figure 4 depicts P(t), i=0,1,2, (note that P3(t)=1−P0(t)−P1(t)−P2(t)) as a function of time t for the initial condition . It may be seen that every P(t) converges to a periodic solution with period 2π. Taking other initial conditions x(0)∈Ω yields convergence to the same periodic solution.

Figure 4.

Probabilities P0(t) (black asterisk), P1(t) (red circle) and P2(t) (blue square) as a function of t in example 3.3.

Note that if the irreducibility condition (3.2) does not hold then the system may have several periodic solutions. To see this, consider, for example, the case b(t)=c(t)≡0. Let denote the vector with entry i equal to one and all other entries zero. Then both x(t)≡e1 and x(t)≡e are (periodic) solutions of the dynamics.

Discussion

In his 1929 paper on periodicity in disease prevalence, Soper [41] states: ‘Perhaps no events of human experience interest us so continuously, from generation to generation, as those which are, or seem to be, periodic.’ Soper also raised the question of whether the observed periodicity in epidemic outbreaks is the result of a ‘seasonal change in perturbing influences, such as might be brought about by school break-up and reassembling, or other annual recurrences?’ In modern terms, this amounts to asking whether the solutions of the system describing the dynamics of the epidemics entrain to periodic variations in the transmission parameters.

Here, we studied entrainment for dynamical systems described by a master equation. We considered a rather general formulation where the transition rates may depend on both time and state. Also, we did not assume any symmetry conditions (e.g. detailed balance conditions [3], ch. V) on the rates. We also note that this formulation implies similar results for nonlinear systems. Indeed, consider the time-varying nonlinear system: and assume that f(t,0)=0, for all t. Let J(t,x):=(∂f/∂x)(t,x) denote the Jacobian of the vector field. Then where . If A(t,x) has the form (2.3) then the results above can be applied to (4.1).

We proved that entrainment indeed holds under quite mild technical conditions. This follows from the fact that the master equation is a cooperative dynamical system admitting a first integral. Owing to the prevalence of the master equation as a model for natural and artificial phenomena, we believe that this result will find many applications. To demonstrate this, we described two applications of our results: a proof of entrainment in TASEP and in a stochastic SIS model.

The rigorous proof that the solutions of the master equation entrain is of course a necessary first step in studying the structure of the periodic trajectory (or trajectories), and its dependence on various parameters. Indeed, in many applications it is of interest to obtain more information on the periodic trajectory, e.g. its amplitude. Of course, one cannot expect in general to obtain a closed-form description of the limit cycle. However, for contractive dynamical systems there do exist efficient methods for obtaining a closed-form approximation of the limit cycle accompanied by explicit error bounds [23]. Developing a similar approach for the attractive limit cycle of the master equation may be an interesting topic for further research. In the specific case of TASEP with fixed rates, there exists a powerful representation of the steady state in terms of a product of matrices [7,37]. It may be of interest to try and represent the periodic steady state using a similar product, but with matrices with periodic entries. This could be used in particular to study the effects of periodic perturbations to the boundary-induced phase transitions that have been observed for TASEP in [42].