Phase space volume scaling of generalized entropies and anomalous diffusion scaling governed by corresponding non-linear Fokker-Planck equations.

Many physical, biological or social systems are governed by history-dependent dynamics or are composed of strongly interacting units, showing an extreme diversity of microscopic behaviour. Macroscopically, however, they can be efficiently modeled by generalizing concepts of the theory of Markovian, ergodic and weakly interacting stochastic processes. In this paper, we model stochastic processes by a family of generalized Fokker-Planck equations whose stationary solutions are equivalent to the maximum entropy distributions according to generalized entropies. We show that at asymptotically large times and volumes, the scaling exponent of the anomalous diffusion process described by the generalized Fokker-Planck equation and the phase space volume scaling exponent of the generalized entropy bijectively determine each other via a simple algebraic relation. This implies that these basic measures characterizing the transient and the stationary behaviour of the processes provide the same information regarding the asymptotic regime, and consequently, the classification of the processes given by these two exponents coincide.

Introduction

Real world processes are often characterized by the presence of a large number of interacting phenomena at multiple time- or length scales[1], and thus, they are usually described by stochastic models that are strongly interacting or history-dependent[2-5]. A way to understand and classify these processes in terms of stationary and non-stationary probability densities is to generalize the concepts of statistical mechanics that already proved to be very powerful for describing weakly interacting, ergodic and Markovian systems[6,7]. One such concept is entropy, which assigns a likelihood to macrostates, that is, to stationary distributions over microstates. Maximizing this likelihood, possibly in the presence of external constraints, yields the most probable stationary distribution characterizing the system, called the Maximum Entropy (MaxEnt) distribution, which plays a key role in describing the stationary behaviour of stochastic systems. For example, the Boltzmann-Gibbs entropy form, , where i runs over the microstates, follows from the assumption that the system realizations are independent and distinguishable[8].

In general, however, the realizations are not independent. Instead, their interaction can be macroscopically modelled by a corresponding entropy functional, which in principle can take infinitely many different forms. Similarly to the theory of renormalization group describing critical phenomena, an apprehensive characterization of these entropies can be made by observing what are the relevant and irrelevant parameters as we approach infinite system size[9]. Axiomatic considerations suggest that the asymptotic scaling of the generalized entropy forms with phase space volume provides a meaningful classification of the entropies. This classification is based on the fundamental result by Hanel and Thurner[10] about the entropy functionals S[p] that can be written as a sum of a pointwise function over microstates

As they showed, the first three Shannon-Khinchin (SK1–SK3) axioms[11,12]or, equivalently, in terms of g,with 0 < z < 1 and 0 < c ≤ 1.

SK1 S is continuous in p,

SK2 S is maximal for the uniform distribution, p ≡ 1/W,

SK3 S is invariant under adding a zero-probability state to the system, S(p1, …, p) = S(p1, …, p, p = 0), permit only the following asymptotic scaling relation for any entropic forms:

Hence, the scaling exponent c can be used to parametrize the equivalence classes of the generalized entropy forms. For example, the Boltzmann-Gibbs entropy, where , is corresponding to c = 1, whereas the Tsallis entropy[6] , with 0 < q ≤ 1 is corresponding to c = q. Consequently, each such equivalence class can be represented by a Tsallis entropy. Note that the fourth Shannon-Khinchin axiom S(p) = S(p) + 〈S(p)〉 is not considered in this analysis, therefore, the entropy of a joint distribution p is not always decomposable to the entropy of the marginal p and the entropy of the conditional distribution p, averaged over p.

These considerations suggest that the asymptotic exponent c provides a measure of deviation from ergodic, uncorrelated and Markovian systems regarding its stationary behaviour. Our main motivation here is to understand how this exponent relates to similar macroscopic measures which are, however, characterizing the non-stationary behaviour of the system. One of the main approaches to model the non-stationary behaviour of stochastic processes macroscopically is through partial differential equations governing the time evolution of the probability density p(x, t), called Fokker-Planck equations (FPE)[13-15]. Once specified, the underlying microscopic rules completely determine the form of the FPE. For example, the assumption of memoryless, Gaussian noise and short range interaction between the units gives rise to linear FPEs in the form ofwhere f(x, t) and D(x, t) are called the drift and diffusion coefficients, respectively. If the diffusion coefficient is constant (D(x, t) ≡ D) and the drift is proportional to the spatial derivative of some time-independent external potential u(x), i.e., f(x, t) = −Dβ∂u(x), (4) simplifies to

Specifically, in the presence of no external potential, (5) becomes

Such non-stationary processes can be classified phenomenologically by the scaling of the spread of p(x, t) over time. This can be phrased mathematically as the invariance of p(x, t) under appropriate rescaling of space and time[16,17]:where the scaling factor τ− of the space coordinate keeps the probability density invariant when the timescale is changed as . In general, (7) is satisfied only in the asymptotic limit, i.e., when p(x, t) → 0. Nevertheless, this scaling relation, parametrized by γ, classifies the governing dynamics described by (8). For example, (6) falls into the equivalence class γ = 1/2.

Non-stationary stochastic processes that are characterized by are termed as anomalous diffusion processes. There are two main types of microscopic rules that can lead to anomalous diffusion of the probability density. In one case, the trajectories of the individual units (e.g., particles) remain to be uncorrelated and Markovian, however, other stochastic properties of these trajectories deviate from those of standard Brownian motion[17-20]. Typically these deviations stem from the fact that either the waiting time distribution between successive jumps or the jump length distribution is characterized by having infinite variance or mean. The corresponding FPEs usually include fractional derivatives, hence, these processes are termed as fractional dynamics. We do not consider this type of processes in the rest of the paper. Instead, we focus exclusively on the other type of processes that can lead to anomalous diffusion: the case in which the dynamics of the units is correlated or non-Markovian[3,6,15,21-26]. One way of modelling macroscopically such systems is through FPEs in which the diffusive term, , is replaced by , where F[p], called the effective density, is a given function of the probability density p, which is either derived from microscopic rules or simply defined based on other macroscopic arguments[15,22,25,27]. According to the above, in the following we consider non-linear FPEs that generalize (5) as

Nonlinear FPEs were used in modelling a variety of phenomena in physical, biological and social sciences, such as diffusion in porous media[16,28], surface growth process[28], stellar dynamics[29], bacterial chemotaxis[25] and financial transactions[30].

As we show later, for a given effective density F[p], the asymptotic anomalous diffusion exponent γ can be determined. Therefore, similarly to the phase space volume scaling exponent c of the entropy, the anomalous diffusion exponent γ might also indicate the deviation of the underlying system from being uncorrelated and Markovian. This specifies the goal of this paper, which is to investigate the relation between these two exponents, c and γ, macroscopically characterizing the stationary and non-stationary regime of the process, respectively. In order to relate entropies to FPEs, in this paper we consider continuous entropy forms, which, analogously to (1), are assumed to be written as[25,31,32]where g is asymptotically characterized by (3), u = u(x) is a time-independent scalar function of the space coordinate x (e.g., a potential), and the integration is performed over the range of u(x). The definition given in (9) provides a very general form, and special cases of this entropy functional have already been applied in studies of statistical mechanics of special relativity[33], chemotaxis of biological populations[25], stellar dynamics and two dimensional turbulence[29].

For the sake of consistency between the description of these two regimes, similarly to refs[27,31,32,34,35], we consider cases where the stationary solution of the FPE, given by (8), equals to the maximum entropy distribution according to the generalized entropy. It is instructive to see how the consistency criterion specified above applies to the most well-known case, the Boltzmann-Gibbs entropy, . In this case, the MaxEnt distribution restricted by a constraint on the expected value of u takes the form of . However, this is also equivalent to the stationary solution of the FPE describing ordinary diffusion in the presence of some external potential u(x), given in (5), which is a special case of (8) with F[p] = p. Setting zero net flux at the boundaries yields .

Results

Based on the above, the Boltzmann-Gibbs entropy, belonging to the entropy class c = 1, is corresponding to the Fokker-Planck equation describing simple diffusion, which in turn is a member of the anomalous diffusion scaling class γ = 1/2. A natural question arising based on this observation is the following: Does every entropy belonging to the c = 1 universality class correspond to a generalized Fokker-Planck equation from the anomalous diffusion class γ = 1/2? And does every generalized Fokker-Planck equation belonging to the class γ = 1/2 correspond to an entropy belonging to the c = 1 class? In other words, does c = 1 and γ = 1/2 give rise to the same equivalence class, therefore, bijectively determine each other? In this paper we show that this is true not only for c = 1 and γ = 1/2, but for every c ∈ (0, 1] and , where the exponents c and γ are connected by a simple algebraic relation. This implies that the asymptotic scaling of generalized entropies with phase space volume and the asymptotic anomalous diffusion scaling of the corresponding generalized Fokker-Planck equation classify the processes in the same way, and consequently, they provide the same information about their asymptotic behaviour. In Fig. 1 we show a schematic illustration of the above concept. Our result also provides an asymptotic generalization of the relation derived by Tsallis and Bukman[34] between c and γ for the class of Tsallis entropies.

Figure 1

Summary of the results presented in this paper. We consider macroscopic descriptions of stochastic processes where the stationary and non-stationary regime are related by identifying the MaxEnt distribution with the stationary solution of the Fokker-Planck equation. We show that under these assumptions, two of the most frequently discussed asymptotic classifications of stochastic processes, one based on the scaling of the entropy with the phase space volume and the other based on the anomalous diffusion scaling described by the Fokker Planck equation, coincide. Thus, for any c ∈ (0, 1], the corresponding equivalence class of entropies (represented by an ellipse on the left) bijectively determines an equivalence class of Fokker-Planck equations (shown by an ellipse on the right), in which the anomalous diffusion scaling exponent is constant.

In order to derive a relationship between the asymptotic exponents c and γ, let us first consider the MaxEnt distribution corresponding to entropies given in the form of (9). By following a variational principle approach and taking into account the normalization and expected value constraints we can writewhere the constants are omitted for simplicity and the λ0 and λ1 Lagrange multipliers are introduced for fixing the zeroth and first moment, respectively. From (10) we obtainwhere Λ(p) is the inverse of the MaxEnt distribution corresponding to the entropy defined by g(p). In case of the Boltzmann-Gibbs entropy the inverse of the MaxEnt distribution is given by the (appropriately shifted and rescaled) logarithm function, . Therefore, Λ(p) is usually referred to as the generalized logarithm for any entropy in general[6,36].

Based on a given entropy S[g(p)] and the corresponding generalized logarithm Λ(p), our next step is to find the related generalized Fokker-Planck equations in the form of (8), where F[p] is chosen such that the stationary solution of the equation becomes equivalent to the MaxEnt distribution of the entropy. By replacing u with Λ(p) in (8) according to (11) we obtain that the stationarity condition ∂p = 0 is fulfilled if

The expression p∂Λ(p) in the r.h.s. of (12) can be rewritten using the chain rule ∂ = (∂p)∂ aswhich in turn is equivalent towhere C(x, t) is an arbitrary function which is independent of p. Substituting (14) into (12) yieldswhich gives the general expression for the effective density F aswhere is an arbitrary function that is constant in p. The obtained relation (16) between F and g has already been established in an implicit form in refs[25,31]. In the following, we assume that F has no explicit space- or time-dependence, consequently, it is defined by g up to an additive constant as . In particular, for the Boltzmann-Gibbs entropy and β = λ1, yielding , which results in FBG[p] = p. Since F[p] is formulated based on g(p) in (16), we call the resulting equationas the g-Fokker-Planck equation in order to distinguish it from the many other possible generalizations of FPEs. Note that in general many possible dynamics can lead to the same stationary state. However, as our derivation shows, if the dynamics, given by a non-linear Fokker-Planck equation, is constrained to be in the form of (8), then F[p] is determined by the stationary state, or, equivalently, by g, up to an additive constant.

In the following, let us consider the g-Fokker-Planck equation with no external potential,

We assume that the solution of (18) exists, at least from an appropriate initial condition, and it reaches the asymptotic limit p(x, t) → 0 for all x. In this asymptotic limit, the scaling rule (7) applies to p(x, t). Thus, if we change to the rescaled variables x′ = x/τ and t′ = t/τ, the derivatives according to the new variables can be written as

Using (7) and (19), the g-Fokker-Planck equation with no external potential given in (18) in the rescaled variables can be formulated as

The F[τ−p(x′, t′)] term on the right hand side can be further transformed based on the scaling of g(p) given in (3), where by a change of variable we obtain (being valid for ). By substituting this into (16) we obtain

According to that, the g-Fokker-Planck equation (20) in the rescaled variables yields

Consequently, the dynamics remains to be governed by the original g-Fokker-Planck equation (18) for any τ if and only if the prefactors of both sides are equal for any τ, that is, the exponents must coincide,

By rearranging (23) we obtain the main result of the paperproviding a general relation between the exponent γ related to the anomalous diffusion, characterizing the scaling of p(x, t) in the p(x, t) → 0 limit and the exponent c, describing the scaling of the generalized entropy with the phase space volume. Note that the derivation above only requires g(p) to obey the asymptotic scaling relation (3) and the existence of the solution to (18) with the asymptotic limit p(x, t) → 0 reached for all x.

In order to demonstrate this general result, in Table 1 we list a few different generalized entropy forms from the literature together with the corresponding g-Fokker-Planck equations and the related γ and c exponents. Although the actual algebraic form of the entropies along with their phase space volume scaling, their MaxEnt distributions and the corresponding generalized Fokker-Planck equations are different for any c, their asymptotic anomalous diffusion scaling is completely determined by c via (24). This exemplifies the fact that although the mapping between entropies and Fokker-Planck equations are defined at any (phase space volume or time) scale, any entropy, characterized by asymptotic exponent c, can only be mapped to a Fokker-Planck equation describing anomalous diffusion with asymptotic exponent given by (24). In close connection to Table 1, Fig. 2 shows the finite scale phase space volume scaling of some generalized entropies, illustrating the numerous possible ways of convergence to the asymptotic value c.

Table 1

Generalized entropies , the inverse of their MaxEnt distribution Λ(p), their corresponding g-Fokker-Planck equation (with no external potential), their phase space scaling exponent c ∈ (0, 1] and their anomalous diffusion scaling exponent γ.

Entropy	g(p)	Λ(p)	g-Fokker-Planck equation	c	γ
Boltzmann-Gibbs	−p ln p	ln p	∂_tp=∂x2p	1	12
exponential[37]	p(1−e^p−1p)	e^p−1p(p+1p)−2	∂_tp=∂x2e^p−1p	1	12
Curado[38]	1 − e−bp + p(e−b − 1), b > 0	1−e^b(1−p)b	∂_tp=∂x2[1−e^−bp(bp+1)]	1	12
Tsallis[39]	p−p^qq−1, 0 < q ≤ 1	1−p^q−11−q	∂_tp=∂x2p^q	q	11+q
Kaniadakis[33]	−p^1+κ−p^1−κ2κ, 0 < κ < 1	(1+κ)p^κ−(1−κ)p^−κ2κ−1	∂_tp=∂x2[p^1+κ+p^1−κ]	1 − κ	12−κ
Shafee[40]	−pq ln p, 0 < q ≤ 1	1−p^q−1(1+qlnp)1−2q	∂_tp=∂x2[p^q(1+(q−1)lnp)]	q	11+q
c, d[10]	erΓ(1 + d, 1 − c ln p) − cr,r=11−c+cd, 0 < c ≤ 1, d∈ℝ	r[1 − (1 − c ln p)dpc−1]	∂_tp=∂x2[dΓ(d,1−clnp)−(1−1c)Γ(1+d,1−clnp)]	c	11+c

Different entropies with the same exponent c might have different algebraic forms, different MaxEnt distributions and different corresponding Fokker-Planck equations, but the asymptotic anomalous diffusion scaling exponent γ is always the same. Note that for the sake of comparison, λ0 and λ1 in are set by the conditions Λ(1) = 0 and Λ′(1) = 1. Also the factor prior to are set to 1 for simplicity which can be understood as the time being appropriately rescaled.

Figure 2

Finite scale phase space volume scaling exponent c(p) of some generalized entropies belonging to asymptotic classes c = 0.1, c = 0.5 and c = 1. According to the rigorous formulation of the asymptotic phase space volume scaling, given by (3), g(p) ~ p when p → 0. Based on this, we define c (p) as the slope of the tangent of g(p) on a log-log plot, . The convergence of the curves in the low p regime indicates that the effects of the phase space volume scaling (which is illustrated in the left panel of Fig. 1) are apparent already at finite scales.

Discussion

In this paper, we considered a class of stochastic processes, describing systems possibly composed of strongly interacting units or governed by non-Markovian dynamics, which can be macroscopically modelled by non-linear Fokker-Planck equations in the form of (8). These equations generalize the linear Fokker-Planck equation by replacing the probability density p(x, t) in the diffusive term by an effective density F[p(x, t)]. The actual form of F[p] determines both the stationary and the non-stationary behaviour of the process. The non-stationary behaviour in the presence of no external potential, i.e., the solution of (18), can be classified according to the spread of the probability density by the anomalous diffusion scaling exponent γ, defined by eq. (7). This exponent γ provides a measure of deviation from ordinary diffusion, characterized by γ = 1/2.

Another macroscopic approach of modelling stochastic processes is through the construction of generalized entropy functionals S which are maximized by the stationary state of the processes. As it has been already shown, a meaningful classification of generalized entropies over a discrete phase space indexed by i, , can be given by their scaling with phase space volume W, characterized asymptotically by the exponent c. Here we consider analogous continuous entropy functionals in the form of . Similarly to the exponent γ regarding the non-stationary regime, the phase space volume scaling exponent c of the entropy quantifies the deviation from uncorrelated, Markovian systems (characterized by c = 1) at the stationary regime.

The two approaches, one based on Fokker-Planck equations and the other on entropies, are consistent at the stationary regime if the stationary solution of the Fokker-Planck equation equals to the maximum entropy distribution according to the generalized entropy. In this paper we show that this consistency criterion implies that asymptotically, i.e., at p → 0, the anomalous diffusion scaling exponent γ and the phase space volume scaling exponent of the entropy c bijectively determine each other via the relation . Asymptotically, this result generalizes that of Tsallis and Bukman[34], now being valid for any generalized entropy functional satisfying some general asymptotic conditions. In addition, since the explicit solution of the corresponding Fokker-Planck equation might either not be available, or possibly have infinite variance, our derivation do not rely on the computation of any of these. Our results suggests that either of the asymptotic exponents γ and c is indeed providing a useful characterization of the systems themselves, and not just describing their behaviour in the stationary or non-stationary regime. Furthermore, the surprising versatility of the theoretical framework behind Tsallis statistics to model various aspects of strongly interacting systems might be explained by the fact that the family of Tsallis entropies, characterized by their deformation index q, provides an algebraically simple representative of each such asymptotic equivalence class.