Associating an Entropy with Power-Law Frequency of Events.

Events occurring with a frequency described by power laws, within a certain range of validity, are very common in natural systems. In many of them, it is possible to associate an energy spectrum and one can show that these types of phenomena are intimately related to Tsallis entropy S q . The relevant parameters become: (i) The entropic index q, which is directly related to the power of the corresponding distribution; (ii) The ground-state energy ε 0 , in terms of which all energies are rescaled. One verifies that the corresponding processes take place at a temperature T q with k T q ∝ ε 0 (i.e., isothermal processes, for a given q), in analogy with those in the class of self-organized criticality, which are known to occur at fixed temperatures. Typical examples are analyzed, like earthquakes, avalanches, and forest fires, and in some of them, the entropic index q and value of T q are estimated. The knowledge of the associated entropic form opens the possibility for a deeper understanding of such phenomena, particularly by using information theory and optimization procedures.

1. Introduction

Power laws are ubiquitous in many areas of knowledge, emerging in economics, natural and social sciences, among others [1]. In the latest years, a particular interest has been given to frequency of events, which very often follow power laws: (i) In humanities, the Zipf’s law states that the relative word frequency in a given text is inversely proportional to its rank (defined as its position in a rank of decreasing frequency); (ii) In natural sciences, the frequency of earthquakes with a magnitude larger than a certain value m, plotted versus m, leads to the Gutenberg–Richter law [2]; furthermore, the frequency of avalanches, as well as of forest fires, of a given size l, plotted versus l, yield power laws [1]. Simple dynamic scale-free models, without tuning of a control parameter, but sharing many features of the critical point in a standard phase transition, like long-range correlations, have been introduced to approach theoretically the types of phenomena in examples (ii). For these reasons, the term self-organized criticality (SOC) [3] was coined, considered as the main characteristic exhibited by these models; since then, a vast literature appeared in this area (for reviews, see References [4,5,6,7]). Although stationary states may occur in SOC models, they are essentially characterized by out-of-equilibrium states, and in many cases jumps between different states occur due to energy changes; consequently, equilibrium thermodynamics does not apply to these models. Moreover, one of the most curious aspects concerns the fact that a critical state is approached without a temperature-like control parameter, and one of the most relevant questions concerns which real systems are well-described by SOC models, and under what conditions SOC applies [7].

Recently, a wide variety of entropic forms have been considered in the literature, either in the context of information theory, or for approaching real phenomena (see, e.g., References [8,9,10,11,12,13,14,15,16,17,18]). Many of these proposals recover the well-known Boltzmann–Gibbs entropy [19,20] as particular limits, and are usually known as generalized entropic forms. In the present work we show a connection between phenomena following power-law frequency of events and Tsallis entropy [9,10,11]. For this purpose, we develop a framework that could be relevant for some of the phenomena described in the previous paragraph. In this proposal we assume the existence of equilibrium (or long-living metastable) states, characterized by an energy spectrum , which represent notorious differences with respect to the SOC models. The main motivation is that in many cases it is possible to define an energy-like variable, related in some way to one of the relevant parameters of the system, e.g., the magnitude of an earthquake, or the size of an avalanche, should be associated to some quantity of energy released. Since these parameters obey power laws, one expects that their corresponding energies should be also power-like distributed, leading an energy probability distribution , where , restricted to , for reasons that will become clear later.

Then, from the distribution we follow previous works, where a procedure for calculating fundamental quantities (like the partition function) was developed, by combining information theory and a key thermodynamical relation (see, e.g., References [21,22,23,24]). More precisely, we calculate the internal, or more generally, average energy U and define a general entropic form satisfying basic requirements [19,20], like being a functional that depends only on the set of probabilities. Furthermore, imposing the fundamental relation of thermodynamics,

We obtain the associated entropy and verify that the temperature should be constant, for consistency. Curiously, the distribution turns up to be temperature-independent, and consequently, all average values calculated from this probability distribution become independent of the temperature. Hence, similarly to what happens in SOC models, in the present approach the temperature does not play a crucial role for these types of phenomena.

In the next section we review some results of References [21,22,23,24], and especially how to combine general concepts of information theory with the fundamental relation of Equation (1), with the purpose of deriving an equation for obtaining the entropic form from a given energy spectrum. In Section 3 we discuss energy power-law distributions, and show a peculiar behavior, namely, that through the normalization procedure its dependence on the temperature disappears. Consequently, all quantities derived from these distributions, like average values, do not depend on the temperature. In Section 4 we analyze data of events within the present framework, by associating the corresponding power-law distributions with the energy distributions discussed in Section 3. Finally, in Section 5 we present our main conclusions.

2. Combining Information Theory and Thermodynamics

Herein we review some basic results of References [21,22,23,24], which were derived by considering a nondegenerate energy spectrum . Hence, a discrete index i will identify uniquely a state with an energy , occurring with a probability , in such a way that the internal energy is defined as

Moreover, let be an arbitrary concave smooth function of ; we assume that the entropic functional may be written in the form [19,20] where k is a positive constant with entropy dimensions.

Let us now consider a small change in the level populations (which may occur, e.g., due to an infinitesimal exchange of heat); then, the probabilities will vary according to, with the last condition resulting from normalization (). This procedure will in turn generate infinitesimal changes in the entropy and internal energy, and we impose the fundamental relation of Equation (1). One obtains (up to first order in ) [21], where the prime indicates a derivative with respect to . As shown in Reference [21], Equations (4) and (5) lead to just one expression for the and further, that all should be equal. The resulting value is found through the normalization condition on the ensuing probability distribution ( is, in fact, related to the partition function), to be determined by the relation,

From now on we will consider, for simplicity, a continuous energy spectrum represented by an energy probability distribution , defined in a given range of energies between a minimum value , and a maximum . Although the events to be studied herein are expressed in terms of discrete sets of data, we will associate to them continuous distributions, which result from fittings of these data in such a range, as will be defined appropriately in the following sections. In the next section we define the probability distribution of interest for the present work, and calculate relevant quantities; moreover, we consider the continuous form of Equation (6) to obtain the associated entropic form.

3. Power-Law Distributions and Associated Entropy

Power-law distributions frequently appear to be valid for certain ranges of its parameters, in variegated empirical settings pertaining to diverse disciplines [1,2,3,4,5,6,7]. We enlarge the scope of our methodology by considering systems for which a strict underlying thermodynamics does not exist, the inverse-temperature being just a measure of the probability-distribution’s ”spread”. Let us then consider an energy spectrum following a power-law distribution, defined in a given range of energies between a minimum value , and a maximum , with a non-negative ground-state energy, . The normalization condition, yields leading to which does not depend upon .

One should notice that, in order to obtain an appropriate power-law decay from the distribution above one should have , restricted to . Furthermore, presents dimensions , as required by Equation (8).

One curious aspect of in Equation (10) concerns its non-dependence on the parameter , which, although introduced in Equation (7), it cancelled by imposing normalization; later on, it will be shown that the parameter takes a constant value, for consistency. Consequently, all properties derived from the probability distribution of Equation (10) will not allow variations on the temperature; as an example, one has the average energy,

As mentioned before, the present approach holds for any ; the particular limit of the internal energy above may be obtained through the l’Hopital rule,

In order to deal appropriately with the continuous form of Equation (6), we define the dimensionless quantities, so that Equation (10) may be expressed as whereas the normalization condition becomes

The continuous form of Equation (6) becomes and we are using the fact that is of monotonic decreasing nature, so that it can be inverted, yielding a function . Notice that Equation (16) is a first-order differential equation for , in fact a Bernoulli equation of zeroth-order; its solution reads,

Now, one can invert Equation (14), so that , and substitute this result in Equation (17), leading to

Using the conditions of Equation (3), i.e., , for and , one obtains that and showing that is indeed related to the normalization of the probability distribution. Hence, Equation (18) becomes leading to

By recourse to the exact mapping detailed below, the expression above may be identified with Tsallis entropy [9,10,11], through where q represents the usual entropic index. This is of practical utility because we have now at our disposal the large set of useful recipes developed since 1988 with regards to Tsallis’ measure. Now, manipulating Equations (14) and (23), we obtain showing that the parameter should assume a real constant value, for a given value of . Hence, defining a fixed pseudo-temperature , such that the spread , one finds

In this way, the probability distribution of Equation (10), which is indeed a power-law, may be expressed in terms of the entropic index q, being defined for only; notice that this restriction is equivalent to (cf. Equation (23)).

For several of the examples to be considered below, the associated energy spectra will be characterized by , so the Equation (25) may be expanded in a power series, e.g., whereas for the probability distribution one has the approximate expression which is not a q-exponential.

The expansions of Equations (27) and (28) show that the maximum energy value only appears in higher-order corrections of and . In such cases, the most relevant parameters in Equation (10) become the exponent [directly related to q through Equation (23)] and the ground-state energy .

One should focus attention upon the curious result we have obtained in this effort. We were able to relate with Tsallis entropy the power-law distribution Equation (7) (not the usual q-exponential distribution). In fact, the equilibrium distribution that arises out of the extremization procedure for a given entropic form depends directly on the constraints imposed and the choices made regarding the corresponding Lagrange multipliers [10]. As shown in Reference [24], the distribution Equation (7) may be obtained from an extremization procedure effected on Tsallis entropy in Equation (22), by considering the usual constraints of probability normalization (associated Lagrange multiplier ) and internal energy definition in Equation (2) (corresponding Lagrange multiplier ), by choosing appropriately the first Lagrange multiplier, i.e.,

In the following section we will analyze examples of real systems governed by a power-law frequency of events.

4. Typical Examples in Natural Systems: From Data of Events to Energy Spectrum

Next, we describe some examples, chosen from the literature, of power-law distributions found in natural systems. In order to associate these examples with the theoretical approach of the previous sections, we will assume that: (i) The relevant variable of each distribution may be related in some way to the energy ; (ii) The fittings describing each class of phenomena may be associated with the continuous probability distribution of Equation (10), defined in the range between its minimum and maximum values ( and , respectively). We discuss separately two types of phenomena: (i) Systems presenting energy power-law distributions that can be directly related to the distribution of Equation (10). In such cases, we calculate, from the corresponding data, important quantities like the entropic index q, the dimensionless ratio , and the fixed value of the temperature ; (ii) Systems presenting power-law distributions , depending on a parameter x that can be related to the energy through some invertible monotonic function. For these cases, we propose a procedure for calculating the quantities of interest.

4.1. Systems Exhibiting Energy Power-Law Distributions

Certainly, one of the most paradigmatic power-law distributions is the Gutenberg–Richter law, which measures the frequency of earthquakes with a magnitude larger than a certain value m [2]. The magnitude m may be related to the seismic energy (or energy released) E [25], so that the Gutenberg–Richter law is sometimes expressed in a form similar to Equation (10),

In fact, as pointed out in Reference [26], the distribution above was proposed previously by Wadati (1932) in a paper written in japonese [27]. By analyzing earthquakes around the Tokyo station, Wadati obtained two different estimates for the exponent , respectively and , under different assumptions for the distributions of hypocenters. One should notice that the first estimate is very close to , which is nowadays generally accepted for the index of the power-law distribution of seismic energies [26,28]. For earthquakes, one can assume that the seismic energy E can be related to the energy in a simple way, e.g., at most, apart from a proportionality constant, , so that Equation (30) can be associated with the probability distribution of Equation (10). Under this assumption one has , and using Equation (23) one obtains the entropic index for earthquakes.

Recently, the possibility of investigating seismic phenomena by means of laboratory experiments has gained a big motivation after the identification of deep associations between earthquakes and the fracture of materials [29,30]. As examples, one may mention experiments of compression on porous glasses [31,32], as well as on small wood samples [33]. This connection is based on the crackling noise idea, where systems under slow perturbations may respond through discrete events covering a wide variety of amplitudes. By recording the amplitudes of these cracking noises, one can compute the associated energies, which may be normalized conveniently in such a way to produce energy probability distributions. Inspired by those, further experiments have been carried out by considering different apparatus, e.g., without compression, through the analysis of the acoustic emission in a variety of systems, like crumpled plastic sheets [34], or ethanol-dampened charcoal [35].

The two examples presented in Figure 1 follow these procedures, where the energy probability distribution is represented versus E for two distinct experiments. Results from the cracking noise produced by charcoal samples, when dampened with ethanol, are presented in Figure 1a; through their experiments, the authors have shown that the most fundamental seismic laws ruling earthquakes could be reproduced [35]. In an analogous way, avalanches were observed recently by means of acoustic emission in small wood samples under compression; these avalanches show results very similar to earthquakes and crackling noise in rocks and laboratory tests on brittle materials [33]. The distributions of energies are shown in Figure 1b, where data from different experimental conditions, i.e., constant strain rate , constant stress rate , and distinct event rates (defined as the number of events in a time interval divided by the interval length), all fall in a universal probability distribution . Like done before for natural earthquakes, in both cases one can identify directly the energy liberated E with , i.e., , so that the probability distribution of Equation (10) can be related with the fitting distributions shown in Figure 1a,b. In this way, these examples correspond respectively, to and , representing smaller values when compared to generally accepted for earthquakes. From Equation (23) one obtains the entropic indexes (Figure 1a) and (Figure 1b). Moreover, in the plots of Figure 1 one has very small values for (typically, ), so that the expansions of Equations (27) and (28) are well approximated by their leading-order contributions. In particular, the dimensionless temperature of Equation (27) becomes , so that the two examples of Figure 1 can be associated with fixed values of the dimensionless temperature, (Figure 1a) and (Figure 1b). One notices that the estimates of q and are very close to one another in these two experiments.

Figure 1

Typical energy power-law distributions found in experiments. (a) Energy distribution versus E, obtained from the cracking noise produced by charcoal samples, when dampened with ethanol (from Reference [35]). (b) Energy distribution versus E, obtained from acoustic emission in small wood samples under compression. Data from different experimental conditions, i.e., constant strain rate , constant stress rate , and various event rates (defined as the number of events in a time interval divided by the interval length), all fall in a universal probability distribution (from Reference [33]). In both cases, the variable E is properly normalized and defined as a dimensionless quantity; within the present approach (cf. Equation (10)), these examples correspond to (case (a)) and (case (b)).

4.2. Systems Exhibiting General Power-Law Distributions: Identifying Relevant Variables with Energy

Let us now analyze systems characterized by a given parameter x and its associated power-law distribution ; contrary to the examples shown in Figure 1, the relation between x and does not follow straightforwardly; two typical examples in this class are shown in Figure 2. In Figure 2a the forest-fire frequency density per year is represented versus forest burned area . The straight line yields a frequency versus area power-law distribution with an exponent ; the data corresponds to Ontario, Canada, during the period 1976–1996 [36]. Results from experiments carried out on a NbTi (conventional superconductor) sample, at the Bean critical state, are exhibited in Figure 2b [37]. For hard superconductors, the Bean critical state corresponds to a marginal stable state, where the Lorentz force acting on each vortex equals the maximum pinning force. A sketch of the experimental arrangement is represented in the inset, where one has a tubular NbTi sample and the pickup coil. An external magnetic field enters the interior of the tube, inducing a voltage on the pickup coil; large variations of the voltage in the pickup coil are associated with avalanches. The corresponding probability density for measuring an avalanche of s vortices is represented versus s (cf. Figure 2b), for three different values of the magnetic field (the exponent of the power-law distribution is field-dependent); one notices that for the higher value of the magnetic field (7.55 kG), one gets avalanches up to 5000 vortices. In both examples shown in Figure 2, one expects the variable of the previous section to be an increasing function of the relevant variable, i.e., of the burned area (Figure 2a), as well as of the energy required for producing s vortices in a given avalanche (Figure 2b).

Figure 2

Typical power-law distributions found in natural systems. (a) Forest-fire frequency density per year is represented versus forest burned area ; the data corresponds to the period 1976–1996 in Ontario, Canada (from Reference [36]). (b) Probability density for measuring an avalanche of s vortices [] in a hard superconductor is represented versus s, for three different values of the magnetic field. The inset shows a sketch of the experimental arrangement, where one has a tubular NbTi sample and the pickup coil. Large variations of the voltage measured in the pickup coil are associated with avalanches (from Reference [37]).

In order to relate probability distributions associated to these types of events to the approach of the previous sections, let us consider a given set of discrete data , given by values , ordered in such a way that . Moreover, each quantity occurs with a frequency , following

Rescaling the set of variables by its minimum value , one gets a discrete set of dimensionless data , , each occurring with a probability [ () representing a set of decreasing probabilities], so that

Herein we will be interested in the kind of phenomena illustrated in Figure 2, which are well-fitted by continuous power-law distributions; furthermore, we define dimensionless quantities similarly to those of Equations (13) and (14), i.e., where now corresponds to the continuous representation of the discrete variables , whereas denotes a dimensionless probability distribution. Moreover, the normalization condition, requires

Accordingly, one can also calculate the average value,

One should notice the resemblance of the probability distribution of Equation (33) with the energy distribution of in Equation (14), as well as of the average value with the internal energy of Equation (11). Such similarities suggest that and should be directly related to one another; herein, we propose where represents an invertible and monotonically increasing function of y, such that . The normalization condition on both distributions and requires that which implies that and should be related through a power, i.e., , with being a positive real number. In this way, one obtains the relation between the two variables,

Therefore, the internal energy of Equation (11) may be written as which recovers the result of Equation (11) by using and imposing the relation

Hence, for systems exhibiting power-law distributions presenting a dependence on a general parameter x, being characterized by an exponent according to Equation (33), the entropic form of Equation (22) still applies. In order to identify the entropic index q, one should carry out the following procedure: (i) Obtain the exponent relating the energy to the relevant parameter x through Equation (39); (ii) The exponent is taken directly from the data, like those in Figure 2, e.g., in the case of forest fires (Figure 2a). Then, use Equation (41) to calculate the exponent of the corresponding energy distribution; (iii) Calculate the entropic index q by means of Equation (23). In many cases step (i) may become the most difficult task, since obtaining an energy distribution from a given set of data of natural systems may not be so obvious.

5. Conclusions

We have analyzed events that occur with a frequency following power laws, within a certain range of validity of their relevant parameters. These types of phenomena are very common in natural systems and are usually associated with self-organized criticality. In many of such cases it is possible to introduce an energy spectrum, defined in a given interval of energies between a minimum value , and a maximum , so that an internal energy may be calculated. Based on this, we have assumed the validity of the fundamental relation , and have calculated important quantities, like the associated entropic form and temperature. As a curious aspect, the power-law probability distribution is temperature-independent, in agreement with self-organized-criticality; however, we have shown that these phenomena occur at a constant temperature and follow Tsallis entropy , with an entropic index ; from the thermodynamical point of view, these phenomena could be identified as isothermal processes. In cases where , the relevant parameters within this procedure become the entropic index q, which is directly related to the power of the corresponding distribution, and the ground-state energy , in terms of which all energies are rescaled. In particular, the corresponding processes take place at a temperature with .

Typical examples were analyzed, like earthquakes, avalanches, and forest fires, and in some of them, the entropic index q and value of were estimated. Specially for earthquakes, we obtained and . It should be mentioned that an analysis of probability distributions of energy differences (returns) of data from the Northern California earthquake catalogue has led to q-Gaussian distributions with [38]. Although the power-law distributions considered herein are very different from the q-Gaussian distribution of Reference [38], both are associated in some way to Tsallis entropy ; curiously, our estimate for the entropic index q agrees, within the error bars, with the result of Reference [38] by considering the usual correspondence .

The main contribution of the present work concerns the association of events occurring with a frequency following power laws with the entropy , and that distinct types of events should be characterized by different values of q. Furthermore, the identification of an associated entropic form opens the possibility for a deeper understanding of such important natural phenomena, particularly by using information theory and optimization procedures.