Derivations of the Core Functions of the Maximum Entropy Theory of Ecology.

The Maximum Entropy Theory of Ecology (METE), is a theoretical framework of macroecology that makes a variety of realistic ecological predictions about how species richness, abundance of species, metabolic rate distributions, and spatial aggregation of species interrelate in a given region. In the METE framework, "ecological state variables" (representing total area, total species richness, total abundance, and total metabolic energy) describe macroecological properties of an ecosystem. METE incorporates these state variables into constraints on underlying probability distributions. The method of Lagrange multipliers and maximization of information entropy (MaxEnt) lead to predicted functional forms of distributions of interest. We demonstrate how information entropy is maximized for the general case of a distribution, which has empirical information that provides constraints on the overall predictions. We then show how METE's two core functions are derived. These functions, called the "Spatial Structure Function" and the "Ecosystem Structure Function" are the core pieces of the theory, from which all the predictions of METE follow (including the Species Area Relationship, the Species Abundance Distribution, and various metabolic distributions). Primarily, we consider the discrete distributions predicted by METE. We also explore the parameter space defined by the METE's state variables and Lagrange multipliers. We aim to provide a comprehensive resource for ecologists who want to understand the derivations and assumptions of the basic mathematical structure of METE.

1. The Maximum Entropy Theory of Ecology

Many of the central questions of macroecology ask how patterns of species richness, abundance, and body size arise from ecosystems, how these patterns scale over increasing area, and how they interrelate [1]. Many macroecological distributions that quantify aspects of community structure, such as the Species–Area Relationship, the Species Abundance Distribution [2], size–density relationships [3,4,5], and the allometric scaling of metabolic rates of biological organisms within a community [6,7,8] have been studied independently, revealing general properties that may be universal across ecosystems. The Maximum Entropy Theory of Ecology (METE) [9,10,11], is a theoretical framework of macroecology that makes a variety of realistic ecological predictions about the diversity and structure of ecological communities [12,13,14,15,16,17,18,19,20]. These predictions relate species richness and abundance to metabolic rate distributions and spatial aggregation of species in a given region. Because METE makes a set of interrelated predictions about community structure, it has the potential to unify disparate parts of macroecology into a single mathematical framework.

The underlying mathematics of METE relies on a method termed “MaxEnt”: the maximization of information entropy. MaxEnt uses the method of Lagrange multipliers to find probability distributions that underlie statistical phenomena. For METE, the MaxEnt method is applied to problems involving measurable “ecological state variables” that describe macroecological properties of an ecosystem (Section 3.1). METE is a “top-down” theory that makes no assumptions regarding the particular details of, or mechanistic interactions between, the species being studied. This is different from other “bottom-up” theories involving the combinatorics of micro-states that result in Boltzmann distributions [21,22,23]. As such, its focus is the maximization of the Shannon information entropy [24,25].

In this paper, we will first demonstrate how information entropy is maximized for the general case of a distribution, which has empirical information that provides constraints on the overall predictions. We then introduce the ecological state variables , , , and , representing total area, total number of species, total abundance, and total metabolic energy of an ecological system, and use them with the method of information entropy maximization to show how METE’s two core functions are derived. These functions, called the “Spatial Structure Function” and the “Ecosystem Structure Function” are the core pieces of the theory, from which all the predictions of METE follow (including the Species–Area Relationship, the Species Abundance Distribution, and various metabolic distributions). Primarily, we consider the discrete distributions predicted by METE. These derivations are not provided in their entirety in Harte (2011) [10], but are the derivations that will produce the core distributions of the Spatial Structure Function and the Ecosystem Structure Function as presented in that work.

In the process of constructing the core structure functions, we derive the Lagrange multipliers that arise from the MaxEnt process, and characterize the ecosystems modeled by METE. We investigate the parameter space of these Lagrange multipliers, and evaluate some of the simplifying assumptions that have been used previously to estimate the Lagrange multiplier values.

We aim to provide a comprehensive resource for ecologists who want to understand the derivations and assumptions of the basic structure of METE. We hope that by providing explicit derivations of METE, we will encourage other ecologists to modify the framework, apply it to their own systems, and make progress in this valuable area of research.

2. Information Entropy Maximization: A Primer

In this section we present the equations that are necessary for information entropy maximization. We then use these equations to derive the form of the probability distribution resulting from the simplest case of a discrete, one-dimensional distribution. Although this derivation will not be new for regular readers of Entropy, we include it for ecologists who are interested in this theoretical framework, because this chain of logic will be applied to the constraints that characterize METE in subsequent sections.

2.1. Writing Down the Constraints

As observed by Haegeman and Etienne (2010) [26], probability distributions with higher information entropy encode less information. Therefore, a probability distribution that corresponds to empirical data without imputing any additional information will maximize information entropy. This is also true of a probability distribution that conforms to a constraint (as, for example, a constraint on the value of the mean) without making additional assumptions or adding other information. In this sense, maximum information entropy methods give the most impartial estimate of the shape of the underlying probability distribution for an observable. MaxEnt also gives the least biased estimators of the moments of a distribution (which include the range, mean, and variance) [25], meaning that there is no difference, for example, between the estimated mean and the empirical mean of a distribution. This feature of MaxEnt is by design, such that the moments of a probability distribution are constrained by the empirical values of those moments.

Here we present the primary equations that regularly occur in information entropy maximization. The general expression for K constraints on the mean values of the variables , where n follows the distribution , is expressed as:

An additional constraint provides for the normalization of the probability distributions, and is expressed as:

The procedure of maximizing entropy results in a particular form of the probability distribution and partition function, proven by Jaynes [25] to result in the least-biased probability that satisfies the constraint equation of Equation (1) and the normalization constraint of Equation (2), where Z is the partition function that serves to normalize the probability distribution, and is expressed as,

Generally, when one wants to use the tools of MaxEnt, one will need to have data from which constraints on the distributions can be inferred (such as average values). Then a process of algebra and numerical methods will begin during which one solves for the Lagrange multipliers . Once the Lagrange multipliers have been determined, then the form of the probability distributions can be inferred (or graphed). For examples using Equations (1)–(4), see Appendix A.1. Equations (3) and (4) are written here for reference, as we will use them repeatedly. In the next section we derive Equations (3) and (4) by maximizing the Shannon information entropy.

2.2. The Method of Lagrange Multipliers and Optimization

Here we derive the generic probability distribution for the specific scenario of a discrete variable with one constraint (beyond normalization). This will serve as a simpler example for deriving the core distributions, or structure functions of METE.

What does it mean for us to “maximize Shannon information entropy”? While the explicit form of the probability distribution, , is unknown, we have information about some of its properties that will serve to constrain its functional form. However, there may exist more than one function that satisfies our constraints, so we must choose between them. By maximizing Shannon information entropy we are maximizing the uncertainty inherent in our function , and choosing the unique form of that represents the least amount of bias in regard to our measurements. The expression for Shannon information entropy [24] is:

We know that no matter what the form of the probability distribution is, it should be properly normalized (that is, the sum of the probabilities of all possible outcomes will equal one). This gives us our normalization constraint, which for a single, discrete variable takes the form of

Finally, we will likely have some information about an aggregated measurement of the variable in our system. This information constitutes our additional constraint, or constraints. Letting represent a measured value dependent on n, and assuming that the aggregated measurement we have is the mean value of , then we can express our knowledge regarding this measurement of an observable quantity that represents some aggregated or average value using the mathematical definition of an average.

Thus, the definition itself constitutes our constraint.

To maximize Shannon information entropy subject to our constraints, we employ the tools of variational calculus and the method of undetermined Lagrange multipliers. A concrete example of this is given in Appendix A.1. We begin by constructing the function , which is an expression that incorporates the measure of Shannon information entropy and the additional constraints.

The constraints are written in such a way so that each constraint is independent of the other, and each term inside the square brackets is zero. In this way, we can incorporate multiple constraints without changing the overall value of . When we perform the maximization step to find local optima, the presence of the constraints will change the subsequent form of . That is, maximizing will subsequently maximize the uncertainty represented by the Shannon information entropy, H, subject to the constraints. Thus we seek to solve for the functional form of that results from variations in , due to variations in . Mathematically this is expressed as . Practically, this requires evaluating derivatives of each term in with respect to , and setting the resulting equation equal to zero,

Note that we can drop the summations at this point. Solving for yields, where . Imposing our normalization constraint , we have

Since k is independent of n, we can factor it out of the summation and solve for it to find,

The expression comes up so often that it is given its own variable representation Z, the partition function, which will eventually just turn out to be a real-valued number,

Thus, we can express the probability as,

To find the explicit form of , one needs further information in the form of data. Having data, it is then possible to use the constraint equations to numerically solve for the undetermined Lagrange multipliers, and thus identify the form of the probability distribution . This relationship between data, the Lagrange multipliers, and the resulting form of the probability distribution is revisited for normal and log-normal distributions in Appendix A.3 and Appendix A.4, and is fundamental to the Maximum Entropy Theory of Ecology.

3. The Structure of METE

In this section, we introduce and discuss the ecological state variables common to macroecology, as used by METE. We apply the MaxEnt method to these ecological state variables to derive the core distributions of METE, namely the Spatial Structure Function, and the Ecosystem Structure Function [10,11].

3.1. A State Variable Theory

Much of macroecology is concerned with detecting patterns in ecosystems, either at large scale, or as an emergent property of scaling over larger aggregates of individuals, species, area, or time [27]. To detect these patterns, we must work with variables that are sufficiently coarse that they capture average conditions of some larger phenomena that is being modeled, or sufficiently aggregated that they apply across systems and capture certain generalities. For example, we may examine average abundance of all individuals in an area. The “average” may refer to an average of repeated measurements in different plots or subplots, for example, which can smooth out heterogeneity and model average conditions on a landscape. The “aggregated” nature of abundance refers to something slightly different, in that abundance measured will be a result of multiple processes, such as birth, death, emigration, and immigration [28]. Abundance therefore represents an “aggregate” metric of all of these processes. Similarly, “species” as an observable may have an average value when measured in many similar sized plots, but is itself an aggregate measure of biodiversity that ignores (or “coarse grains”) genetic variation among individuals, and processes such as hybridization. These variables are easily measured and compared across ecological systems, and have an overall generality and transferability that make them relevant to ecological studies at large scales.

Within METE, variables representing total area, total number of species, total abundance, and total metabolic energy of an ecological system are central to the definitions of the core probability distributions. These ecological state variables are represented as , , , and , respectively. They are static (not time-dependent), and can be used to model macroecological distributions of interest, such as the Species–Area Relationship, the Species Abundance Distribution, and various metabolic rate distributions, both within a species and across an entire community. This so-called “ASNE” version of METE, where “ASNE" represents the four state variables above, has been the subject of the most study [10], but other constraints featuring additional state variables are possible. In one case, METE has been extended to include higher taxonomic constraints [29].

It is worth clarifying that although “ecological state variables” may call to mind the true state variables of thermodynamics, the use of this term is merely an analogy to thermodynamics concepts [30], but differs in underlying justification and interpretation. Ecological state variables capture information that defines the state of the system at a given point in time and space, but cannot be interpreted as true macroscopic variables for many reasons, including the less-than-ideal behavior of the variables themselves. For example, , the total number of species in an area, neither adds nor averages when considering larger and larger areas. It is therefore neither truly an intensive nor an extensive variable. “Species” as a defined concept only applies to a fraction of real organisms [31], and is used in macroecology as an aggregating variable, either sorting organisms by their actual species designation, or as a stand-in for phylogenetic relatedness or functional traits [32]. Because of these and other issues, ecological state variables are seen to embody information at an aggregate level, and are not interpreted as direct translation from countable microstates to macrostates, though this has been the subject of some debate [30,33,34,35]. This raises a related issue, which is the use of the Shannon form of entropy in METE and the information-entropy interpretation.

Work with applications of entropy maximization to macroecological patterns began in 2006 with Shipley, Vile, and Garnier’s use of Shannon information entropy in the problem of constructing Species Abundance Distributions [33]. This advance sparked a small explosion of the use of entropy maximization in macroecology, including applications to new problems (variously using frequentist and Bayesian interpretations of MaxEnt), a search for generalities among ecosystems, and among the approaches themselves [9,13,14,17,36,37,38,39,40,41,42,43,44,45]. It also sparked a debate about the use of information entropy in macroecology, in which Shipley [30,33] and Haegeman and Loreau [34,35], discuss the use and justification of Shannon information entropy in ecology. In this exchange, Shipley maintains that ecological systems are more complex in their constraints than systems of particles, and that we may never know (or be able to measure) all of the relevant constraints. He therefore suggests focusing on the most important constraints, and using information entropy to evaluate the constraints and distributions. Although Haegeman and Loreau make a clear argument that the frequentist interpretation of MaxEnt should be complementary to the information entropy interpretation, Shipley disagreed, arguing incorporating all relevant constraints may not be possible for ecological systems, but would be needed for the constraint-based information entropy and state-counting, combinatorics interpretations of entropy to be equivalent. A later exchange between Favretti [21,23] and Harte [22] revisits this topic, and more productive work may be done to address the issues raised in all of these exchanges.

METE follows the logic of Shipley’s arguments about the use of important constraints in complex ecosystems, and uses the information entropy approach. In the following sections, we will demonstrate how METE’s ecological state variables and their ratios are used with the MaxEnt procedure to constrain patterns of individuals over area for the Spatial Structure Function, and patterns of metabolic requirement (or body size) across individuals and abundances per species through the Ecosystem Structure Function.

3.2. The Spatial Structure Function

This distribution goes by multiple names among practitioners, including the “Pi distribution” (informally), the “Species-level Spatial Abundance Distribution” (SSAD) [14,46], and the Spatial Structure Function (SSF) [47], which is how we will refer to it here. The SSF is a by-species prediction of the clustering of individuals over space, and is defined as the “probability that n individuals of a species are found in a cell of area A if it has individuals in ” [10]. We derive this distribution before the Ecosystem Structure Function for pedagogical reasons, as it is a one-dimensional distribution for a discrete variable. This variable is the the abundance of a single species, n, at a given scale, A, where A is a smaller area within the total area under consideration, . The variables that are used to constrain the system are A, , and the abundance of a single species at the total spatial scale, . The derivation of the Spatial Structure Function will follow exactly the general approach provided in the previous section. We represent this function as . To ensure that the Spatial Structure Function is properly normalized, we express our normalization constraint as, where we have dropped the conditional variables , and for notational convenience. Note also that the lower limit on the summation is set to . This is because n represents the per-species abundance, and it is possible for a species that has individuals to have zero abundance in an area A.

Our additional constraint comes from the simultaneous definition and measurement of the average value of the per-species abundance . From measurement, . From definition, . Combining these two expressions gives us our other constraint,

Now, as we want to maximize uncertainty, or Shannon information entropy, related to the Spatial Structure Function subject to the above two constraints, we construct the function in the form,

From here we maximize by evaluating the expression . This results in,

From here we can solve for to arrive at, where . Imposing our normalization constraint, we can write,

Because k is independent of n, we can factor it out of the summation and rewrite as,

Conventionally this normalization constant is defined as and denoted as the partition function, where,

This brings us to the following compact expression for the Spatial Structure Function,

In order to actually calculate, or graph, the Spatial Structure Function for a given set of values and A, we must first calculate the Lagrange multiplier using our constraint equation relating the measured average per-species abundance to the definition of the average per-species abundance . This time, substituting our known expression for , we have,

Recalling that Z is independent of n, it can be pulled out of the summation. Furthermore, substituting our definition of Z, but using m as a dummy index instead of n to avoid mixing up our indices, we can write,

In principle, the above expression allows one to solve for the Lagrange multiplier . However, analytical solutions for are intractable, thus one must resort to numerical methods. We graph the parameter space of the state variables and in Figure 1.

Figure 1

The relationship between the Maximum Entropy Theory of Ecology (METE)’s Lagrange multipliers , , and , and the ecological state variables in the mathematical constraints that produce them. Values for each were generated with the software package meteR [47] in the statistical computing language R [48], and a surface was interpolated to aid in visualization. In panel (A), we see the greater influence of log() than on the overall value of the Lagrange multiplier , and a compression of values at low . In panel (B), we can see a near-linear relationship on the log-log scale between and log(), while does not affect its value as greatly over this range of values. In panel (C), we see a highly non-linear relationship between , the state variable , and the smaller area under consideration, A.

3.3. The Ecosystem Structure Function

The Ecosystem Structure Function (ESF) is the second of METE’s core distributions. Unlike the SSF, it does not have a simple definition, but can be described as a kind of “container function” that describes the probability space, , of how abundances are assigned to species, n, and how metabolic energy, , is partitioned over individuals in a community. As described by Bertram (2015) [49], is a joint probability distribution, with by definition being “the probability that a randomly selected species has abundance n, and that a randomly selected individual from any species with abundance n has metabolic requirement in the interval ()” (page 55). This definition differs from and corrects the one given in Harte (2011) [10] (also used in [45]), and is consistent with a later correction to the Harte (2011) definition [21] (accepted by Harte [22]). The ESF is the distribution from which the Species Abundance Distribution, Species–Area Relationship, and metabolic rate distributions [14,18] can eventually be derived. Constrained by empirical values measured from real systems, the ESF produces actual predictions of these probability distributions.

As the ESF , depends on one discrete variable n, and one continuous variable , we will this time need to integrate over in addition to summing over n. Thus, our normalization constraint now takes the form of,

Note that in the above expression we have specified the lower limit of metabolism as, . In so doing we have defined metabolism as a dimensionless quantity. This decision has several consequences. For comparison against data, a researcher must standardize their metabolic measurements to conform with this definition. That is, they divide all metabolic measurements by the smallest measured value [10,14,19]. A second consequence of this decision is that it preempts the need to incorporate reference distributions when expressing Shannon information entropy for a continuous variable [10,25].

Our additional constraints are aggregated measures of variables n and , that is , and . The measures themselves are the ratios and , that is , and . This gives us the pair of constraints,

Now, the function that we will want to maximize takes the form of

As before, we maximize by evaluating . This results in,

Solving for gives, where . Using our normalization condition to define Z we have,

This allows us to express the full form of the Ecosystem Structure Function as, where we have replaced n with m and and in the denominator to ensure there is no confusion over which variables belong in the numerator or denominator.

At this point we can simplify the expression for Z by performing the integral over . Factoring out the term in Z independent of we have,

After integrating, we have and with further simplification, where and .

From here we can examine the constraint Equations (27) and (28). In particular, now that we have an explicit form for the structure function, we can perform the integrals in the constraint equations to fix the values of the unknown Lagrange multipliers in terms of the measured quantities and . Upon substitution of our expression for into Equation (27) we have,

Factoring from the integral the term independent of ,

Upon integration, and with further simplification, where and .

Turning our attention now to constraint Equation (28), upon substitution of we have,

Factoring from the integral the term independent of ,

After integrating by parts we have, and with further simplification, where and . Substituting in our expression for Z, we can write the integrated versions of the constraint equations in full as,

The values of and are often difficult to calculate by conventional means, so some approximations were used in Harte (2011). With the use of the software package meteR [47,48], which implements a number of METE’s predictions and tests them against empirical data sets, these approximations are no longer necessary. However, we consider them in more depth in Appendix B.

4. Relationships between State Variables and Lagrange Multipliers

By using the MaxEnt approach with the METE ecological state variables, we derive three Lagrange multipliers: associated with the Spatial Structure Function, and and , associated with the Ecosystem Structure Function. The METE Lagrange multipliers represent all the possible relationships between , , and , and METE’s predicted relationships of n, A, and . We graph these, along with the ecological state variables in their respective constraints, in Figure 1.

Examining in panel (A), we see the greater influence of log() than on the overall value of the Lagrange multiplier , and a compression of values at low . This may correspond to METE’s own stated limitations, and its requirement that [10]. It also implies that may be quite sensitive to the total area being sampled in empirical studies, and that this factor should be explicitly controlled for when comparing diversity patterns, such as Species Abundance Distributions, across plots. In panel (B), we can see a near-linear relationship on the log-log scale between and log(), while does not affect its value as greatly over this range of values. It is apparent, then, that the METE distributions that rely on , such as the species-level energy distributions [14,18] are driven by body size, and may be largely insensitive to species richness, which in turn implies that the metabolic predictions of METE should hold equally well in very different ecosystems that share size characteristics, such as boreal forests and tropical ones.

In panel (C), is clearly non-linear in both the state variable , and the smaller area under consideration, A. The graph of shows that very different values of may be obtained by varying the ratio of A to slightly, and this may in turn suggest that it does not have the properties we would desire in a metric describing clustering. It has been demonstrated that the SSF does not always produce reliable predictions for clustering of individuals of a species within a a given area [16,46], and this area of METE could be extended and modified in future work. Further investigations of the relationships of and with data from varied ecosystems may also allow us to investigate patterns of diversity, abundance, body size, and the relationships between these macroecological variables in new ways. These kinds of parameter space representations are also useful in generating hypotheses about changing ecosystems, and what new distributions are expected as one or more state variable changes.

In Figure 2, we graph the parameter space that is defined by the ESF through the Lagrange multipliers and . In this graph, the boundaries of the defined parameter space become interesting. High values of always correspond to one or more “singleton species”, or species with a single individual, where = . This can only happen when a single individual is measured in order to estimate the values of , , , and (that is, small numbers of measurements), or in cases where there are the most species possible given the number of individuals present (extreme diversity). These cases therefore represent theoretical limits of possible outcomes of measurement. We expect that most real systems will have many measurements of species with more than one individual, and will fall into the range of low values. The behavior of the lower values of the graph may therefore be worth investigating further in relation to spatial scale. Comparisons to empirical data sets may yield new, emergent patterns in this parameter space.

Figure 2

The parameter space of ecosystems as defined by the METE Lagrange multipliers , corresponding to the constraint on , and , corresponding to the constraint on . The highest values of for any value of correspond to values of = 1 (shown in purple), or situations where there is only one individual per species (small numbers of measurements or high diversity). Most real ecosystems and empirical systems with more than a few individuals are expected to fall closer to the low values for any given value (shown in green).

5. Summary

In this paper, we provide the explicit chain of logic that produces the core structure functions of Maximum Entropy Theory of Ecology, or METE. These derivations fill a gap in the ecological literature for researchers who would like to see the explicit construction and assumptions of the central equations from which the predictions of METE follow. In presenting this theory, we take a different pedagogical approach than is employed in Harte (2011) [10]. Namely, we provide a general case of a discrete MaxEnt problem in one dimension. We then give worked examples of constraints on moments beyond just the mean of a distribution (in Appendix A). From there, we first construct the simpler Spatial Structure Function, which is a discrete probability distribution, and then work through the logic of the Ecosystem Structure Function, which contains more constraints, and has a combination of discrete and continuous variables.

We do not cover the applications of METE to testing theory against empirical data, which has been done extensively elsewhere [9,10,11,12,13,14,15,16,17,18,19,20], however, we explicitly show the MaxEnt process and how it is applied to the ecological state variables of METE to produce the core structure functions from which all the predictions of METE derive.

Finally, we discuss the three Lagrange multipliers that result from applying the MaxEnt procedure to METE’s ecological state variables, and how these Lagrange multipliers characterize the system being studied. We investigate (in Appendix B) some of the simplifying approximations that were previously used to evaluate the Lagrange multipliers and assess their realism.

The equations presented here should make it easier for other researchers to make advances in MaxEnt-based macroecology, and METE in particular, either by investigating new state variables, or new functional forms of the mathematical constraints, and alternate forms of the entropy function. The Lagrange multipliers, graphed for the first time here, form a parameter space that may be useful to further studies involving comparisons between ecosystems and across scales.