An Analytical Framework for Runtime of a Class of Continuous Evolutionary Algorithms.

Although there have been many studies on the runtime of evolutionary algorithms in discrete optimization, relatively few theoretical results have been proposed on continuous optimization, such as evolutionary programming (EP). This paper proposes an analysis of the runtime of two EP algorithms based on Gaussian and Cauchy mutations, using an absorbing Markov chain. Given a constant variation, we calculate the runtime upper bound of special Gaussian mutation EP and Cauchy mutation EP. Our analysis reveals that the upper bounds are impacted by individual number, problem dimension number n, searching range, and the Lebesgue measure of the optimal neighborhood. Furthermore, we provide conditions whereby the average runtime of the considered EP can be no more than a polynomial of n. The condition is that the Lebesgue measure of the optimal neighborhood is larger than a combinatorial calculation of an exponential and the given polynomial of n.

1. Introduction

The running time to optimum is a key factor in determining the success of an evolutionary programming (EP) approach. Ideally, an implementation of an EP approach should run for a sufficient number of generations when the probability of achieving an optimum is greater than some desired value. However, few results on the running time of EP approaches can be found in the current literature.

As a technique of finite state machine, EP was first proposed for continuous optimization [1]. It has since been widely adopted as a powerful optimizing framework for solving continuous optimization problems [2-4]. More recently, EP research has mainly concentrated on the function of its parameters and its improvement by adaptive strategies. As a result, several EP variants have been proposed, distinguishing from each other mainly with different mutation schemes based on different probability distributions.

Arguably, the first EP algorithm that was widely considered successful was the one with Gauss mutation that has been termed as classical evolutionary programming (CEP) [2]. CEP has been intensively analyzed by Fogel [3, 4], Bäck and Schwefel [5], and Schwefel [5, 6]. Subsequently, Yao et al. proposed a fast evolutionary programming algorithm (FEP) with a new mutation scheme based on the Cauchy distribution [7]. Computational experiments showed that FEP is superior to CEP when tackling the optimization problems of multimodal and dispersed peak functions. Another EP variant [8] was later proposed by using the Lévy distribution based mutation, which we call Lévy evolutionary programming (LEP) in this paper. Empirical analyses show that, overall, LEP exceeds CEP and FEP when solving the benchmark problems of multimodal and highly dispersed peak functions.

CEP [2], FEP [7], and LEP [8] can be considered as classical evolutionary programming algorithms. Several modified types of EP have since been designed based on these three basic approaches [2, 7, 8].

The performances of EP approaches such as CEP, FEP, and LEP have oftentimes been verified experimentally rather than theoretically. The theoretical foundations of EP have been an open problem ever since it was first put forward [1, 2]. In particular, Bäck and Schwefel [5] and Schwefel [6] suggested the convergence analysis of EP algorithms as a research topic in their surveys of evolutionary algorithms. Fogel [2-4] presented an initial proof of EP convergence on the basis of the discrete solution space. Rudolph [9-11] then showed that CEP and FEP can converge with an arbitrary initialization; this result is more general since it applies to a continuous solution space.

Previous convergence studies only considered whether an EP algorithm is able to find an optimum within infinite iteration but did not mention the speed of convergence, that is, lacking running time analysis. To date, running time analyses have mainly focused on Boolean individual EAs like (1 + 1)EA [12], (N + N)EA [13], multiobjective EA [14-16], and so forth. Alternative theoretical measures for evaluating the running time of Boolean-individual EA approaches have also been proposed [17, 18]. Recently, the impact of particular components of EA on runtime has also been studied, on mutation, selection [19], and population size [18, 20, 21]. In addition to these studies on EAs solving Boolean functions, some results of runtime analysis have been obtained on some combinatorial optimization problems [22-24]. Lehre and Yao [25] completed the runtime analysis of the (1 + 1)EA on computing unique input output sequences. Zhou et al. presented a series of EA analysis results for discrete optimization like the minimum label spanning tree problem [26], the multiprocessor scheduling problem [27], and the maximum cut problem [28]. Other proposed studies are on the topics of tight bounds on the running time of EA and randomized search heuristic [29-31].

As summarized above, the majority of runtime analyses are limited to discrete search space; analyses for continuous search space require a more sophisticated modeling and remain relatively few, which is unsatisfactory from a theoretical point of view. Jägersküpper conducted a rigorous runtime analysis on (1 + 1)ES, (1 + λ)ES minimizing the Sphere function [32, 33]. Agapie et al. modeled the continuous EA as a renewal process under some strong assumption and analyzed the first hitting time of (1 + 1)EA on inclined plane and Sphere function [34, 35]. Chen et al. [36] proposed general drift conditions to estimate the upper bound of the first hitting time for EAs to find ϵ-approximation solutions. Inspired by the studies above, especially the estimating approach in [17] for discrete situation, this paper presents an analytical framework for the running time of evolutionary programming. We also discuss whether the running time of CEP and FEP is influenced by individual number, problem dimension number, searching range, and the Lebesgue measure of the optimal neighborhood in the target problem. Furthermore, we give the approximate conditions under which the EP can converge in a timespan less than a polynomial of n.

2. EP Algorithms and the Associated Markov Chain

2.1. Introduction to EP Algorithms

This section introduces the two EP algorithms CEP [2] and FEP [7] that are studied in this paper. The skeleton of EP algorithms analyzed in this paper is given in Algorithm 1; the sole difference among CEP, FEP, and LEP lies in their treatments of Step  (3).

Algorithm 1 (framework of EP algorithms).

Generate individuals randomly.

Calculate the fitness of each individual.

Generate offspring by mutation.

Evaluate the offspring.

Select individuals by a tournament rule.

If the terminal condition is satisfied, output the best-so-far solution found and exit; otherwise, go back to Step  (3).

In Step (1), the generated k individuals are denoted by a couple of real vectors v = (x , σ ), i = 1,2,…, k, where the elements x = (x , x ,…, x ) are the variables to be optimized, and the elements σ = (σ , σ ,…, σ ) are variation variables that affect the generation of offspring. We set the iteration number t = 0 and the initial σ ≤ 2  (j = 1,…, n) as proposed in [8]. The fitness values in Steps  (2) and (4) are calculated according to the objective function of the target problem.

In Step  (3), a single is generated for each individual v , where i = 1,…, k. For j = 1,…, n, where V(σ ) denotes a renewing function of variation variable σ . The renewing function may have various forms, and a representative implementation of it can be found in [7]. However, for ease of theoretical analysis, we will consider a kind of simple renewing function, that is, constant function, in our case studies (see Section 4):The three EP algorithms differ in how is derived, which can be explained as follows (in this paper, we only consider CEP and FEP): where N (0,1) is a newly generated random variable by Gaussian distribution for each j and δ is a Cauchy random number produced anew for jth dimension, whose density function is

In Step  (5), for each individual in the set of all parents and offsprings, q different opponent individuals are uniformly and randomly selected to be compared. If the selected individual's fitness value is more than the opponent's, the individual obtains a “win.” The top k individuals with the most “wins” are selected to be the parents in the next iteration, breaking ties by greater fitness.

2.2. Target Problem of EP Algorithms

Without loss of generality, we assume that the EP approaches analyzed in this study aim to solve a minimization problem in a continuous search space, defined as follows.

Definition 2 (minimization problem).

Let S⊆R be a finite subspace of the n-dimensional real domain R , and let f : S → R be an n-dimensional real function. A minimization problem, denoted by the 2-tuple (S, f), is to find an n-dimensional vector x min ∈ S such that, ∀x ∈ S, f(x min) ≤ f(x).

Without loss of generality, we can assume that S = ∏ [−b , b ] where b = b > 0. The function f : S → R is called the objective function of the minimization problem. We do not require f to be continuous, but it must be bounded. Furthermore, we only consider the unconstrained minimization.

The following properties are assumed, and we will make use of them in our analyses:

The subset of optimal solutions in S is nonempty.

Let f = min{f(x)∣x ∈ S} be the fitness value of an optimal solution, and let S (ɛ) = {x ∈ S∣f(x) < f + ɛ} be the optimal epsilon neighborhood. Every element of S (ɛ) is an optimal solution of the minimization problem.

∀ɛ > 0, m(S (ɛ)) > 0, where we denote the Lebesgue measure of S (ɛ) as m(S (ɛ)).

The first assumption describes the existence of optimal solutions to the problem. The second assumption presents a rigorous definition of optimal solution for continuous minimization optimization problems. The third assumption indicates that there are always solutions whose objective values are distributed continuously and arbitrarily close to the optimal, which makes the minimization problem solvable.

2.3. Markov Chain Associated with EP Algorithms

Our running time analyses are based on representing the EP algorithms as Markov chain. In this section, we explain the terminologies and notations used throughout the rest of this study.

Definition 3 (stochastic process of EP).

The stochastic process of an evolutionary programming algorithm EP is denoted by {ξ EP} +; ξ EP = (v 1 (, v 2 (,…, v (), where v ( = (x (, σ () is the ith individual at the tth iteration.

The stochastic status ξ EP represents the individuals of the tth iteration population for the algorithm EP. All stochastic processes examined in this paper are discrete-time; that is, t ∈ Z +.

Definition 4 (status space).

The status space of EP is Ω EP = {(v 1, v 2,…, v )∣v = (x , σ ), x ∈ S, σ ≤ 2, j = 1,…, n}.

Ω EP is the set of all possible population statuses for EP. Intuitively, each element of Ω EP is associated with a possible population in the implementation of EP. Let x ∈ S (ɛ) be an optimal solution. We define the optimal status space as follows.

Definition 5 (optimal status space).

The optimal status space of EP is the subset Ω EP (ɛ)⊆Ω EP such that, ∀u ∈ Ω EP (ɛ), ∃(x , σ) ∈ u , where σ = (σ, σ,…, σ) and σ > 0.

Hence, all members of Ω EP (ɛ) contain at least one optimal solution represented by individual x .

Definition 6 (Markov chain).

A stochastic process {ξ } + with status space Ω is a Markov chain if , .

Lemma 7 .

The stochastic process {ξ } + of EP is a Markov chain.

Proof

The proof is given in the appendix.

We now show that the stochastic process {ξ EP} + of EP is an absorbing Markov chain, defined as follows.

Definition 8 (absorbing Markov chain to optimal status space).

A Markov chain {ξ } + is an absorbing Markov chain to Ω if ∃Ω ⊂ Ω, such that P{ξ ∉ Ω ∣ξ ∈ Ω } = 0 for t = 0,1,….

Lemma 9 .

The stochastic process {ξ } + of EP is an absorbing Markov chain to Ω (ɛ).

The proof is given in the appendix.

This property implies that once an EP algorithm attains an optimal state, it will never leave optimality.

3. General Runtime Analysis Framework for EP Algorithms

The analysis of EP algorithms has usually been approximated using a simpler measure known as the first hitting time [13, 17], which is employed in this study.

Definition 10 (first hitting time of EP).

μ EP is the first hitting time of EP if μ EP = min{t ≥ 0 : ξ EP ∈ Ω EP (ɛ)}.

If an EP algorithm is modeled as an absorbing Markov chain, the running time of the EP can be measured by its first hitting time μ EP. We denote its expected value by Eμ EP which can be calculated by Theorem 11.

Let λ EP = P{ξ EP ∈ Ω EP (ɛ)} = P{μ EP ≤ t} be the probability that EP has attained an optimal state by time t.

Theorem 11 .

If lim⁡λ = 1, the expected first hitting time Eμ = ∑ +(1 − λ ).

Corollary 12 .

The expected first hitting time can also be expressed as Eμ = (1 − λ 0 )∑ +∏ (1 − P{ξ ∈ Ω (ɛ)∣ξ ∉ Ω (ɛ)}).

The proof is given in the appendix.

The conclusions of Theorem 11 and Corollary 12 are a direct approach to calculating the first hitting time of EP. However, Corollary 12 is more practical than Theorem 11 for this purpose since p = P{ξ EP ∈ Ω EP (ɛ)∣ξ EP ∉ Ω EP (ɛ)}, which is the probability that the process first finds an optimal solution at time i, is dependent on the mutation and selection techniques of the EP. Hence, the framework of Corollary 12 is similar to the one in [17]. However, the estimating method [17] is based on a probability q in discrete optimization q = ∑ P{ξ EP ∈ Ω EP (ɛ)∣ξ EP = y}P{ξ EP = y}. p is a probability for continuous status while q is for the discrete one. In general, the exact value of p is difficult to calculate. Alternatively, first hitting time can also be analyzed in terms of an upper and a lower bound of p , as shown by the following Corollaries 13 and 15.

Corollary 13 .

If α ≤ P{ξ ∈ Ω (ɛ)∣ξ ∉ Ω (ɛ)} ≤ β ,

∑ +[(1 − λ 0 )∏ (1 − β )] ≤ Eμ ≤ ∑ +[(1 − λ 0 )∏ (1 − α )] where 0 < α and β < 1;

β −1(1 − λ 0 ) ≤ Eμ ≤ α −1(1 − λ 0 ) when α = α and β = β.

The proof is given in the appendix.

Corollary 13 indicates that Eμ EP can be studied by the lower bound and upper bound of P{ξ EP ∈ Ω EP (ɛ)∣ξ EP ∉ Ω EP (ɛ)}. Therefore, the theorem and corollaries introduce a first-hitting-time framework for the running time analysis of EP. The running times of EP based on Gaussian and Cauchy mutation are studied following the framework.

4. Runtime Upper Bounds of CEP and FEP

In this section, we use the framework proposed in Section 3 to study the running time of EP based on Gaussian and Cauchy mutations, that is, CEP and FEP. Moreover, our running time analysis aims at a class of EP with constant variation, as shown in (2).

4.1. Mean Running Time of CEP

CEP [2] is a classical EP, from which several continuous evolutionary algorithms are derived. This subsection mainly focuses on the running time of CEP proposed by Fogel [3], Schwefel [6], and Schwefel [5, 6]. The mutation of CEP is based on the standard normal distribution indicated by (3).

According to the running time analysis framework, the probability p = P{ξ ∈ Ω EP (ɛ)∣ξ ∉ Ω EP (ɛ)} for CEP is a crucial factor, and its property is introduced by the following.

Theorem 14 .

Let the stochastic process of CEP be denoted by {ξ } +. Let k be the population size of CEP with solution space S = ∏ [−b , b ] where b = b > 0. Then ∀ɛ > 0, one has the following:

For fixed individual , where the renewing function V(σ ) = σ.

The right part of the inequality of conclusion (1) is maximum when σ = 2b.

if σ = 2b.

The proof is given in the appendix.

In Theorem 14, Part (1) indicates a lower bound of the probability that an individual x can be renewed to be in the optimal neighborhood S (ɛ). As a result, the lower bound is a function of variation σ. Part (2) presents the lower bound that Part (1) will be tightest if σ = 2b. By this, the key factor of Corollary 13   P{ξ ∈ Ω EP (ɛ)∣ξ ∉ Ω EP (ɛ)} may have a tight lower bound shown by Part (3).

Theorem 14 indicates that a larger m(S (ɛ)) leads to faster convergence for CEP since P{ξ ∈ Ω EP (ɛ)∣ξ ∉ Ω EP (ɛ)} becomes larger when m(S (ɛ)) is larger. Moreover, Theorem 14 produces the lower bound of P{ξ ∈ Ω EP (ɛ)∣ξ ∉ Ω EP (ɛ)}, with which the first-hitting time of CEP can be estimated following Corollary 13. Corollary 15 indicates the convergence capacity and running time upper bound of CEP.

Corollary 15 .

Supposed conditions of Theorem 14 are satisfied:

lim⁡λ = 1  (λ = P{ξ ∈ Ω (ɛ)});

∀ɛ > 0,

The proof is given in the appendix.

Corollary 15 shows that CEP converges globally and that the upper bound of CEP's running time decreases as the Lebesgue measure of the optimal ɛ neighborhood m(S (ɛ)) of S (ɛ) increases. A larger m(S (ɛ)) translates to a larger optimal ɛ neighborhood ∀ɛ > 0, which allows the solution of CEP more easily. Moreover, an increase in problem dimension number can increase the upper bound, and enlarging the individual number will make the upper bound smaller.

According to (6), Eμ has a smaller upper bound when the population size k increases. Approximately,since when k is large. Furthermore, the running time of CEP is similar to an exponential order function of size n, if m(S (ɛ)) ≥ C 0 > 0 where C 0 is a positive constant. Hence, the running time of CEP is nearly .

Moreover, we can give an approximate condition under which CEP can converge in time polynomial to n when , where c = (1 − λ 0 )/P > 0.

Suppose (7) holds. Then Thus, the running time of CEP can be polynomial to n, under the constraint that, ∀ɛ > 0, , where S (ɛ) = {x ∈ S∣f(x) < f + ɛ}.

4.2. Mean Running Time of FEP

FEP was proposed by Yao et al. [7] as an improvement to CEP. The mutation for FEP is based on the Cauchy distribution indicated by (4). The property of the probability p = P{ξ ∈ Ω EP (ɛ)∣ξ ∉ Ω EP (ɛ)} is discussed by Theorem 16  (t = 0,1,…).

Theorem 16 .

Let {ξ } + be the stochastic process of FEP, and then one has the following:

.

The right part of the inequality of conclusion (1) is maximum when σ = 2b.

P{ξ ∈ Ω (ɛ)∣ξ ∉ Ω (ɛ)} ≥ 1 − (1 − m(S (ɛ))/(4πb)) if σ = 2b.

The proof is given in the appendix.

Similar to Theorem 14, FEP with σ = 2b may lead to a tight lower bound of the probability in Part (1) and Part (3). Theorem 16 indicates that m(S (ɛ)) affects the convergence of FEP directly. A larger m(S (ɛ)) allows FEP to more easily arrive at a status in the optimal state space (Definition 5). Conversely, a bigger b (the bounds of search space) increases the difficulty of the problem for FEP. Given this convergence time framework, Corollary 17 summarizes the convergence capacity and running time upper bound of FEP.

Corollary 17 .

Supposed conditions of Theorem 16 are satisfied:

lim⁡λ = 1  (λ = P{ξ ∈ Ω (ɛ)});

∀ɛ > 0, Eμ ≤ (1 − λ 0 )(1 − (1 − m(S (ɛ))/(4bπ))).

The proof is given in the appendix.

Corollary 17 proves the convergence of FEP and indicates that a larger m(S (ɛ)) can make FEP converge faster. Using a similar analysis to that for CEP, a larger dimension number of problems n can increase the upper bound of FEP, but larger individual number k can lead to a smaller upper bound. Furthermore, [−b, b] is the maximum interval bound for each dimension of the variables to be optimized (Definition 2). As a result, the second conclusion of Corollary 17 also implies that a larger search space will increase the upper bound of Eμ EP.

If k becomes large enough, we have (1 − m(S (ɛ))/(4bπ)) ≈ 1 − k(m(S (ɛ))/(4bπ)) such that

Hence, the running time of FEP is nearly O((4bπ)) when m(S (ɛ)) is a constant greater than zero.

Moreover, we can give an approximate condition where FEP can converge in polynomial time P(n); that is, m(S (ɛ))≥((1 − λ 0 )/k)·((4bπ)/P(n)), ∀ɛ > 0 when (8) is true. The analysis is similar to the one for CEP at the end of Section 4.1.

5. Conclusion

In this paper, we propose a running time framework to calculate the mean running time of EP. Based on this framework, the convergence and mean running time of CEP and FEP with constant variation are studied. We also obtain some results at the worst running time of the considered EP, although the results show that the upper bounds can be tighter if the variation σ = 2b where 2b is the length of the searching interval per dimension. It is shown that the individual number, problem dimension number, searching range, and the Lebesgue measure of the optimal neighborhood of the optimal solution have a direct impact on the bounds of the expected convergence times of the considered EP. Moreover, the larger the Lebesgue measure of the optimal neighborhood of the optimum, the lower the upper bound of the mean convergence time. In addition, the convergence time of the EP can be polynomial on average on the condition that the Lebesgue measure is greater than a value that is exponential to b.

However, it is possible to make an improvement on the running time framework and analysis given in this study. The deduction process in the proofs for Theorems 14 and 16 uses few properties of the distribution functions of the mutations, and so it is possible to tighten the results. By introducing more information on the specific mutation operations, more sound theoretical conclusions may be derivable. More importantly, unlike the rigorous conditions under which CEP and FEP can converge in polynomial time, the running time analysis of specific EP algorithms for real-world and constructive problems would have a more significant and practical impact. Future research could also focus on the runtime analysis of specific case studies of EP.

The proposed framework and results can be considered as a first step for analyzing the running time of evolutionary programming algorithms. We hope that our results can serve as a basis for further theoretical studies.