Literature DB >> 28630619

A Swarm Optimization Genetic Algorithm Based on Quantum-Behaved Particle Swarm Optimization.

Tao Sun1,2, Ming-Hai Xu1.   

Abstract

Quantum-behaved particle swarm optimization (QPSO) algorithm is a variant of the traditional particle swarm optimization (PSO). The QPSO that was originally developed for continuous search spaces outperforms the traditional PSO in search ability. This paper analyzes the main factors that impact the search ability of QPSO and converts the particle movement formula to the mutation condition by introducing the rejection region, thus proposing a new binary algorithm, named swarm optimization genetic algorithm (SOGA), because it is more like genetic algorithm (GA) than PSO in form. SOGA has crossover and mutation operator as GA but does not need to set the crossover and mutation probability, so it has fewer parameters to control. The proposed algorithm was tested with several nonlinear high-dimension functions in the binary search space, and the results were compared with those from BPSO, BQPSO, and GA. The experimental results show that SOGA is distinctly superior to the other three algorithms in terms of solution accuracy and convergence.

Entities:  

Mesh:

Year:  2017        PMID: 28630619      PMCID: PMC5463199          DOI: 10.1155/2017/2782679

Source DB:  PubMed          Journal:  Comput Intell Neurosci


1. Introduction

Particle swarm optimization (PSO) algorithm is a population-based optimization method, which was originally introduced by Eberhart and Kennedy in 1995 [1]. In PSO, the position of a particle is represented by a vector in search space, and the movement of the particle is determined by an assigned vector called the velocity vector. Each particle updates the velocity based on its current velocity, the best previous position of the particle, and the global best position of the population. PSO is extensively used for the optimization problems because it has simple structures and is easy to implement. However, it has some disadvantages, such that it easily falls into local optima when solving the complex and high-dimension problems [2, 3]. Hence a number of variant algorithms have been proposed to overcome the disadvantages of PSO [4, 5]. The particle swarm algorithm based on the probability convergence is one of the variant algorithms. This kind of particle swarm algorithm allows the particles to move according to probability instead of using velocity-displacement particle movement way. The Bare Bones PSO (BBPSO) family is a typical class of probabilistic PSO algorithms [6-8]. The Gaussian distribution was used in the original version of BBPSO, which was proposed by Kennedy [6]. then several new BBPSO variants used other distributions which seem to generate better results [7-9]. Inspired by the quantum theory and the trajectory analysis of PSO [10], Sun et al. proposed a new probabilistic algorithm, quantum-behaved particle swarm optimization (QPSO) algorithm [11]. In QPSO, each particle has a target point, which is defined as a linear combination of the best previous position of the particle and the global best position. The particle appears around the target point following a double exponential distribution. The QPSO algorithm essentially belongs to the BBPSO family, and its update equation uses an adaptive strategy and has fewer parameters to be adjusted [12-14]. The QPSO has been shown to perform well in finding the optimal solutions for continuous optimization problems and successfully applied to a wide range of areas such as multiobjective optimization [15, 16], clustering [17-19], neural network training [20-22], image processing [23, 24], engineering design [25], and dynamic optimization [26]. PSO and QPSO have been effective tools for solving global optimization problems, but they were originally developed for continuous search spaces. Kennedy and Eberhart introduced a binary version of PSO for discrete problems named binary PSO (BPSO) [27], where the trajectories are defined as changes in the probability that each particle changes its state to 1. Binary PSO has simple structure and is easy to implement; hence, it is extensively employed in the optimization problems [28-30]. But it also suffers from some disadvantages when solving the complex and high-dimension problems [28]. Sun et al. proposed binary QPSO (BQPSO), in which the target point is obtained by using the crossover operator at the best previous position of the particle and the global best position. Experiment results show that BQPSO can find better solution generally than BPSO [31]. In recent years, BQPSO has been used successfully in many fields [32-34]. However, although BQPSO broadens the application fields of QPSO, it did not show the same advantage as in the continuous space. QPSO algorithm should have better performance in solving the problems based on discrete space. This paper analyzes the main factors that impact the search ability of QPSO and converts the particle movement formula to the mutation condition by the introduction of rejection region. It then designed a new binary coding QPSO, which has crossover and mutation operator and is like genetic algorithm (GA) in form; that is, the proposed algorithm is a new genetic algorithm but incorporates the core idea of QPSO. So it was named swarm optimization genetic algorithm (SOGA). Compared with the GA, the SOGA has no selection operator, and each individual participates in evolution based on the information of the population and its own information. At the same time, the mutation probability of the SOGA is not fixed. In the early stage of the algorithm, the probability of mutation is large and the population can keep the diversity, with the iteration of the algorithm, the mutation probability tends to zero, and the algorithm can finally converge. The rest of this paper is organized as follows. Section 2 is a brief introduction of PSO and binary PSO; Section 3 summarizes QPSO and binary QPSO; Section 4 introduces the mutation condition of binary coding converted from the particle movement formula in QPSO; Section 5 proposes the new binary QPSO algorithm, SOGA, and then discusses the difference between this algorithm and QPSO, GA; Section 6 presents the experiment results from the benchmark functions; finally, the paper is concluded in Section 7.

2. Particle Swarm Optimization

Particle swarm optimization (PSO) algorithm is a population-based optimization technique used in continuous spaces. It can be mathematically described as follows. Assume the size of the population is n and the dimension of the search space is q; then the ith particle of the swarm can be represented by a position vector X = (x, x,…, x); the velocity of a particle i is denoted by vector V = (v, v,…, v); vector P = (p, p,…, p) is the best previous position of particle i, called personal best position, and P = (p, p,…, p) is the best position of the population, called global best position. The velocity of particle i is calculated accordingly:where i = 1,2,…, n, j = 1,2,…, q,  n is population size, k is the number of iterations, ω is inertia weight, c1 and c2 are acceleration coefficients, and r1 and r1 are random numbers in the interval [0,1]. Then the next position is updated as follows:The PSO algorithm is applied to solve optimization problems in the real search space, but many optimization problems are set in discrete space. Kennedy and Eberhart proposed a discrete binary version of PSO, named binary PSO (BPSO), where the particle position has two possible values, “0” or “1.” The velocity formula in BPSO remains unchanged, and the particle position is updated as follows:where rand is a random number in the interval [0,1] and the function S(v) is a Sigmoid function as

3. Quantum-Behaved Particle Swarm Optimization

Inspired by trajectory analyses of PSO in [10], Sun et al. proposed a novel variant of PSO, named quantum-behaved particle swarm optimization (QPSO), which outperforms the traditional PSO in search ability. QPSO sets a target point for each particle; denote G = (g, g,…, g) as the target point for particle i, of which the coordinates arewhere ϕ is a random number in the interval [0,1]. the trajectory analysis in [10] shows that G is the local attractor of particle i; that is, in PSO, particle i converges to it. The position of particle i is updated as follows:where u is a random number in the interval [0,1] and C = [c1, c1,…, c] is known as the mean best position that is defined by the average of the personal best position of all particles, accordingly,Parameter α is called Contraction-Expansion Coefficient, which can be tuned to control the convergence speed of the algorithms. Because the iterations of QPSO are different from those of PSO, the methodology of BPSO cannot be applied to QPSO. Sun et al. introduced the crossover operator of GA into QPSO and proposed binary QPSO (BQPSO). In BQPSO, X = (x, x,…, x) still represents the position of particle i, but it is necessary to emphasize that X is a binary string rather than a vector, and x is the jth substring of X, not the jth bit in the binary string. Assume the length of each substring is l; then the length of X is lq. The target point G for particle i is generated through crossover operator; that is, BQPSO exerts crossover operation on the personal best position P and the global best position P to generate two offspring binary strings, and G is randomly selected from them. Definewhere k is the number of iterations and d(c, x) is the Hamming distance between c and x. Compared with the two bit strings, the Hamming distance is the count of bit difference in the two strings. c is the jth substring of the mean best position, and the dth bit of c is determined by the states of the dth bit of all particles' personal best positions. If more particles take on 1 at the dth bit, the dth bit of c is 1; otherwise the bit will be 0. For each bit of g, when p > rand execute operations as follows: if the state of the bit is 1, then set its state to 0; else set its state to 0.

4. A Mutation Condition Using in Binary Space

The reason why the QPSO algorithm has better global search capability than the traditional PSO algorithm is that it changes the velocity-displacement model of the traditional PSO algorithm; in QPSO, the movement of particle to its target point has no determined trajectory; it can appear at any position in the whole feasible search space with a certain distribution, which is the double exponential distribution [13, 14]. Such a position can be far from the target point and may be superior to the current global best position of the population. This should also be reflected in the construction of binary QPSO algorithm. The probability density function of particle i in QPSO isSet λ = 2/L, and y = X − G; then (9) can be rewritten asThat is, y obeys the double exponential distribution, of which the mean and variance are E(y) = 0 and D(y) = 2/λ2. The graph of probability density function (10) is Figure 1. Since the domain of y is (−∞, +∞), particle can appear in any position of the search space, but the probability that a particle appears in a position far away from its target point is small. When λ → +∞, the variance D(y) = 2/λ2 → 0 which means that X converge to G with probability 1.
Figure 1

Probability density function of double exponential distribution.

When the position of a particle uses binary encoding, it is hard to describe the relative position of two points using the measure of two binary strings. Similar to set a rejection region, we set a threshold value v  (v > 0). When the value of y falls into the rejection region, as shown in Figure 2, set y = 0; that is, X = G, else X = mutation(G). mutation(G) means mutation operation on G.
Figure 2

Refused domain of probability density function.

For any u, which is a random number in the interval [0,1], the condition that y does not fall into the rejection region isThe left side of Condition (11) can be written asThus Condition (11) means e− > u, accordingly:In order to ensure that the algorithm can converge, setwhere C is the mean best position of the population. Then Condition (13) iswhere d(·) is used to measure the difference of two binary strings. Hamming distance can be used here. Assume y = ln⁡(1/u); then For (16), when the value of y is small, the function has fast rates of change as shown in Figure 3, so Condition (15) suffers from the effect of the initial value of d(C, X). So Condition (15) can be changed into its equivalent form:where parameter σ is a constant that is greater than zero.
Figure 3

Figure of equation (16).

5. Swarm Optimization Genetic Algorithm

Based on the mutation condition (17), mutation operator is introduced into BQPSO. X still represents the position of particle i, P is the personal best position of particle i, P is the global best position, and C is the mean best position which is defined the same as in BQPSO. Different from BQPSO, crossover or mutation operation process is applied to the whole binary string, instead of bits. Because the procedure of the algorithm is similar to GA, it is named as swarm optimization genetic algorithm (SOGA). The process can be described as follows.Compared to the GA with the same crossover and mutation operator, SOGA has the following characteristics: Initialize a population of particles X in binary space; Set personal best position P = X, and compute C; Evaluate the fitness of particles f(X) and determine the global best position P; while terminate condition is not reached do for each particle i do Exert crossover operation on P and P to generate two offspring binary strings, G is randomly selected from them. if condition (17) is true, Exert mutation operation on G; end if Set X = G; Compute the fitness of particles f(X), and update P, end for i Update P and the mean best position C; end while SOGA does not have selection operator and crossover probability and its crossover operator is exerted directly on P and P. Therefore, the form of the fitness function f(X) has no effect on the algorithm, and the target function of the maximization problem can be set as the fitness function. Condition (17) can be turned into Since σd(C, X) ≥ 0, the range of exp(−σd(C, X)) is (0,1), and u is a random number in the interval [0,1]; thus Condition (18) is equivalent to an adaptive mutation probability: where σ is a constant that is greater than zero and d(C, X) decreases with the increase of iteration times. Therefore, p is shrunk, which causes the algorithm to converge. σ is the only parameter of SOGA, which can be tuned to control the convergence speed of the algorithms as Contraction-Expansion Coefficient α in BQPSO. When the value of σ is 0.5, 1, and 2, the curves of mutation probability p changing with d(·) are shown in Figure 4. The figure demonstrates that the smaller the value of σ, the faster the convergence speed of the algorithm. It also can be seen that the global searching ability of the algorithm is reduced when σ is too small. So set σ = 1 in SOGA.
Figure 4

Figures of mutation probability.

6. Experimental Results

The proposed SOGA is compared with BPSO, BQPSO, and GA. They are tested on the following 10 benchmark problems to be minimized [28, 35]: (1) Sphere Function (2) Schwefel's Problem 2.22 (3) Schwefel's Problem 1.2 (4) Step Function (5) Schwefel's Problem 2.21 (6) 2 Minima Function (7) Schwefel's Problem 1.2 (8) Ackley Function (9) Generalized Penalized Function (10) Griewank FunctionIn these functions, F1 ~ F5 are unimodal and F6 ~ F10 are multimodal. Their optimum values are all zeros except F6 and F7. The minimum values of F6 and F7 are −78.3323 and −418.9829∗n, respectively, where n is the dimension of a function. In the experiments, the dimension of each function is 8, and the binary code length of each continuous variable is 15, so the length of particle is 120 for each function. The size of population is 50 and the total number of iterations is set to 500. The parameters of algorithms are listed in Table 1, where p is crossover probability and p is mutation probability in GA.
Table 1

Parameters of algorithms applied in the experiments.

AlgorithmParameter settings
SOGA σ = 1
BPSO ω = 0.7,  c1 = c2 = 2,  Vmax = 6
BQPSO α = 1.1~1.4
GA p c = 0.90,  pm = 0.10~0.15
Four algorithms ran independently 30 times on the benchmark functions, and the best target function value was recorded at each run. To compare the four algorithms, 30 data sets were analyzed using the following statistic parameters: the mean, the standard deviation (STD), the best, the worst, and the median; these results are reported in Tables 2 and 3.
Table 2

Minimization results for BPSO, BQPSO, and SOGA.

FunctionAlgorithmThe bestMeanSDThe worstMedian
F 1 BPSO65.1674278.1128160.0441702.2221244.4259
BQPSO1.40963.36471.42726.53273.1107
SOGA7.4510e − 051.6641e − 044.8945e − 040.00287.4510e − 05

F 2 BPSO1.59183.02680.93005.34012.9463
BQPSO0.21550.40800.12260.70250.4025
SOGA0.00240.00260.00050.00490.0024

F 3 BPSO121.83951171.3480911.21653261.0044860.7055
BQPSO8.211230.875225.8621120.603421.4084
SOGA263.91701919.60761078.96503782.16232175.0068

F 4 BPSO56312.7667248.96191154250.5
BQPSO14.86672.4174115
SOGA00.10000.402620

F 5 BPSO9.024316.30895.567727.549114.3406
BQPSO0.94301.84370.51413.32961.8342
SOGA0.00310.22280.62413.12810.0275

F 6 BPSO−77.5179−74.51461.8144−70.6644−74.5941
BQPSO−72.7114−69.33851.7327−66.4601−68.9703
SOGA−77.7357−76.27890.6355−75.1676−76.2048

F 7 BPSO−3136.9403−2724.3377194.8185−2263.5189−2752.3562
BQPSO−1810.3356−1498.0555157.2211−1272.7048−1454.7974
SOGA−3248.4725−2913.3245218.1944−2467.9053−2952.6730

F 8 BPSO4.54567.63152.023015.52157.8838
BQPSO0.94461.92300.38422.58181.9273
SOGA0.00401.36141.17033.12761.8407

F 9 BPSO2.692512.68177.353531.350210.9785
BQPSO0.72941.76350.57482.94551.7687
SOGA0.07421.46391.43865.25940.9397

F 10 BPSO1.57983.66061.55827.76803.4519
BQPSO0.35520.83600.20631.15610.8923
SOGA0.05460.37020.19330.67380.4183
Table 3

Minimization results for SOGA and GA.

FunctionAlgorithmThe best MeanSDThe worstMedian
F 1 SOGA7.4510e − 051.6641e − 044.8945e − 040.00287.4510e − 05
GA0.001376.0248166.2636664.49372.5246
SOGA7.4510e − 050.00740.01890.07730.0007
GA0.00161.13022.371711.09530.2626

F 2 SOGA0.00240.00260.00050.00490.0024
GA0.00310.32200.42941.35870.1172
SOGA0.00310.00690.00760.04330.1059
GA0.02200.14230.14520.77032.5471

F 3 SOGA263.91701919.60761078.96503782.16232175.0068
GA823.09823805.954621587.46658753.21863602.7854
SOGA0.023827.576595.7945434.99000.5557
GA482.15194922.39812417.119810470.92394705.2068

F 4 SOGA00.10000.402620
GA094.3667457.214225091
SOGA00000
GA03.63337.5177362

F 5 SOGA0.00310.22280.62413.12810.0275
GA9.378327.019311.295353.129727.1523
SOGA0.04580.19080.09520.49130.1831
GA6.491320.078310.593749.601718.9795

F 6 SOGA−77.7357−76.27890.6355−75.1676−76.2048
GA−77.9857−75.26361.8587−69.1535−75.5698
SOGA−78.3316−77.22890.6240−75.8823−77.1074
GA−77.5952−74.88861.902−70.5101−75.457

F 7 SOGA−3248.4725−2642.8652256.1202−2001.7012−2952.6730
GA−3113.4015−2088.3848342.5752−1355.6609−2692.5141
SOGA−3351.7352−3111.4602154.6330−2736.5711−3114.6197
GA−3150.5510−2733.6038180.6351−2424.3367−2726.4449

F 8 SOGA0.00401.36141.17033.12761.8407
GA1.84093.34751.862610.21852.6024
SOGA0.00400.02230.02160.09110.0141
GA0.33842.77640.92624.34582.7676

F 9 SOGA0.07421.46391.43865.25940.9397
GA1.546810.28837.670334.34958.7194
SOGA0.03480.61950.53171.93440.4575
GA0.66796.00074.007413.66475.7359

F 10 SOGA0.05460.37020.19330.67380.4183
GA0.16000.96331.38117.93500.6882
SOGA0.25500.47150.12080.75480.4758
GA0.18050.74920.24311.29990.7741

The crossover and mutation operation act on substring.

Moreover, the statistical test is conducted in order to determine whether the average best results are different with a statistical significance. The confidence level is fixed at 0.95, and the tests return p value which are shown in Tables 4 and 5. We use the SAS for statistical testing; in the SAS system, if the p value is less than 0.0001, the system displays <0.0001. The value of h in Tables 4 and 5 shows the result of pairwise comparison; h = 1 indicates the previous comparison algorithm is significantly better than the latter; h = 0 represents no significant difference between the two compared algorithms; h = −1 indicates the previous comparison algorithm is significantly worse than the latter.
Table 4

Comparison of SOGA with other algorithms and GA with GA.

FunctionTestSOGAGA
BPSOBQPSOGAGAGA
F 1 p value<0.0001<0.00010.01510.01150.0166
h 11111
F 2 p value<0.0001<0.00010.0001<0.00010.0340
h 11111
F 3 p value0.0724<0.0001<0.0001<0.00010.4280
h −1−1110
F 4 p value<0.0001<0.00010.26330.01190.2816
h 11010
F 5 p value<0.0001<0.0001<0.0001<0.00010.0171
h 11111
F 6 p value<0.0001<0.00010.0006<0.00010.4430
h 11110
F 7 p value0.0081<0.00010.00040.00800.2582
h 11110
F 8 p value<0.00010.0002<0.0001<0.00010.2063
h 11110
F 9 p value<0.00010.3884<0.0001<0.00010.0043
h 10111
F 10 p value<0.0001<0.00010.0291<0.00010.4065
h 11110

The crossover and mutation operation act on substring.

Table 5

Comparison of SOGA with other algorithms.

FunctionTestSOGA
BPSOBQPSOGAGASOGA
F 1 p value<0.0001<0.00010.01510.01200.0409
h 1111−1
F 2 p value<0.0001<0.00010.0002<0.00010.0308
h 1111−1
F 3 p value<0.00010.8561<0.0001<0.0001<0.0001
h 10111
F 4 p value<0.0001<0.00010.26290.01040.1555
h 11010
F 5 p value<0.0001<0.0001<0.0001<0.00010.2768
h 11110
F 6 p value<0.0001<0.00010.0002<0.00010.3923
h 11110
F 7 p value<0.0001<0.0001<0.0001<0.0001<0.0001
h 11111
F 8 p value<0.0001<0.0001<0.0001<0.0001<0.0001
h 11111
F 9 p value<0.0001<0.0001<0.0001<0.00010.0044
h 11111
F 10 p value<0.0001<0.00010.0435<0.00010.1231
h 11110

The crossover and mutation operation act on substring.

The results of SOGA compared with BPSO and BQPSO are listed in Tables 2 and 4. The results show that SOGA surpasses BPSO and BQPSO in minimizing the ten benchmark functions except F3. Figure 5 illustrates the convergence process of the best target function value of population in one running. As shown in Figure 5, the SOGA converges faster than BPSO and BQPSO.
Figure 5

Figures of the convergence processes of BPSO, BQPSO, and SOGA.

Since SOGA has almost the same form as GA, the same crossover and mutation operator, single-point crossover and single-point mutation, are used in both algorithms. In GA, the elitist strategy is applied to improve the convergence and optimization results. It should be noted that the GA can not converge after 500 iterations for most of the functions; for better comparison, the number of iterations of the GA is set to 2000 in Table 3 to ensure that the algorithm is fully convergent. For high-dimension functions, assume X = (x, x,…, x) is the binary string of the particle (or individual) i, where q is the number of dimensions and x is the jth substring of X. It is easy for GA or SOGA to exert crossover and mutation operation on each substring x in turn, instead of on the whole X. For instance, in SOGA, Condition (17) can be written asfor each substring x of X, where c are the jth substring of C. Then the process of SOGA when the crossover and mutation operation act on substring can be described as follows: the same operation can also be used in GA. Initialize a population of particles X in binary space; Set personal best position P = X, and compute C; Evaluate the fitness of particles f(X) and determine the global best position P; while terminate condition is not reached do for each particle i do for each substring of particle j do Exert crossover operation on P and P to generate two offspring binary strings, G is randomly selected from them. if condition (30) is true, Exert mutation operation on G; end if Set X = G; end for j Compute the fitness of particles f(X), and update P, end for i Update P and the mean best position C; end while The convergence processes of SOGA and GA, when crossover and mutation operation act on substrings of particles (or individual), are shown in Figure 6; the results of SOGA and GA are listed in Tables 3, 4, and 5. The experimental results show that SOGA is obviously superior to the GA on the solution accuracy and the convergence. For high-dimension functions, it is effective to improve the convergence speed and optimization ability, exerting crossover and mutation operation on substrings; as shown in Tables 3 and 4, it significantly improves the convergence rate of GA. But its influence is not significant for SOGA; according to Table 5, it has better performance in minimizing functions F3, F7, F8, and F9, especially for function F3.
Figure 6

Figures of the convergence processes of SOGA and GA when the crossover and mutation operation act on substring.

7. Conclusions

In this study, SOGA, a binary swarm intelligence algorithm, which is based on QPSO and binary QPSO, is introduced. It converts the movement formula of QPSO to mutation conditions, thus introducing the mutation operator of GA. SOGA has the similar form to GA but does not need to set the crossover and mutation probability, so it has fewer parameters to control. SOGA integrates strongpoint of GA and PSO. The experimental results show that SOGA is distinctly superior to BPSO, BQPSO, and GA in terms of solution accuracy and convergence. Furthermore, since SOGA has the same crossover and mutation operator as GA, many improvements on the GA can be applied to it; therefore, this algorithm has better applications and research prospects.
  2 in total

1.  Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection.

Authors:  Jun Sun; Wei Fang; Xiaojun Wu; Vasile Palade; Wenbo Xu
Journal:  Evol Comput       Date:  2011-12-12       Impact factor: 3.277

2.  Cancer Feature Selection and Classification Using a Binary Quantum-Behaved Particle Swarm Optimization and Support Vector Machine.

Authors:  Maolong Xi; Jun Sun; Li Liu; Fangyun Fan; Xiaojun Wu
Journal:  Comput Math Methods Med       Date:  2016-08-24       Impact factor: 2.238

  2 in total
  2 in total

1.  Z-Shaped Transfer Functions for Binary Particle Swarm Optimization Algorithm.

Authors:  Sha-Sha Guo; Jie-Sheng Wang; Meng-Wei Guo
Journal:  Comput Intell Neurosci       Date:  2020-06-08

2.  Binary Particle Swarm Optimization Intelligent Feature Optimization Algorithm-Based Magnetic Resonance Image in the Diagnosis of Adrenal Tumor.

Authors:  Jian Xu; Fei Tian; Lei Wang; Zhongchang Miao
Journal:  Contrast Media Mol Imaging       Date:  2022-02-28       Impact factor: 3.161

  2 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.