Literature DB >> 28403224

HSTLBO: A hybrid algorithm based on Harmony Search and Teaching-Learning-Based Optimization for complex high-dimensional optimization problems.

Shouheng Tuo1, Longquan Yong1, Fang'an Deng1, Yanhai Li1, Yong Lin1, Qiuju Lu1.   

Abstract

Harmony Search (HS) and Teaching-Learning-Based Optimization (TLBO) as new swarm intelligent optimization algorithms have received much attention in recent years. Both of them have shown outstanding performance for solving NP-Hard optimization problems. However, they also suffer dramatic performance degradation for some complex high-dimensional optimization problems. Through a lot of experiments, we find that the HS and TLBO have strong complementarity each other. The HS has strong global exploration power but low convergence speed. Reversely, the TLBO has much fast convergence speed but it is easily trapped into local search. In this work, we propose a hybrid search algorithm named HSTLBO that merges the two algorithms together for synergistically solving complex optimization problems using a self-adaptive selection strategy. In the HSTLBO, both HS and TLBO are modified with the aim of balancing the global exploration and exploitation abilities, where the HS aims mainly to explore the unknown regions and the TLBO aims to rapidly exploit high-precision solutions in the known regions. Our experimental results demonstrate better performance and faster speed than five state-of-the-art HS variants and show better exploration power than five good TLBO variants with similar run time, which illustrates that our method is promising in solving complex high-dimensional optimization problems. The experiment on portfolio optimization problems also demonstrate that the HSTLBO is effective in solving complex read-world application.

Entities:  

Mesh：

Year:  2017        PMID： 28403224      PMCID： PMC5389630          DOI： 10.1371/journal.pone.0175114

Source DB:  PubMed          Journal:  PLoS One        ISSN： 1932-6203            Impact factor:   3.240


1 Introduction

With the scientific and social progress, new complex problems are more and more encountered in the fields of science and engineering. Especially, many high-dimensional optimization problems in engineering design, production scheduling and scientific calculation need urgently to be solved with high performance and high efficiency, for which there are three challenges: the first one is the very large search space owing to the very high dimensional problems (e.g., >500), which makes the enormous computation burden; the second one is the large number of modals (multi-extremum points), which makes the search algorithm be easily trapped into local search; the third one is the particularity of optimization problem that may be discontinuous, non-differentiable and even have no objective function, for which the traditional mathematical optimization algorithms are powerless due to the requiring of substantial gradient information. Therefore, it is of great challenge to discover the globally optimal solution in an efficient time for solving a complex multimodal optimization problem with more than 1000 dimensions, possibly infinite number of local minima and non-differentiable. To address complex optimization problems, the swarm intelligent algorithms, mimicking the collective behavior of decentralized, self-organized systems, natural or artificial, have received much attention in recent years, most of which are nature inspired, such as genetic algorithm (GA)[1] inspired by biological evolution, particle swarm optimization (PSO)[2] mimicking the foraging process of bird flock, differential evolution (DE)[3], artificial bee colony (ABC)[4], Symbiotic organisms Search (SOS)[5] and so on. Comparing with traditional mathematical optimization algorithms, the swarm intelligent optimization algorithms are not limited by requiring substantial gradient information and not dependent on an initialization. Both Harmony Search (HS) [6-7] and Teaching-Learning-Based Optimization (TLBO) [8-9] are new swarm intelligent optimization methods that have attracted increasing interests owing to their excellent characteristics, such as less parameters, simplicity, utilizing real-number encoding and fewer mathematical requirements and so forth. The advantage of HS is that it maintains population diversity very well during the search process; it has strong exploration power for exploring the unknown space. TLBO is powerful in obtaining extraordinary precision solution due to having very strong convergence ability. However, the HS and TLBO algorithms also have some limitations for solving high-dimensional optimization problems with multimodality. The HS has disadvantages on convergence speed, precision of globally optimal solution over TLBO; conversely the TLBO is easy to fall into a local search owing to convergence speed very rapid, which easily results in globally optimal solution lost for some multimodal problems. In the case, several HS variants [10-25] and modified TLBO algorithms [26-29] have been presented to improve the performance for solving complex optimization problems in recent years, such as, SGHS[11], IHS[12], ITHS[14], EHS[15], NGHS[18], DIHS[19], NDHS[20], DSHS[23], ATLBO [26], WTLBO[27], TLBO_GC[28], ITLBO[29] and so on [30-31]. However, these improved variants still are not competent enough to tackle the optimization problems with high-dimensionality (larger than 500) and multimodality. For example, the solution precision of IHS is not satisfactory; NGHS, SGHS and NDHS are easy to trap into local search. EHS and DSHS require taking much time for high-dimensional problems; WTLBO, TLBO_GC and ITLBO cannot avoid premature convergence for complex optimization problems with multimodality. The reason is these state-of-the-art intelligent algorithms have not considered an important that, with the increase of dimensionality, the probability that all values of one of dimensions in population became assimilated and lost the diversity will increase, which will make the algorithm lose exploration power if the algorithm has not good disturbance strategy for escaping from the local search. In this work, we find from the merit and demerit of HS and TLBO that the HS and TLBO have many complementary performance each other. We think it is a viable way in solving the high-dimensional optimization problems with multimodality if a good integration of HS and TLBO can be realized. Commonly, to solve multimodal optimization problems that have one globally optimal solution and many local minimum (maximum) values by employing swarm intelligent algorithm, the overarching goal is to effectively implement the balance between exploration power and exploitation power, where the mission of exploration is to discover the unexplored regions at the early stage of search process, and the exploitation aims to obtain high precision optimal solution in a known region that has been found in the exploration stage. As a consequence, balancing the exploration and the exploitation is very important for solving a high-dimensional optimization with multimodality. Generally, the exploration power is strongly required before discovering the region in which the globally optimal solution is contained inside, however, after the globally optimal region has been found, the exploitation power should be intensified immediately and the exploration power is degraded gradually. To address the balance, in this study, we propose a hybrid optimization algorithm (HSTLBO) based on HS and TLBO, in which a self-adaptive selection strategy is designed to balance the exploration power and the exploitation power. At the early stage of search, the HS algorithm obtains higher probability for exploring the region that contains the globally optimal solution. When the globally optimal region might have been located, the TLBO algorithm begins to obtain higher probability at the later stage of search process for intensifying the local search and exploiting high precision solution. In the HSTLBO, a self-adaptive selection probability is used to choose HS or TLBO in terms of the population diversity and update-success-rate. The update-success-rate denotes the proportion that the new generated solutions are superior to the old ones, which mean the rate that a new generated solution can successfully replace the old one in one generation. The rest of this paper is organized as follows: Section 2 introduces the HS and TLBO algorithm. Hybrid HSTLBO algorithm is proposed and self-adaption selection strategy is analyzed in Section 3. In Section 4, the numerical experiments on twenty complex benchmark test functions are performed, the results are analyzed by comparing and statistical test. And the convergence of HSTLBO is also investigated. In Section 5, HSTLBO is used to solve portfolio optimization problem. Section 6 concludes this work. Some symbols are explained as Box 1.

Box 1

NP  --The population size which is equal to HMS. Tmax --The maximum evaluation times of objective function. t   --The current iteration times. SR  --the selection rate that new generated harmony insteads of the worst harmony. T   --The cycle length for recalculating the SR. r   --An uniform distributed random number between 0 and 1. c1/ c2 --The times of updating old solutions successfully of HS/ TLBO in the tth iteration.

2 Harmony Search and Teaching-Learning-Based Optimization

2.1 Optimization model

Where, X consists of D decision variables (x1, x2, …, x), D denotes the dimensionality (the number of decision variables) of the optimization problem, x (i = 1, 2, …, D) represents the ith decision variable. and separately indicate the upper and lower bound of x.

2.2 HS algorithm

HS algorithm mimics the process of improvising a musical harmony, in which X denotes the harmony, x(i = 1, 2,…, D) indicates the note of harmony, D is the number of notes in a harmony. Harmony memory (HM) contains HMS harmonies: {X1, X2, …, X}, where , HMS denotes the population size. The pseudo code of HS algorithm is as Fig 1.
Fig 1

Pseudo code of standard HS algorithm.

In standard HS, three operators (harmony memory consideration, pitch-adjusting and random disturbance) are employed to optimize the harmonies in HM (population), which are good at exploring new region of search space [14]. However, due to the absence of learning-operator, the convergence speed of HS is much slower than that of TLBO.

2.3 TLBO algorithm

A new swarm intelligent optimization algorithm, TLBO, is proposed by R.V.Rao in 2012, which is inspired by the teaching and learning process of a class [8]. In TLBO, The class {X1, X2,…, X} is composed of one teacher and some learners, where (j = 1, 2,…, NP) (see Box 1) denotes the jth learner, NP is the class size, D represents the number of major subjects in the class; the represents the learning status of jth learner on ith major subject. The optimization process of TLBO is divided into two stages: “teacher phase” and “learner phase”.

Teacher phase

In the teacher phase, the learners increase their knowledge depending on the teacher who tries to improve the mean ability of all learners. The teaching operator is as follow, Where X and X denote the ith learner’s learning status after and before learning from teacher X, T is teaching factor, r is a uniformly distributed random vector in the range [x, x]. M denotes the mean knowledge level of all learners.

Learner phase

In the learner phase, each learner increases knowledge depending on communicating with other learners. The learning operator is as follow, Where r(r ≠ j) is a random integer in the range [1, NP].

3. Proposed HSTLBO algorithm

To balance the exploration power and the exploitation power during the searching process, we propose a complementary HSTLBO algorithm.

3.1 HSTLBO algorithm

The flow chart of HSTLBO algorithm is as Fig 2.
Fig 2

Flow chart of HSTLBO.

In the HSTLBO algorithm, we merge HS and TLBO together to compensate for each other’s deficiencies, where the HS is mainly used for exploring the search space, and the TLBO aims to speed up the exploitation process. In the search process of HSTLBO, the HS and the TLBO compete for the opportunity in each iteration according to a self-adaptive selection rate (SR) (see Box 1). The selection rate (SR) is dynamically changed in terms of the success times that new generated solution is superior to the worst solution of population during T cycle. In the beginning stage, the HS can obtain more opportunity for exploring the unknown regions, when the region containing the globally optimal solution has been found in the later stage, the TLBO will obtain more opportunity for exploiting high-precision solution. For a new unknown problem, exploring the unknown space in the beginning stage is the first consideration for the HSTLBO algorithm, for which the HS is a good choice. As a consequence, a high selection rate SR (≥ 0.95) is assigned to HS algorithm for exploring unknown regions in the first half of search. In the remaining time, the value of SR will continue to adapt the population status. If the global region has not been found before or the space distribution of population is still extensive, the HS might be still able to obtain a high SR for exploring the unknown areas. However, if the HS obtains no or very low selection rate, (in other words, the TLBO has very high probability), the diversity of population might be lost quickly for high-dimensional optimization problems owing to the quick convergence of TLBO. Therefore, to keep the diversity of population in a certain level, in the second half of stage, the HS is also given more than 0.3 probabilities to run. In the hybrid HSTLBO algorithm, both HS and TLBO are modified as follows.

3.2 Modified HS

In the modified HS algorithm (see Fig 3), all the steps are identical to the steps of standard HS algorithm except for step 3 (improvising a new harmony). The key difference of step 3 between modified HS and standard HS are:
Fig 3

Pseudo code of modified HS algorithm.

Standard HS algorithm produces a new harmony in which each element (decision variable) is generated based on three HS rules (a. harmony memory consideration; b. pitch-adjusting; c. randomly disturbance). In our proposed algorithm, the producing process of new harmony is similar to that of DIHS [19]. Dynamic selection strategy is adopted to select some elements of the worst harmony with probability SP for adjusting. The formula of SP is expressed in Eq (1). The selection probability SP is adjusted dynamically according to current iteration t, which is introduced in DIHS [19]. In the standard HS, parameters PAR and fw are constant values. The modified HSTLBO algorithm adopts dynamic strategies to change PAR and fw for balancing the exploration power and the exploitation power (see Eqs (2) and (3)). The fw is divided into two stages. In the first half of the generation, the fw is dynamically changed with the increasing of iteration, which is the same as the bw in IHS [12]. The overarching goal of fw is to maintain strong exploration power. In the second half of generations, in order to adapt the characteristics of problems, the value of fw is changed adaptively in terms of the values of individuals.

3.3 Modified TLBO

The primary difference between standard TLBO and modified TLBO (see Fig 4) is as follows.
Fig 4

Pseudo code of modified TLBO algorithm.

In standard TLBO, M is equal to the mean value of learners X. In modified TLBO, it is a combination vector in which each subject M(i) is randomly chosen from the ith subject of all learners. In each iteration. TLBO performs teacher phase and learner phase, respectively. The modified TLBO only randomly chooses either teacher phase or learner phase to perform. In standard TLBO, all dimensions of X are produced by learning from teacher or other one learner. Whereas, in modified TLBO, only a portion of dimensions of X are generated by learning from teacher or other one learner, and other dimensions inherit from X directly, which is because an excellent learner is also imperfect on some subjects, selective learning on parts of subjects is more effective for improving knowledge level of learner than learning all subjects from one learner. The selection probability SP is shown in Eq (1), the selection of parameters has been explained and analyzed in our proposed DIHS algorithm [19]. As we known, in our real lives, selective learning from multiple excellent learners on some subjects is more effective for improving our knowledge level than learning all subjects only from one excellent learner. As a consequence, in the modified TLBO, the learner on each subject will select one other learner from population for better learning new knowledge.

4. Experimental study

To investigate the performance of proposed HSTLBO algorithm, numerical simulation experiments on twenty benchmark functions [32-35] are tested. Parameter settings are listed for all compared HS and TLBO variants in Table 1.
Table 1

Algorithm parameter settings.

AlgorithmPopulation sizeHMCRPARfw (bw)
HS100.990.330.01
IHS100.90PARmax = 0.99; PARmax = 0.1fwmax = (xU-xL)/1000; fwmin = 0.0001
ITHS100.99PARmax = 1; PARmax = 0/
NDHS100.99PARmax = 0.99; PARmax = 0.1/
DSHS100.99PARmax = 0.99; PARmax = 0.1/
EHS100.990.33/
HSTLBO100.99PARmax = 0.9; PARmax = 0.1fwmax = (xU-xL)/1000; fwmin = 1e-10
AlgorithmPopulation size
TLBO10/
ATLBO10/
WTLBO10wmin = 0.1,wmax = 0.9
TLBO-GC10/
ITLBO10Pc = 0.8; M = 5; (rearrange) m = 100
Twenty well-known functions are listed in Table 2, which include 16 multimodal functions (F1-F6, F8, F12-F20) and 4 complex uni-modal problems (F7, F9-F11), 4 hybrid functions (F17-F20) and 10 shift functions (F9-F18).
Table 2

Sixteen complex benchmark functions (F1-F20).

Function NameSearch RangeOptimum ValueFunction TypeMulti/Shifted/Separable/Hybrid
F1:Ackley Function[–32,32] DX* = (0,0,⋯,0), F(X*) = 0Y/N/Y/N
F2: Griewank Function[–600,600] DX* = (0,0,⋯,0), F(X*) = 0Y/N/N/N
F3:Levy Function[–10,10] DX* = (1,1,⋯,1), F(X*) = 0Y/N/N/N
F4:Michalewics Function[-10, π] DunknownY/N/N/N
F5:Rastrigin Function[-5.12,5.12]DX* = (0,0,⋯,0), F(X*) = 0Y/N/N/N
F6:Schwefel 2.26 Function[–512,512]DX* = (420.9687,420.9687,⋯,420.9687), F(X*) = 0Y/N/N/N
F7:Rosenbrock Function[–100,100]DX* = (1,1,⋯,1), F(X*) = 0N/N/N/N
F8:Schwefel2.22 Function[–5,5]DX* = (0,0,⋯,0), F(X*) = 0Y/N/N/N
F9:Sphere Shift Function[–100,100]DX* = o, F(X*) = 0N/Y/N/N
F10:Schwefel_Shift Function[–100,100]DX* = o, F(X*) = 0N/Y/N/N
F11:Rosenbrock shift Function[–100,100]DX* = o, F(X*) = 0N/Y/N/N
F12:Griewank Shift Function[–600,600]DX* = o, F(X*) = 0Y/Y/N/N
F13:Rastrigin Shift Function[-5.12,5.12]DX* = o, F(X*) = 0Y/Y/N/N
F14:Ackley Shift Function[–32,32]DX* = o, F(X*) = 0Y/Y/N/N
F15:FastFractal ‘DoubleDip’ unction[–1,1]DUnknownY/Y/N/N
F16: Schaffer Shift function[–100,100]DX* = o, F(X*) = 0Y/Y/N/N
F17: Extended_f10 Shift Function[–100,100]Dx* = o, F21(x*) = 0Y/Y/N/Y
F18: Bohachevsky Shift Function[–15,15]Dx* = o, F22(x*) = 0Y/Y/N/Y
F19: Extended Function[–100,100]Dx* = o, F21(x*) = 0Y/N/N/Y
F20: Bohachevsky Function[–15,15]Dx* = o, F22(x*) = 0Y/N/N/Y
In the simulation experiment, all the test programs were performed on Windows XP 32 system with Intel(R) Core(TM) i3-2120 CPU@3.30 GHz and 4 GB RAM, and all the program codes were written in MATLAB R2014b. In our simulation experiments, the dimension of test functions is set to 1000 and each function is run independently 20 times with 5E+6 function evaluations (FEs) as the termination condition. The precision (=|f(X)−f(X*)|) (Prec), standard deviation of the precision (Std dev) and mean run time (Mtime) for each function are calculated over 20 independent runs, where X is the best solution in population when the terminate condition is meet, and X* is the global optimal solution.

4.1. Comparison with state-of-the-art HS variants

HSTLBO is compared with standard HS algorithm and four state-of-the-art HS variants: IHS [12], ITHS [14], EHS [15], and NGHS [18]. To ensure the comparison fair, for all compared algorithms, each test function is run 20 times independently with 5E+6 FEs as the terminal criterion. The parameters of five HS variants are set the same value as the recommended value of original paper. The experimental results of six compared algorithms are summarized in Table 3. Fig 5 displays the convergence curve and box plots of distribution of optimal solutions after 20 independent runs are displayed in Fig 6.
Table 3

Experimental results of HS, IHS, ITHS, NGHS, EHS and HSTLBO over 20 independent runs on 20 test functions of 1000 variables with 5E+6 FEs.

“Prec” and “Std Dev” denote the precision and standard deviation of the function error values in 20 runs, respectively. Time(s) is the mean run time over 20 independent runs on 5000000 FEs.

ALGPrecStd DevMtime(s)PrecStd DevMtime(s)PrecStd DevMtime(s)PrecStd DevMtime(s)
F1~F4HSTLBO1.34E-134.05E-153.48E+029.33E-166.08E-173.94E+022.18E-261.53E-279.20E+02-9.93E+023.96E-011.18E+03
HS9.34E+001.44E-017.25E+025.13E+021.39E+018.68E+021.35E+026.20E+001.28E+03-2.77E+023.55E+001.63E+03
IHS8.55E+008.71E-021.03E+033.71E+022.75E+011.19E+031.39E+024.96E+001.66E+03-8.37E+025.79E+001.91E+03
ITHS1.62E+001.57E+008.58E+021.45E+012.45E+011.00E+036.96E+017.75E+001.50E+03-8.16E+025.46E+001.81E+03
NGHS1.01E+018.98E-026.47E+026.29E+021.60E+017.51E+022.21E+027.21E+001.31E+03-4.51E+023.25E+001.45E+03
EHS1.33E+011.14E-011.39E+031.64E+035.89E+011.52E+032.33E+031.01E+022.04E+03-2.43E+021.30E+002.22E+03
F5~F8HSTLBO4.37E-121.63E-122.63E+023.91E-091.77E-103.29E+021.89E+038.93E+012.33E+026.09E-211.48E-212.13E+02
HS1.44E+037.50E+006.46E+023.18E+041.73E+037.58E+021.03E+055.88E+035.55E+022.02E+021.62E+005.88E+02
IHS1.57E+033.98E+011.03E+033.17E+041.79E+039.43E+029.35E+041.25E+039.12E+021.37E+023.53E+008.99E+02
ITHS6.39E+004.60E+008.85E+023.79E+043.35E+039.87E+027.21E+041.61E+048.64E+021.66E+011.16E+018.13E+02
NGHS4.70E+037.99E+015.76E+021.33E+051.57E+036.88E+029.72E+047.18E+034.86E+022.98E+025.58E+004.75E+02
EHS1.13E+046.36E+011.35E+033.07E+052.37E+031.22E+033.03E+061.58E+051.18E+033.89E+025.60E+001.13E+03
F9~F12HSTLBO5.55E-243.40E-252.08E+022.81E+015.73E-012.11E+021.32E+031.85E+022.58E+025.37E-153.73E-164.90E+02
HS6.77E+041.24E+035.69E+025.55E+017.77E-015.32E+022.17E+091.68E+086.02E+026.02E+022.32E+019.02E+02
IHS5.39E+041.21E+039.28E+026.05E+019.71E-018.85E+022.14E+091.45E+089.72E+024.70E+021.66E+011.26E+03
ITHS4.38E+051.54E+047.96E+026.52E+015.59E-018.02E+028.48E+104.63E+098.62E+023.84E+031.90E+021.16E+03
NGHS7.70E+044.15E+034.67E+024.40E+014.25E-014.63E+021.63E+091.61E+085.40E+026.97E+026.20E+007.50E+02
EHS1.63E+056.80E+031.16E+031.28E+021.91E+001.13E+035.22E+104.04E+091.17E+031.46E+035.44E+011.41E+03
F13~F16HSTLBO5.90E-141.27E-142.72E+028.11E-132.43E-143.51E+023.02E+029.86E+001.56E+036.63E+023.75E+012.13E+03
HS1.58E+031.94E+016.25E+029.56E+001.23E-017.27E+023.04E+033.33E+011.87E+034.00E+034.54E+012.39E+03
IHS1.75E+033.12E+018.50E+029.15E+002.67E-018.44E+021.91E+032.50E+012.25E+033.32E+038.09E+013.13E+03
ITHS4.53E+031.79E+021.02E+031.58E+011.48E-018.74E+023.03E+037.55E+012.10E+034.78E+031.25E+022.85E+03
NGHS4.93E+031.10E+026.86E+021.01E+011.26E-015.75E+024.20E+033.28E+011.92E+035.29E+037.02E+012.47E+03
EHS1.01E+046.64E+011.20E+031.31E+018.10E-021.07E+038.03E+035.64E+012.77E+036.65E+037.44E+013.45E+03
F17~F20HSTLBO6.53E+022.48E+012.37E+031.02E-151.99E-167.10E+023.94E-106.58E-112.04E+030.00E+000.00E+003.86E+02
HS4.08E+034.39E+012.56E+034.57E+031.80E+021.22E+034.06E+031.16E+022.40E+034.64E+031.22E+029.94E+02
IHS3.31E+037.20E+012.87E+033.56E+031.84E+021.59E+033.33E+035.52E+012.72E+033.46E+031.08E+021.31E+03
ITHS4.73E+037.23E+012.73E+031.41E+043.94E+021.32E+033.85E+022.09E+022.48E+037.33E+025.46E+021.09E+03
NGHS5.38E+035.13E+012.69E+035.54E+032.14E+021.02E+035.33E+031.45E+022.43E+035.28E+032.05E+027.25E+02
EHS6.61E+037.71E+013.45E+031.35E+045.02E+021.81E+036.68E+036.27E+013.21E+031.26E+041.94E+021.46E+03
Fig 5

Convergence curves of four functions.

Fig 6

Box plots of four functions.

Quality of solution. For twenty test functions with dimension of 1000, the results provided by the five HS variants are far away from the global optimal solution. However, it can be found from Table 3 that HSTLBO is obviously superior to other algorithms, and the optimal solutions obtained by the proposed algorithm are very close to the global optimal solutions for all test functions except for F7,F10,F11,F15-F17. CPU runs time. It can be seen from Table 3 that the HSTLBO algorithm takes less run time for all 20 test functions than other five HS variants. Robustness. From Fig 5, the convergence curves of the HSTLBO is much active in whole search process, which demonstrates that the HSTLBO algorithm can maintain the strong search ability during the search process. Nevertheless, the other HS algorithms are easy convergence premature, which demonstrate the algorithms lose the search ability owing to the stagnation of the search. The boxplots (see Fig 6) indicate that optimal solutions of the HSTLBO have a more narrow distribution than those of other algorithms, which illustrates that our method has strong stability and robustness on 20 runs.

Experimental results of HS, IHS, ITHS, NGHS, EHS and HSTLBO over 20 independent runs on 20 test functions of 1000 variables with 5E+6 FEs.

“Prec” and “Std Dev” denote the precision and standard deviation of the function error values in 20 runs, respectively. Time(s) is the mean run time over 20 independent runs on 5000000 FEs.

4.2 Comparison with TLBO variants

In this section, we compare the HSTLBO with standard TLBO and four state-of-the-art TLBO variants: ATLBO [26], WTLBO [27], TLBO_GC [28], and ITLBO [29]. In ATLBO, elitist strategy, weight function and acceleration coefficient are employed to improve the performance. WTLBO introduces a weighted TLBO algorithm for balancing the exploration and the exploitation. In TLBO_GC, a global crossover strategy is proposed for solving global optimization problems. ITLBO adopts a local learning and self-learning methods for improving the global search ability of TLBO. To ensure the comparison fair, for all compared TLBO algorithms, each test function is run 20 times independently with 5E+6 FEs as the terminal criterion. The experimental results of six compared algorithms are summarized in Table 4. Fig 7 displays the convergence curves and Fig 8 shows the distribution of optimal solutions on 20 independent runs for six TLBO algorithms using box plots.
Table 4

Experimental results of TLBO, ATLBO, WTLBO, TLBO_GC, ITLBO and HSTLBO over 20 independent runs on 20 test functions of 1000 variables with 5E+6 FEs.

“Prec” and “Std Dev” denote the precision and standard deviation of the function error values in 20 runs, respectively. Time(s) is the mean run time over 20 independent runs on 5000000 FEs.

ALGPrecStd DevMtime(s)PrecStd DevMtime(s)PrecStd DevMtime(s)PrecStd DevMtime(s)
PrecStd DevMtime(s)1.83E-151.62E+020.00E+000.00E+002.88E+028.53E+011.42E+009.07E+025.70E+021.48E+021.17E+03
ATLBO4.44E-150.00E+001.50E+020.00E+000.00E+002.97E+029.14E+011.19E-028.95E+028.43E+021.36E+011.12E+03
WTLBO8.88E-161.50E-152.63E+020.00E+000.00E+003.31E+028.92E+012.69E-018.88E+028.49E+025.72E+001.01E+03
TLBO_GC7.99E-151.12E-155.51E+020.00E+000.00E+007.09E+029.02E+012.89E-011.25E+037.47E+026.53E+001.53E+03
ITLBO7.99E-150.00E+002.79E+021.11E-164.68E-174.28E+027.59E+012.92E+009.67E+026.04E+029.87E+011.28E+03
HSTLBO1.36E-134.57E-152.52E+029.99E-167.02E-174.53E+022.16E-261.10E-279.81E+026.35E+003.12E-011.19E+03
F5~F8TLBO0.00E+000.00E+002.06E+022.36E+056.06E+042.22E+029.93E+023.51E-011.80E+020.00E+000.00E+001.49E+02
ATLBO0.00E+000.00E+001.80E+021.43E+051.13E+051.90E+029.97E+021.97E+001.46E+020.00E+000.00E+001.21E+02
WTLBO0.00E+000.00E+002.11E+023.75E+058.43E+032.47E+029.97E+022.51E-011.75E+020.00E+000.00E+002.46E+02
TLBO_GC0.00E+000.00E+005.25E+023.66E+056.10E+036.45E+029.99E+021.06E-015.23E+020.00E+000.00E+004.61E+02
ITLBO0.00E+000.00E+002.95E+022.32E+051.45E+043.47E+029.69E+022.71E+002.21E+020.00E+000.00E+002.17E+02
HSTLBO3.64E-121.23E-122.94E+023.70E-091.76E-103.08E+021.94E+031.27E+022.41E+025.73E-218.83E-222.18E+02
F9~F12TLBO1.38E+065.25E+041.40E+029.96E+011.05E-011.47E+024.70E+112.84E+102.20E+021.25E+047.99E+024.88E+02
ATLBO3.38E+061.39E+041.17E+029.98E+016.15E-021.20E+021.28E+125.11E+092.00E+022.99E+043.14E+014.77E+02
WTLBO2.64E+064.16E+041.46E+029.83E+012.20E-011.43E+028.43E+113.98E+102.28E+022.33E+045.80E+024.91E+02
TLBO_GC3.22E+064.21E+044.92E+029.96E+011.10E-014.55E+021.18E+122.09E+105.92E+022.85E+043.48E+027.93E+02
ITLBO1.07E+046.91E+032.16E+029.91E+013.10E-012.32E+023.17E+063.35E+072.55E+021.21E+026.68E+015.31E+02
HSTLBO5.28E-244.72E-251.98E+022.84E+015.99E-012.18E+021.41E+031.13E+022.74E+025.33E-155.73E-164.59E+02
F13~F16TLBO1.23E+041.99E+022.83E+022.02E+016.65E-022.96E+027.28E+037.54E+021.48E+037.36E+033.94E+021.92E+03
ATLBO1.81E+041.86E+022.33E+022.11E+011.33E-022.45E+028.69E+034.70E+011.39E+038.99E+033.27E+011.90E+03
WTLBO1.66E+041.51E+022.92E+022.10E+014.18E-023.26E+028.33E+032.89E+011.44E+038.72E+034.01E+011.94E+03
TLBO_GC1.67E+042.61E+026.83E+022.09E+013.21E-027.45E+027.75E+039.05E+011.85E+038.66E+038.00E+012.25E+03
ITLBO8.31E+031.21E+023.56E+021.94E+011.01E-024.18E+025.76E+034.16E+021.64E+036.75E+032.02E+021.97E+03
HSTLBO6.31E-149.17E-153.47E+027.78E-132.62E-143.45E+022.98E+021.71E+011.61E+035.50E+032.07E+011.97E+03
F17~F20TLBO7.30E+034.75E+012.28E+032.27E+049.10E+027.06E+020.00E+000.00E+007.21E+020.00E+000.00E+002.15E+02
ATLBO9.01E+033.79E+012.50E+034.82E+042.51E+027.13E+020.00E+000.00E+009.91E+020.00E+000.00E+002.01E+02
WTLBO8.58E+034.39E+012.53E+034.05E+043.56E+027.51E+020.00E+000.00E+001.02E+030.00E+000.00E+002.40E+02
TLBO_GC8.57E+036.78E+012.89E+034.62E+045.38E+021.07E+030.00E+000.00E+001.02E+030.00E+000.00E+005.42E+02
ITLBO6.27E+035.85E+012.56E+036.48E+022.13E+026.75E+020.00E+000.00E+008.86E+020.00E+000.00E+002.65E+02
HSTLBO6.64E+021.88E+012.20E+039.99E-163.92E-156.46E+023.91E-108.16E-111.91E+030.00E+000.00E+003.41E+02
Fig 7

Convergence curves of four functions.

Fig 8

Box plots of four functions.

Quality of solution. In twenty test functions, our method is the winner on 13 functions and is very close to the winner algorithm on other seven functions. Especially, for the complex multimodal functions, such as F3, F6, F9, F13-F15 and F18, the precision of proposed algorithm is much better than other five TLBO variants. For functions F1, F2, F5, F7, F8 and F19, the optimal solution of our method is worse than those of other algorithms, however, the solutions of our method are on the verge of global optimal solutions, which are acceptable to the application. Robustness. It can be seen from Fig 7 that, comparing with five TLBO algorithms, the convergence curves of the HSTLBO is also much more active in the search process than those of other TLBO algorithms, and the HSTLBO has strong exploitation power (the convergence curve keeps decreasing) in the later stage of search process, which demonstrates that the HSTLBO can keep diverse distribution of population during the search process. From the boxplots (see Fig 8), we can find the HSTLBO is more stable on obtaining the optimal solution.

Experimental results of TLBO, ATLBO, WTLBO, TLBO_GC, ITLBO and HSTLBO over 20 independent runs on 20 test functions of 1000 variables with 5E+6 FEs.

“Prec” and “Std Dev” denote the precision and standard deviation of the function error values in 20 runs, respectively. Time(s) is the mean run time over 20 independent runs on 5000000 FEs.

4.3 Statistical test

To investigate the significant difference between our method and ten compared algorithms, in this section, the Wilcoxon signed rank test is conducted at 5% significance level to judge whether the optimal solutions on 20 independent runs with our method differ significantly from those of compared algorithms. Table 5 records the corresponding p-values for 20 functions between the proposed algorithm and other algorithms, which indicate that all the p-values are less than 0.05. The last three lines of Table 5 records the performance of algorithms, where “+”, “=“, and “-” separately represent the optimal solutions on 20 independent runs of the corresponding algorithm are better than, similar to, and worse than those of HSTLBO. We can see from Table 5 that the performance of proposed algorithm is significantly different with those of other algorithms for each function, and our method is superior to other algorithms for most of functions.
Table 5

Results of Wilcoxon’s rank sum test at 0.05 significance level between HSTLBO and other ten algorithms.

The p-value is shown (NaN denotes no difference).

FunctionsHSIHSITHSNGHSEHSTLBOATLBOWTLBOTLBO_GCITLBO
F11.2E-081.2E-082.9E-041.2E-081.2E-081.0E-096.0E-116.0E-116.0E-116.0E-11
F21.0E-081.0E-082.7E-041.0E-081.0E-081.0E-091.0E-091.0E-091.0E-091.0E-09
F31.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F41.2E-081.2E-081.2E-081.2E-081.2E-086.0E-116.0E-116.0E-116.0E-116.0E-11
F51.0E-081.0E-081.0E-081.0E-081.0E-085.99E-115.99E-115.99E-115.99E-115.99E-11
F61.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F71.2E-081.2E-081.2E-081.2E-081.2E-081.2E-091.2E-091.2E-091.2E-091.2E-09
F81.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F91.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F101.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-082.9E-04
F111.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F121.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F131.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F141.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F151.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F161.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F171.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F181.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-081.2E-08
F191.2E-081.2E-081.2E-081.2E-081.2E-081.2E-091.2E-091.2E-091.2E-091.2E-09
F201.2E-081.2E-081.2E-081.2E-081.2E-08NaNNaNNaNNaNNaN
+0000055555
=0000011111
-20202020201414141414

Results of Wilcoxon’s rank sum test at 0.05 significance level between HSTLBO and other ten algorithms.

The p-value is shown (NaN denotes no difference). To further detect the significant differences between HSTLBO and ten compared algorithms, the multiple-problem Wilcoxon’s test is employed to check the comparisons. Table 6 records the statistical results, where “W+” is the number of cases in which the null hypothesis was rejected and our method shows a statistically superior performance at the 95% significance level, “W-” denotes the number of cases in which the null hypothesis was rejected and the HSTLBO displays an inferior performance, “W=“ represents the number of cases in which the null hypothesis was accepted [36] [37]. We can find from Table 6 that our method has higher “W+” values than “W-” values in all cases, which demonstrates that our method is significantly better than other ten algorithms on 20 test functions.
Table 6

Multi-problem based statistical pairwise comparison of HSTLBO and other ten algorithms.

(α = 0.05, D = 1000).

HSTLBO vs. AlgorithmD = 1000
P-valueW+W-W=Winner
HSTLBO vs. HS8.86E-0521000HSTLBO
HSTLBO vs. IHS8.86E-0521000HSTLBO
HSTLBO vs. ITHS8.86E-0521000HSTLBO
HSTLBO vs. NGHS8.86E-0521000HSTLBO
HSTLBO vs. EHS8.86E-0521000HSTLBO
HSTLBO vs. TLBO0.0048481207020HSTLBO
HSTLBO vs. ATLBO0.0048481207020HSTLBO
HSTLBO vs. WTLBO0.0048481207020HSTLBO
HSTLBO vs. TLBO_GC0.0048481207020HSTLBO
HSTLBO vs. ITLBO0.006211167420HSTLBO

Multi-problem based statistical pairwise comparison of HSTLBO and other ten algorithms.

(α = 0.05, D = 1000).

4.4 Analysis of exploration and exploitation

In this section, we investigate the convergence of HSTLBO by tracing population diversity in the search process. The population diversity is defined as follows, Where denotes the mean value of ith decision variable in population. Three complex multimodal optimization problems (F1:Ackley, F3:Levy, F12:Griewank Shift) are employed to investigate the balance between the exploration power and the exploitation. During the search process, we trace the changes of population diversity of each compared algorithm. The Fig 9 displays the curve changes of population diversity of eleven algorithms, from which we can easily find that the diversity curves of HSTLBO algorithm decrease gradually during the search process, which are more asymptotic and stable than other algorithms. In this way, the HSTLBO possess strong exploration power in the early stage of search for exploring the unknown search regions, with the search continue, the exploitation power increases and the exploration power decreases gradually, in the later stage, the population has gathered into the globally optimal region. And by this time HSTLBO should have obtained much high exploitation power for exploiting the high precision solution. However, compared with HSTLBO, five HS variations (HS, IHS, ITHS, EHS and NGHS) keep high population diversity from beginning to end, which make them have strong exploration power but exploitation power very weak. Conversely, the diversity of five TLBO variations decrease very quickly in the begin stage, which makes them easily be trapped into local search owing to losing the exploration power prematurely.
Fig 9

Curves of population diversity of 11 algorithms.

5. HSTLBO for solving complex portfolio optimization problem

To further investigate the performance of HSTLBO algorithm, complex portfolio optimization problem are employed to test the ability of solving real-world application. The portfolio optimization aims to choose the optimal proportions of various assets for obtaining maximum portfolio return with minimum risk. In this work, we apply HSTLBO algorithm to choose the optimal portfolio proportions for Nikkei 225 stock index that maps companies on the Tokyo Stock Exchange (TSE) (http://en.wikipedia.org/wiki/Nikkei_225) and compare the test results of HSTLBO with four intelligent algorithms (GA, PSO, TS, SA)[38-40]. We employ mean Euclidian distance (MED), variance of returns error (VRE) and mean return error (MRE) as performance indexes that are defined in literatures [39,41]. The test data is from http://people.brunel.ac.uk/~mastjjb/jeb/orlib/portinfo.html and the experiments are performed on two conditions: Unconstraint. The portfolio proportion and desired number of investment assets are not constrained. Constraint. The portfolio proportion of each asset and ; the desired number of portfolio selection assets K = 10. In this work, we employ the same method of constraint handle as the literature [41] which can handle the boundary constraint of the portfolio proportion and the desired number of portfolio selection assets very well. The test results on three evaluation indexes (MED, VRE and MRE) are presented in Table 7, and The comparison of efficient frontiers for different constraint conditions are shown in Fig 10. We can see easily from Table 7 that our method obtains more outstanding performance on MED, VRE and MRE than modified GA, PSO, TS and SA. From Fig 10, we can find that optimal frontiers of our method are almost overlapped with standard efficient frontiers for unconstraint CCMV model, and for constraint CCMV model, the optimal frontiers of our algorithm are also very close to the standard efficient frontiers that are obtained without considering the constraint conditions. Therefore, our approach is effective for solving complex portfolio optimization problems.
Table 7

Simulation results of five algorithms on Nikkei index 225.

UnconstraintConstraint
GAPSOTSSAHSTLBOGAPSOTSSAHSTLBO
MED1.50E-032.90E-041.50E-041.90E-048.33E-079.93E-031.90E-031.00E-031.23E-036.63E-05
VRE2.10E-014.30E-012.20E-012.10E-016.36E-021.21E+002.43E+001.24E+001.20E+005.23E+00
MRE9.30E-011.40E-017.40E-027.20E-021.34E-025.33E+008.00E-014.21E-014.13E-011.35E+00
Fig 10

Optimal frontiers for unconstraint and constraint portfolio Nikkei index 225.

6. Conclusion

Both Harmony Search and Teaching-Learning-Based Optimization are new swarm intelligent optimization algorithms, which have got much attention in recent years. In this work, in order to improve the performance of HS and TLBO, a hybrid HSTLBO algorithm is presented. In the HSTLBO, both HS and TLBO are improved to enhance the global search ability. A self-adaptive selection strategy is presented to balance the exploration power and the exploitation power. At the early stage of search process, the HS algorithm gets a higher opportunity than TLBO, which aims to explore the unknown regions and avoid lose the globally optimal solution. With the increasing number of iteration, the opportunity of TLBO is raised step by step. At the later stage of search, when the population has gathered into one region which maybe contains the global optimal solution, the TLBO has obtained much more opportunity for exploiting high precision optimal solution. The experimental results also demonstrate that our method is a promising optimization algorithm in solving large scale and complex optimization problems.
  1 in total

1.  Exploratory power of the harmony search algorithm: analysis and improvements for global numerical optimization.

Authors:  Swagatam Das; Arpan Mukhopadhyay; Anwit Roy; Ajith Abraham; Bijaya K Panigrahi
Journal:  IEEE Trans Syst Man Cybern B Cybern       Date:  2010-04-26
  1 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.