A modified nonmonotone BFGS algorithm for unconstrained optimization.

In this paper, a modified BFGS algorithm is proposed for unconstrained optimization. The proposed algorithm has the following properties: (i) a nonmonotone line search technique is used to obtain the step size [Formula: see text] to improve the effectiveness of the algorithm; (ii) the algorithm possesses not only global convergence but also superlinear convergence for generally convex functions; (iii) the algorithm produces better numerical results than those of the normal BFGS method.

Introduction

Consider where is continuously differentiable. Many similar problems can be transformed into the above optimization problem (see [1-16] etc.). The following iteration formula is used to address the iteration point of (1.1): where is the kth iterative point, is the step length, and is the search direction of f at . The search direction determines the line search method (see [17-25]). The quasi-Newton method is defined by where , is the quasi-Newton update matrix, and the sequence satisfies the so-called quasi-Newton equation where , , and . The following update of : is the BFGS formula (Broyden [26], Fletcher [27], Goldfar [28], and Shanno [29]), which is one of the most effective quasi-Newton methods. Convex functions can be combined with exact line or certain special inexact line search techniques that have global convergence (see [30-32] etc.) and superlinear convergence (see [33, 34] etc.). For general functions, under inexact line search techniques, Dai [35] constructed an example to show that the BFGS method fails. Mascarenhas [36] proved the nonconvergence of this method, even with the exact line search technique. To obtain global convergence of a BFGS method without the convexity assumption, Li and Fukushima [37, 38] proposed the following modified BFGS methods.

Formula 1

[37]

The BFGS update formula is defined by where and function satisfies: (i) for all ; (ii) if and only if ; (iii) if t is in a bounded set, and is bounded. Using the definition of , it is not difficult to obtain This is sufficient to guarantee the positive definiteness of as long as is positive definite. Li and Fukashima presented with some constant .

Formula 2

[38]

The BFGS update formula is defined by where , ϕ and the properties are the same as those in Formula 1. For nonconvex functions, these two methods possess global convergence and superlinear convergence.

Some scholars have conducted further research to obtain a better approximation of the Hessian matrix of the objective function.

Formula 3

[39]

The BFGS update formula is defined by where and . It is easy to conclude that this formula contains both gradient and function value information. One may believe that the resulting methods will outperform the normal BFGS method. In fact, the practical computation shows that the method is better than the normal BFGS method and that it has some theoretical advantages (see [39, 40]). Under the WWP line search, Wei et al. [39] proposed the quasi-Newton method and established its superlinear convergence for uniformly convex functions. Its global convergence can be found in [40], but the method fails for general convex functions. One of the main reasons for the failure is the non-positive definiteness of matrix for general convex functions. Byrd et al. [31, 32] showed that the positive definiteness of matrix plays an important role in the convergence of the quasi-Newton algorithm. Yuan and Wei [41] first analyzed the global convergence and superlinear convergence of the modified BFGS formula in [39] using gradient and function value information for general convex functions. Based on equation (1.9), Yuan and Wei [41] proposed another BFGS formula.

Formula 4

[41]

The BFGS update formula is defined by where . This modified method obtains global convergence and superlinear convergence for generally convex functions. The same work was previously performed by Zhang et al. [42].

Formula 5

[42]

The BFGS update formula is defined by where , . It is clear that the quasi-Newton equation (1.10) also contains both gradient and function value information, and it has been proved that the new formula has a higher order approximation to . Furthermore, Yuan et al. [43] extended a similar technique to in a limited memory BFGS method, where global convergence is only obtained for uniformly convex functions. Several other modified quasi-Newton methods have been reported (see [23, 40, 44, 45]).

The monotone line search technique is often used to determine the step size . One famous technique is the weak Wolfe-Powell (WWP) technique.

WWP line search technique. is determined by where . Recently, a modified WWP line search technique was proposed by Yuan, Wei, and Lu [46] to ensure that the BFGS and the PRP methods have global convergence for nonconvex functions; these two open problems have been solved. However, monotonicity may generate a series of extremely small steps if the contours of the objective functions are a family of curves with large curvature [47]. Nonmonotonic line search to solve unconstrained optimization was proposed by Grippo et al. in [47-49] and was further studied by [50]. Grippo, Lamparillo, and Lucidi [47] proposed the following nonmonotone line search and called it GLL line search.

GLL nonmonotone line search. is determined by where ,  , , , is a nonnegative integer. By combining this line search with the normal BFGS formula, Han and Liu [51] established the global convergence of the convex objective function; its superlinear convergence was established by Yuan and Wei [52]. Although these nonmonotone techniques perform well in many cases, the numerical performance is dependent on the choice of to some extent (see [47, 53, 54] in detail). Zhang and Hager [55] presented another nonmonotone line search technique.

Zhang and Hager nonmonotone line search technique [55]. In this technique is found by where , , and . It is easy to conclude that is a convex combination of and . The numerical results show that this technique is more competitive than the nonmonotone method of [47], but it requires strong assumption conditions for convergence analysis.

Motivated by the above observations, we study the modified BFGS-type method of Yuan et al. [43] based on the formula (1.10). The modified BFGS-type method and the proposed algorithm have the following characteristics:

The GLL line search technique is used in the algorithm to ensure good convergence.

The major contribution of the new algorithm is an extension of the modified BFGS update from [43] and [42].

Another contribution is the proof of global convergence for generally convex functions.

The major aim of the proposed method is to establish the superlinear convergence and the global convergence for generally convex functions.

The experimental problems, including both normal unconstrained optimization and engineering problems (benchmark problems), indicate that the proposed algorithm is competitive with the normal method.

This paper is organized as follows. In the next section, we present the algorithm. The global convergence and superlinear convergence are established in Section 3 and Section 4, respectively. Numerical results are reported in Section 5. In the final section, we present a conclusion. Throughout this paper, denotes the Euclidean norm of a vector or matrix.

Algorithm

In this paper, we study the modified formula of [43] and obtain global convergence and superlinear convergence under generally convex conditions. The modified BFGS update of (1.10) is presented as where , . The corresponding quasi-Newton equation is By the definition of the convex property of f, holds (see [43] in detail). Therefore, the update matrix from (2.1) inherits the positive definiteness of for generally convex functions. Now, we state the algorithm as follows.

Algorithm 1

Mod-non-BFGS-A

Given a symmetric and positive definite matrix and an integer , choose an initial point , , , ; Set .

, stop; Otherwise, go to the next step.

Solve to obtain .

The step length is determined by GLL (1.12) and (1.13).

Let .

Generate from (2.1) and set ; Go to Step 1.

Global convergence

The following assumptions are required to obtain the global convergence of Algorithm 1.

Assumption A

The level set is bounded.

The objective function f is continuously differentiable and convex on . Moreover, there exists a constant satisfying

Assumption A implies that there exist constants and satisfying and

Lemma 3.1

Suppose Assumption A holds. Then there exists a constant such that

The proof is similar to [41], so it is not presented here.

Lemma 3.2

Let be updated by (2.1); then the relation holds, where denotes the determinant of .

Lemma 3.3

Assume that Assumption A holds and that sequence is generated by Algorithm 1. If then there exists a constant satisfying where .

Proof

For , by the positive definiteness of , we have . Then is generated by (2.1), and is positive definite. Assume that is positive definite; for all , we prove that holds by the following three cases.

Case 1: . The definition of , the convexity of , and Assumption A generate

Case 2: . By (1.13), (2.3), Assumption A, the definition of , and the positive definiteness of , we get where .

Case 3: . The proof can be found in [41]

Similar to the proof of Theorem 3.1 in [51], we can establish the global convergence theorem of Algorithm 1. Here, we state the theorem but omit the proof. □

Theorem 3.1

Let the conditions of Lemma 3.3 hold; then we have

Superlinear convergence analysis

Based on Theorem 3.1, we suppose that is the limit of the sequence . To establish the superlinear convergence of Algorithm 1, the following additional assumption is needed.

Assumption B

with . is positive definite and Hölder continuous at , namely, for all x in the neighborhood of , there exist constants and satisfying where .

In a way similar to [41], we can obtain the superlinear convergence of Algorithm 1, which we state as follows but we omit its proof.

Theorem 4.1

Let Assumption A and B hold and be generated by Algorithm 1. Then the sequence superlinearly tends to .

This section reports the numerical results of Algorithm 1. All code was written in MATLAB 7.0 and run on a PC with a 2.60 GHz CPU processor, 256 MB memory and the Windows XP operating system. The parameters are chosen as , , , , , , , and the initial matrix is the unit matrix. Since the line search cannot ensure the descent condition , an uphill search direction may occur in the numerical experiments. In this case, the line search rule may fail. To avoid this case, the step size is accepted if the search number is greater than 25 in the line search. The following is the Himmeblau stop rule: If , let ; otherwise, let . In the experiment, if or satisfies , we end the program.

[57] problems

It has been proved that [57] problems with initial points are an effective tool to estimate the performance of algorithms and are one of the most commonly used sets of optimization problems. Many scholars use these problems to assess their algorithms (see [23, 40, 42, 51]). In this paper, we also perform experiments on these problems. The detailed numerical results are listed in Table 1, where the columns of Table 1 have the following meaning:

Table 1

Numerical results

Problem	Dim	BFGS-WP NI/NFG/Time	BFGS-WP-Zhang NI/NFG/Time	BFGS-Non NI/NFG/Time	BFGS-M-Non NI/NFG/Time
ROSE	2	35/590/4.506480e−002	31/611/4.882020e−002	2/19/6.259000e−003	2/19/6.259000e−003
FROTH	2	9/116/1.376980e−002	7/90/1.001440e−002	2/19/6.259000e−003	2/19/7.510800e−003
BADSCP	2	43/706/5.507920e−002	43/706/5.507920e−002	8/264/2.753960e−002	8/264/2.753960e−002
BADSCB	2	3/60/1.126620e−002	3/60/1.001440e−002	3/32/7.510800e−003	3/32/6.259000e−003
BEALE	2	15/220/2.128060e−002	16/226/2.002880e−002	2/19/6.259000e−003	2/19/6.259000e−003
JENSAM	2	2/42/1.126620e−002	2/42/1.001440e−002	2/19/6.259000e−003	2/19/8.762600e−003
HELIX	3	34/483/4.381300e−002	23/325/3.004320e−002	169/2,191/2.090506e−001	87/1,163/1.114102e−001
BARD	3	16/229/3.004320e−002	14/182/2.503600e−002	72/930/1.226764e−001	72/930/1.226764e−001
GAUSS	3	2/19/6.259000e−003	2/19/6.259000e−003	2/19/7.510800e−003	2/19/7.510800e−003
MEYER	3	2/42/1.376980e−002	2/42/1.251800e−002	2/32/1.126620e−002	2/32/1.251800e−002
GULF	3	2/42/1.502160e−002	2/42/1.502160e−002	2/19/3.755400e−003	2/19/1.001440e−002
BOX	3	2/42/1.251800e−002	2/42/1.126620e−002	2/19/7.510800e−003	2/19/8.762600e−003
SING	4	20/280/2.503600e−002	18/269/2.503600e−002	2/19/6.259000e−003	2/19/7.510800e−003
WOOD	4	19/271/2.628780e−002	20/289/2.753960e−002	2/19/6.259000e−003	2/19/6.259000e−003
KOWOSB	4	21/295/3.505040e−002	23/324/3.630220e−002	83/1,077/1.314390e−001	104/1,345/1.664894e−001
BD	4	17/244/3.505040e−002	19/276/3.880580e−002	2/19/7.510800e−003	2/19/1.001440e−002
OSB1	5	2/42/2.128060e−002	2/42/1.877700e−002	2/19/7.510800e−003	2/19/1.001440e−002
BIGGS	6	25/322/4.506480e−002	7/108/2.253240e−002	15/330/4.381300e−002	21/287/4.130940e−002
OSB2	11	3/56/6.259000e−002	3/56/6.259000e−002	3/33/1.877700e−002	3/33/2.002880e−002
WATSON	20	31/457/3.880580e−001	29/412/3.555112e−001	2/19/2.002880e−002	2/19/2.253240e−002
ROSEX	100	229/3,704/1.268073e+000	276/4,359/1.512174e+000	2/19/1.126620e−002	2/19/1.251800e−002
SINGX	400	65/922/1.174939e+001	155/2,375/2.844465e+001	2/19/2.065470e−001	2/19/2.115542e−001
PEN1	400	2/47/7.247922e−001	2/47/7.310512e−001	2/19/1.940290e−001	2/19/1.927772e−001
PEN2	200	2/25/6.884900e−002	2/25/6.634540e−002	2/19/6.008640e−002	2/19/6.384180e−002
VARDIM	100	2/47/2.879140e−002	2/47/2.879140e−002	2/19/1.001440e−002	2/19/8.762600e−003
TRIG	500	9/138/1.627340e+002	9/144/1.671604e+002	8/146/1.700345e+002	50/876/1.039274e+003
BV	500	2/19/3.492522e−001	2/19/3.492522e−001	2/19/3.480004e−001	2/19/3.517558e−001
IE	500	6/71/7.711088e+000	6/71/7.706081e+000	6/71/7.722354e+000	6/71/7.772426e+000
TRID	500	53/760/1.622333e+001	50/727/1.501159e+001	564/7,325/1.690631e+002	564/7,325/1.692333e+002
BAND	500	12/275/5.551733e+000	12/238/4.696754e+000	2/19/4.781876e−001	2/19/4.431372e−001
LIN	500	2/19/4.719286e−001	2/19/4.744322e−001	2/19/4.806912e−001	2/19/4.719286e−001
LIN1	500	3/32/9.363464e−001	3/32/9.388500e−001	3/31/9.050514e−001	3/31/9.025478e−001
LIN0	500	3/32/1.165426e+000	3/32/1.161670e+000	3/31/1.119109e+000	3/31/1.130375e+000

the name of the test problem;

the dimensions of the problem;

the total number of iterations;

the cpu time in seconds;

, where NF and NG are the total number of function and gradient evaluations, respectively (see [47]).

In Table 1, ‘BFGS-WP’, ‘BFGS-Non’, ‘BFGS-WP-Zhang’, and ‘BFGS-M-Non’ stand for the normal BFGS formula with WWP rule, the normal BFGS formula with GLL rule, the modified BFGS equation (1.10) with WWP rule, and MN-BFGS-A, respectively. The numerical results in Table 1 indicate that the proposed method is competitive with the other three similar methods.

To directly illustrate the performance of these methods, we utilize the tool of Dolan and Moré [58] to analyze their efficiency. Figures 1, 2, and 3 show that the performance is related to NI, NFG, and Time, respectively. According to these three figures, the MN-BFGS-A method has the best performance (the highest probability of being the optimal solver).

Figure 1

Performance profiles of these methods (NI).

Figure 2

Performance profiles of these methods (NFG).

Figure 3

Performance profiles of these methods (Time).

Figure 1 shows that BFGS-M-Non and BFGS-Non outperform BFGS-WP and BFGS-WP-Zhang on approximately 9% and 6% of the problems, respectively. The BFGS-WP-Zhang and BFGS-WP methods can successfully solve 94% and 91% of the test problems, respectively.

Figure 2 shows that BFGS-M-Non and BFGS-Non are superior to BFGS-WP and BFGS-WP-Zhang on approximately 12% and 9% of these problems, respectively. The BFGS-M-Non and BFGS-Non methods solve 100% of the test problems at . The BFGS-WP-Zhang and the BFGS-WP methods solve the test problems with probabilities of 91% and 88%, respectively.

Figure 3 shows that the success rates when using the BFGS-M-Non and BFGS-Non methods to address the test problems are higher than the success rates when using BFGS-WP and BFGS-WP-Zhang by approximately 6% and 9%, respectively. Additionally, the BFGS-M-Non and BFGS-Non algorithms can address almost all the test problems. Moreover, BFGS-WP-Zhang has better results than BFGS-WP.

Benchmark problems

The benchmark problems listed in Table 2 are widely applied in various practical engineering situations. A function is multimodal if it has two or more local optima. A function p of the responding variables is separable provided that it can be rewritten as a sum of p functions of just one variable [59]. Separability is closely related to the concept of epistasis or interrelation among the variables of a function. Non-separable functions are more difficult to optimize because the accuracy of the searching direction depends on two or more variables. By contrast, separable functions can be optimized for each variable in turn. The problem is even more difficult if the function is multimodal. The search process must be able to avoid the regions around local minima in order to approximate, as closely as possible, the global optimum. The most complex case appears when the local optima are randomly distributed in the search space.

Table 2

Definition of the benchmark problems and their features

Function	Definition	Multimodal?	Separable?	Regular?
Sphere	f_Sph(x)=∑i=1pxi2	no	yes	n/a
	x_i∈[−5.12,5.12], x^∗=(0,0,…,0), f_Sph(x^∗)=0.
Schwefel’s	f_SchDS(x)=∑i=1p(∑j=1ix_j)^2	no	no	n/a
	x_i∈[−65.536,65.536], x^∗=(0,0,…,0), f_SchDS(x^∗)=0.
Griewank	f_Gri(x)=1+∑i=1pxi24,000−∏i=1pcosx_ii	yes	no	yes
	x_i∈[−600,600], x^∗=(0,0,…,0), f_Gri(x^∗)=0.
Rosenbrock	f_Ros(x)=∑i=1p[100(x_i+1−xi2)^2+(x_i−1)^2]	no	no	n/a
	x_i∈[−2.048,2.048], x^∗=(1,1,…,1), f_Ros(x^∗)=0.
Ackley	f_Ack(x)=20+e−20e^−0.21p∑i=1pxi2−e^1p∑i=1pcos(2πx_i)	yes	no	yes
	x_i∈[−30,30], x^∗=(0,0,…,0), f_Ack(x^∗)=0.

The dimensionality of the search space is another important factor in the complexity of the problem. A study of the dimensionality problem and its features was conducted by Friedman [60]. To establish the same degree of difficulty in all cases, a search space of dimensionality is chosen for all the functions. In the experiment, we do not fix the value to , namely, it can be larger than 30. The exact dimensions can be found in Table 3.

Table 3

Numerical results of the benchmark problems

Problem/ x_0	Dim	BFGS-WP NI/NFG/Time	BFGS-WP-Zhang NI/NFG/Time	BFGS-Non NI/NFG/Time	BFGS-M-Non NI/NFG/Time
Sphere/x_Sph10	30	2/19/1.562500e−001	2/19/1.562500e−002	2/19/4.687500e−002	2/19/4.687500e−002
	500	2/19/2.031250e−001	2/19/3.125000e−001	2/19/2.656250e−001	2/19/2.187500e−001
	1,000	2/19/1.015625e+000	2/19/1.093750e+000	2/19/1.062500e+000	2/19/1.046875e+000
Sphere/x_Sph20	30	2/19/0	2/19/0	2/19/0	2/19/0
	500	2/19/1.875000e−001	2/19/2.500000e−001	2/19/2.187500e−001	2/19/1.875000e−001
	1,000	2/19/9.531250e−001	2/19/1.046875e+000	2/19/1.031250e+000	2/19/1.218750e+000
Sphere/x_Sph30	30	2/19/0	2/19/0	2/19/0	2/19/0
	500	2/19/2.031250e−001	2/19/2.812500e−001	2/19/2.343750e−001	2/19/1.718750e−001
	1,000	2/19/1.015625e+000	2/19/9.687500e−001	2/19/9.531250e−001	2/19/9.843750e−001
Sphere/x_Sph40	30	2/19/0	2/19/0	2/19/0	2/19/0
	500	2/19/1.718750e−001	2/19/2.343750e−001	2/19/2.187500e−001	2/19/1.250000e−001
	1,000	2/19/9.218750e−001	2/19/1	2/19/1	2/19/1.015625e+000
Schwefel’s/x_SchDs10	30	3/32/0	3/32/6.250000e−002	3/32/6.250000e−002	3/32/0
	50	3/32/0	3/32/0	3/32/6.250000e−002	3/32/6.250000e−002
	100	4/45/1.562500e−001	4/45/2.500000e−001	6/70/3.750000e−001	6/70/4.062500e−001
Schwefel’s/x_SchDs20	30	2/19/6.250000e−002	2/19/0	2/19/0	2/19/0
	50	2/19/0	2/19/6.250000e−002	2/19/0	2/19/0
	100	3/32/1.875000e−001	3/32/1.250000e−001	3/32/1.875000e−001	3/32/1.718750e−001
Schwefel’s/x_SchDs30	30	3/32/0	3/32/6.250000e−002	3/32/0	3/32/0
	50	3/32/6.250000e−002	3/32/0	3/32/0	3/32/6.250000e−002
	100	3/32/1.875000e−001	3/32/1.250000e−001	3/32/1.875000e−001	3/32/1.250000e−001
Schwefel’s/x_SchDs40	30	2/19/0	2/19/0	2/19/0	2/19/0
	50	2/19/0	2/19/6.250000e−002	2/19/0	2/19/0
	100	2/19/6.250000e−002	2/19/6.250000e−002	2/19/1.250000e−001	2/19/6.250000e−002
Griewank/x_Gri10	30	3/37/0	3/37/0	11/258/6.250000e−002	9/130/6.250000e−002
	500	2/24/5.781250e−001	2/24/5.312500e−001	2/24/5.781250e−001	2/24/6.406250e−001
	1,000	2/24/1.984375e+000	2/24/1.656250e+000	2/24/1.671875e+000	2/24/1.625000e+000
Griewank/x_Gri20	30	4/75/0	4/75/4.687500e−002	4/59/0	4/58/0
	500	2/24/6.718750e−001	2/24/3.437500e−001	2/24/4.062500e−001	2/24/6.562500e−001
	1,000	2/24/1.765625e+000	2/24/1.796875e+000	2/24/1.859375e+000	2/24/1.640625e+000
Griewank/x_Gri30	30	3/38/0	3/37/4.687500e−002	11/394/1.250000e−001	9/178/0
	500	2/24/5.625000e−001	2/24/5.468750e−001	2/24/5.625000e−001	2/24/5.781250e−001
	1,000	2/24/2.046875e+000	2/24/1.531250e+000	2/24/1.468750e+000	2/24/1.421875e+000
Griewank/x_Gri40	30	15/200/6.250000e−002	19/249/6.250000e−002	9/502/6.250000e−002	18/446/1.250000e−001
	500	2/24/6.093750e−001	2/24/2.968750e−001	2/24/5.468750e−001	2/24/5.468750e−001
	1,000	2/24/1.843750e+000	2/24/1.468750e+000	2/24/1.828125e+000	2/24/1.781250e+000
Rosenbrock/x_Ros10	30	34/483/1.406250e−001	5/116/0	2/19/0	2/19/0
	500	30/419/3.431250e+001	5/116/2.031250e+000	2/19/2.187500e−001	2/19/1.875000e−001
	1,000	28/393/2.136875e+002	6/152/2.207813e+001	2/19/1.078125e+000	2/19/9.375000e−001
Rosenbrock/x_Ros20	30	30/467/9.375000e−002	5/121/0	2/19/0	2/19/0
	500	16/268/1.650000e+001	3/38/6.250000e−001	2/19/1.875000e−001	2/19/2.187500e−001
	1,000	17/286/1.181094e+002	3/38/3.453125e+000	2/19/1.062500e+000	2/19/9.062500e−001
Rosenbrock/x_Ros30	30	8/134/0	7/141/0	2/19/0	2/19/0
	500	9/154/6.828125e+000	6/110/3.546875e+000	2/19/2.031250e−001	2/19/2.187500e−001
	1,000	7/115/3.090625e+001	5/92/1.373438e+001	2/19/1.125000e+000	2/19/1.156250e+000
Rosenbrock/x_Ros40	30	8/140/0	5/102/0	2/19/6.250000e−002	2/19/0
	500	12/186/1.185938e+001	6/105/5.203125e+000	2/19/2.343750e−001	2/19/2.031250e−001
	1,000	15/226/101	6/105/2.275000e+001	2/19/1.062500e+000	2/19/1.015625e+000
Ackley/x_Ack10	30	5/68/6.250000e−002	6/80/0	6/83/0	6/80/0
	500	5/67/2.343750e+000	5/64/1.937500e+000	5/67/2.046875e+000	5/68/2.171875e+000
	1,000	5/66/1.407813e+001	6/79/2.229688e+001	5/66/1.410938e+001	6/79/2.278125e+001
Ackley/x_Ack20	30	2/42/0	2/42/0	7/99/6.250000e−002	7/97/6.250000e−002
	500	6/79/3.250000e+000	6/77/3.640625e+000	6/79/3.671875e+000	6/77/3.593750e+000
	1,000	5/66/1.354688e+001	5/63/1.443750e+001	5/65/1.423438e+001	5/66/1.429688e+001
Ackley/x_Ack30	30	9/126/0	5/67/0	9/126/6.250000e−002	6/83/0
	500	6/88/3.500000e+000	4/50/1.187500e+000	6/88/3.437500e+000	6/78/2.828125e+000
	1,000	4/53/7.531250e+000	4/51/7.671875e+000	7/95/3.085938e+001	6/77/2.229688e+001
Ackley/x_Ack40	30	4/56/6.250000e−002	4/57/6.250000e−002	8/108/0	7/92/4.687500e−002
	500	4/55/1.343750e+000	4/54/1.015625e+000	7/98/4.062500e+000	7/92/4.562500e+000
	1,000	6/84/2.232813e+001	6/79/2.256250e+001	6/84/2.310938e+001	6/77/2.254688e+001
Total CPU Time		516.1562	161.5781	115.0938	115.0156

However, the effectiveness of one algorithm compared another algorithm cannot be determined based on the number of problems that it solves better. The ‘no free lunch’ theorem (see [61]) states that provided we compare two searching algorithms with all possible functions, the performance of any two algorithms will be, on average, the same. As a result, attempting to find a perfect test set where all the functions are present to determine whether an algorithm is better than another algorithm for every function is a fruitless task. Therefore, when an algorithm is evaluated, we identify the types of problems where its performance is good to characterize the types of problems for which the algorithm is suitable. The authors previously studied functions to be optimized to construct a test set with a better selection of fewer functions (see [62, 63]). This enables us to draw conclusions about the performance of the algorithm depending on the type of function.

The above benchmark problems and the discussions of the choice of test problems for an algorithm can be found at Many scholars use these problems to test numerical optimization methods (see [64, 65] etc.). Based on the above discussions, in this subsection, we test the four algorithms on the Benchmark problems. The test results are presented in Table 3, where denotes the initial point, , , , , , , , , , , , , , , , , , , , and .

http://www.cs.cmu.edu/afs/cs/project/jair/pub/volume24/ortizboyer05a-html/node6.html.

The numerical results in Table 3 show that the proposed algorithm performs the best among the four methods. The total cpu time of the proposed algorithm is the shortest. BFGS-Non performs better than BFGS-WP and BFGS-WP-Zhang, which is consistent with the results of [51]. Additionally, BFGS-WP-Zhang performs better than BFGS-WP, which is consistent with the results of [42]. To directly illustrate the performances of these four methods, we also use the tool of Dolan and Moré [58] to analyze the results with respect to NI and NFG in Table 3. Figures 4 and 5 show their performances.

Figure 4

Performance profiles of these methods (NI).

Figure 5

Performance profiles of these methods (NFG).

Figure 4 indicates that BFGS-WP can solve approximately 93% of the test problems and that the other three methods can solve all the problems. The proposed algorithm solves the problems in the shortest amount of time.

The performance in Figure 5 is similar to that in Figure 4. BFGS-WP can solve approximately 95% of the test problems, while the other methods can solve all the problems.

According to these two figures, the proposed algorithm has the best performance among these four methods, and the BFGS-WP performs the worst. In summary, based on the numerical results of the [57] and benchmark problems, the GLL nonmonotone line search with quasi-Newton update is more effective than the normal WWP line search with quasi-Newton update, which is consistent with the results of [47, 51]. Moreover, these numerical results indicate that the modified BFGS equation (1.10) is better than the normal BFGS update, which is consistent with the results of [42]. Furthermore, the proposed algorithm is competitive with the related methods.

Conclusion

This paper conducts a further study of the modified BFGS update formula in [43]. The main contribution is the global convergence and superlinear convergence for generally convex functions. The numerical results show that the proposed method is competitive with other quasi-Newton methods for the test problems.

In contrast to [42] and [43], this paper achieves both superlinear and global convergence. Moreover, the convergence is obtained for generally convex functions, whereas the other two papers only obtained convergence for uniformly convex functions. The conditions of this paper are weaker than those of the previous research.

For further research, we should study the performance of the new algorithm under different stop rules and in different testing environments (such as [66]). Moreover, more numerical experiments for large practical problems should be performed in the future.